Hello,
take the following PlainTeX example (as always ;-):
%&luatex
\catcode`@=11
\def\log#1{\immediate\write\sixt@@n{#1}}
\let~\space
\tracingassigns1
\tracingrestores1
\tracingonline1
\count@=5
\log{Before:~~~~~~~~~~~~~~~~~\the\count@}
\begingroup
\directlua0{tex.count["count@"]=23}
\log{In group:~~~~~~~~~~~~~~~\the\count@}
\endgroup
\log{After \string\endgroup:~~~~~~~~\the\count@}
\bye
This results in:
This is LuaTeX, Version snapshot-0.25.3-2008051713 (Web2C 7.5.6)
(Assignments.tex{into \tracingonline=1}
{changing \count255=92}
{into \count255=5}
Before: 5
{reassigning [no_local_whatsits]=0}
{reassigning [no_local_dirs]=0}
{changing \count255=5}
{into \count255=23}
In group: 23
{restoring \count255=5}
After \endgroup: 5
)
(see the transcript file for additional information)
No pages of output.
Transcript written on Assignments.log.
Now, a few months and betas ago, assignments made inside Lua to
tex.count et cetera were always global (see my posting on August 14th
2007 on this mailing list). Now it seems that they respect grouping.
Is this meant to stay or is it an unintended side-effect of some other
change? Looking into the source ('set_tex_count_register' in
luatex.web), it seems that this change is deliberate.
If it is a feature (which I'd be very grateful for, since it would
simplify a lot of things), how do I perform \global assignments in
Lua? As far as I can see from the source, I cannot pass an optional
third parameter to tex.setcount:
tex.setcount(
Jonathan Sauer wrote:
Now, a few months and betas ago, assignments made inside Lua to tex.count et cetera were always global (see my posting on August 14th 2007 on this mailing list).
Actually that behaviour was a bug, not intended, not consistent. I changed it a little while after your posting so that is now obeys grouping. The rationale for picking grouped instead of global was that you could temporarily set tex.globaldefs=1 if you really needed global (note: that does not actually work either) The issue I had was trying to find a nice solution for tex.count[0] = 5 Now I just realized (this morning) that I could create a second virtual array named tex.globalcount[0] = 5 or some such. Your tex.setcount (0,5,true) proposal is fine, but the array access is so neat that I would like to keep it.
I would try my hand at a patch to implement this, but unfortunately, I cannot find the function prototype for set_tex_count_register. A
It is in luatex.web, just below get_tex_count_register.
sketch:
- Modify
et.al. so that 'a' is no longer a constant local variable, but a parameter.
- In ltexlib.c, check for an optional boolean using
, then pass either zero (local/false) or four (global/true) as the third parameter to et.al.
- Add a call to @
in et.al. so that \globaldefs works correctly.
Sounds fine.
I'm not sure about the last point, though, since it would mean that TeX would influence Lua's behaviour.
It was supposed to, I simply forgot. Your web implementation proposal sounds fine, but before we do anything about extending the lua syntax, we have to think about those array accesses and the other tex internals, I think. The interface to the tex internals is still incomplete, and that is mostly because of syntax issues. It is fun that we can write tex.globalsdefs = 1 as an assignment instead of as a function call, and tex.count[0] = 5 as well, but many internal parameters are not interfaced yet because they are not so simple. The tricky bit is that I fear that some of the internals really require a functioncall instead of assignment, for example \parshape and \fontdimen.
Another thing: Setting an attribute using tex.attribute does not seem to add the attribute to nodes, contrary to setting it in TeX. Most ... And while we're at it: In luatex.wb line 27777,
help1("I'll pretend you didn't say \long or \outer or \global.");
Thanks, I will apply thse right away. Best wishes, Taco
Hello,
The issue I had was trying to find a nice solution for
tex.count[0] = 5
Now I just realized (this morning) that I could create a second virtual array named
tex.globalcount[0] = 5
or some such.
Maybe tex.count.local[0] and tex.count.global[0]?
Your
tex.setcount (0,5,true)
proposal is fine, but the array access is so neat that I would like to keep it.
I agree.
I would try my hand at a patch to implement this, but unfortunately, I cannot find the function prototype for set_tex_count_register. A
It is in luatex.web, just below get_tex_count_register.
But that is the implementation. I meant the function prototype used by the C code to know how the WEB function is called (parameter types etc.).
Your web implementation proposal sounds fine, but before we do anything about extending the lua syntax, we have to think about those array accesses and the other tex internals, I think.
The interface to the tex internals is still incomplete, and that is mostly because of syntax issues. It is fun that we can write
tex.globalsdefs = 1
as an assignment instead of as a function call, and
tex.count[0] = 5
as well, but many internal parameters are not interfaced yet because they are not so simple. The tricky bit is that I fear that some of the internals really require a functioncall instead of assignment, for example \parshape and \fontdimen.
Maybe tex.parshape = { { "0pt", "30pt" }, { "10pt", "20pt" } } This would result in \parshape=2 0pt 30pt 10pt 20pt Read access would return 2 (the number of lines). Of course, this would require temporary tables, so an alternative syntax might indeed be a function call (the number of lines is calculated from the number of parameters): tex.parshape("0pt", "30pt", "10pt", "20pt") Maybe there is a way to implement both syntaxes, although I'm currently pessimistic: tex:__index("parshape") could return a function which sets \parshape (leaving tex:__newindex("parshape", ...) to perform the table assignment), but then read access would also return this function, not the number of lines. Maybe it would be better to use tex.setparshape for the function call. fontdimen could be a two-dimensional array, with the first dimension being the font and the second the dimension to set: fontdimen[1][1] = 0 fontdimen[2] = { 0, 1, 2, 3 } Access to TeX's control sequences might be difficult without functions, though. OTOH: tex.cs.from = { "#1 to #2", "From #1 to #2" } being equivalent to \def\from#1 to #2{From #1 to #2} The first element of the array could be an optional bitfield to specify \long, \outer etc: tex.cs.from = { 1 + 2, "#1 to #2", "From #1 to #2" } i.e. being equivalent to (1 = \long, 2 = \protected) \long\protected\def\from#1 to #2{From #1 to #2} Now, catcode tables ... maybe: tex.cs.from = { 1 + 2, "#1 to #2", { 1, "From #1 to #2" } } Still, this would be a nightmare to use. Maybe use factory functions: tex.cs.from = tex.macro.new("#1 to #2", "From #1 to #2") tex.cs.from = tex.macro.newL("#1 to #2", "From #1 to #2") ... tex.cs.from = tex.macro.newLP("#1 to #2", "From #1 to #2", 1) The latter would create a \long \protected macro using catcode table 1. This scheme would require 8 new variations for \long, \protected and \outer (\global is a matter of assignment and should be handled by the tex.cs-table the macro is assigned to). Or: tex.cs.from = tex.new.macro... tex.cs.from = tex.new.macroL... ... This could be extended to: tex.parshape = tex.new.parshape(...) While we're at it: Maybe dimensions and glue could get their own data type (implemented as a user data object), so that calculations etc. are easier to perform in Lua: local d = dimen.new(12, "pt") -- 12pt local g1 = glue.new(12, "pt") -- 12pt plus 0pt minus 0pt local g2 = glue.new(12, "fill", "pt") -- 12pt plus 1fill minus 0pt local g3 = glue.new(12, "2fill", 10, "pt") -- 12pt plus 2fill minus 10pt
Best wishes, Taco
Jonathan
Jonathan Sauer wrote:
tex.parshape = { { "0pt", "30pt" }, { "10pt", "20pt" } }
This would result in
\parshape=2 0pt 30pt 10pt 20pt
or just tex.parshape = { "0pt", "30pt", "10pt", "20pt" } after all it's just a variant on tex; a more extensive parshaper could have named keys in subtables
Maybe there is a way to implement both syntaxes, although I'm currently
we should avoid too much overhead (for several other mechanisms we rejected former ideas of multiple interfaces and went for simplicity) user scan always build a shell around it
fontdimen could be a two-dimensional array, with the first dimension being the font and the second the dimension to set:
fontdimen[1][1] = 0 fontdimen[2] = { 0, 1, 2, 3 }
fontdimens are dealt with in the font array; maybe we don't need that kind of access then
Access to TeX's control sequences might be difficult without functions, though. OTOH:
tex.cs.from = { "#1 to #2", "From #1 to #2" }
being equivalent to
\def\from#1 to #2{From #1 to #2}
brrr, do we really need to do that at the lua level? one can just pipr definitions to tex; extending the tex syntax is a real long term objective (opening up has more priority) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hello,
tex.parshape = { "0pt", "30pt", "10pt", "20pt" }
after all it's just a variant on tex; a more extensive parshaper could have named keys in subtables
I agree.
Access to TeX's control sequences might be difficult without functions, though. [...]
brrr, do we really need to do that at the lua level? one can just pipr
definitions to tex; extending the tex syntax is a real long term objective (opening up has more priority)
Are there any ideas on how to extend TeX's syntax? More than nine parameters (if that is useful)? Parameter matching using regular expressions or Lua patterns? Optional parameters without having to use \futurelet (expandable, no fixed catcodes)?
Hans
Jonathan
Jonathan Sauer wrote:
Are there any ideas on how to extend TeX's syntax? More than nine parameters (if that is useful)? Parameter matching using regular expressions or Lua patterns? Optional parameters without having to use \futurelet (expandable, no fixed catcodes)?
As Hans said, this is low priority, but it would be simple to map simple lua functions to simple tex csnames. Once inside a lua function, you could then call token.get_next() and possibly some other helpers to do more advanced stuff. There could well be a frontend to fetch the wanted arguments using something like a (fictional) token.match function. We have not actually given this any thought yet, and the most important thing is therefore not coding, but coming up with a nice interface. Suggestions in that area are certainly welcome. Best wishes, Taco
Hello,
Are there any ideas on how to extend TeX's syntax? More than nine parameters (if that is useful)? Parameter matching using regular expressions or Lua patterns? Optional parameters without having to use \futurelet (expandable, no fixed catcodes)?
As Hans said, this is low priority, but it would be simple to map simple lua functions to simple tex csnames.
Expanding on that: TeX's csnames could be represented in Lua as the table "tex.cs"; each entry either contains a TeX macro (a userdata type in Lua), a TeX primitive or a Lua function. "Calling" the cs inside TeX would then check its type and perform the appropriate action. That would do away with the need for a glue macro which is currently required for calling Lua functions from TeX (using \directlua).
Once inside a lua function, you could then call token.get_next() and possibly some other helpers to do more advanced stuff. There could well be a frontend to fetch the wanted arguments using something like a (fictional) token.match function.
Maybe: local optional if token.isnext("[") then optional = token.match("[", "]") else optional = "default value" end where token.match correctly handles characters with catcodes 1 and 2 (most likely braces).
We have not actually given this any thought yet, and the most important thing is therefore not coding, but coming up with a nice interface. Suggestions in that area are certainly welcome.
There was a paper a few years back on extending TeX that discussed this subject. I do not recall the name of the paper anymore, most likely it was a Tugboat article. I will look for it at home and post it if anyone is interested.
Best wishes, Taco
Jonathan
Jonathan Sauer wrote:
Are there any ideas on how to extend TeX's syntax? More than nine parameters (if that is useful)? Parameter matching using regular expressions or Lua patterns? Optional parameters without having to use \futurelet (expandable, no fixed catcodes)?
the main short term objectives are (1) opening up the existing functionality (2) opentype (mostly done) (3) unicode (done) since luatex is supposed to replace pdftex, we don't mess around too much with the basics extending the syntax is not a priority; in principle one can write another input language, map the parsed result onto primitives or lua functions etc etc maybe a few etex like thingies will show up after all, when you want a different syntax you can use lua to parse your input (e.g. preprocess the file and then pipe it into the existing machinery) if we start looking into that direction, we need a pretty stable luatex i.e. 1.0+ and that will take a few years Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hello,
extending the syntax is not a priority; in principle one can write another input language, map the parsed result onto primitives or lua functions etc etc
That's what I'm doing right now: I have extended the use of "#" inside the macro body by transforming the macro body to Lua code using lpeg, compiling this Lua code into a function and calling the function using a glue macro. Example: \deflua\repeat#1#2{% #{for i = 1, tonumber(p1) do}% #(i): #2% #{end}% } This repeats #2 #1 times and prefixes it with the iteration's number. It is transformed to (more or less) the following Lua function: function repeat(p1, p2) for i = 1, tonumber(p1) do tex.sprint(tostring(i), tostring(p2)) end end The macro \repeat then is: \repeat#1#2{% \directlua0{repeat('\luaescapestring{\unexpanded{#1}}',% '\luaescapestring{\unexpanded{#2}}')}% } Of course, this is slower than a traditional TeX macro.
Hans
Jonathan
Jonathan Sauer wrote:
Maybe tex.count.local[0] and tex.count.global[0]?
that would make the interface too complex (we should avoind too many subtables, if only because of speed), also if someone redefines \global to be a counter of defined \local to be one ... tex.count['local'] is valid and catching tex.count['local'][0] as a special case is messy Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Jonathan Sauer wrote:
But that is the implementation. I meant the function prototype used by the C code to know how the WEB function is called (parameter types etc.).
That is autogenerated by web2c. You only have to change the web. I'll respond to the rest of your message later. Best wishes, Taco
participants (3)
-
Hans Hagen -
Jonathan Sauer -
Taco Hoekwater