protected macro vs protected luacall
Dear list, I have noticed a somewhat strange difference between protected macros and protected luacalls, namely that the latter cannot be expanded by \the. In the example below I define a Lua function that prints \numexpr 17\relax to the token stream and I call it once from a protected TeX macro wrapping \directlua and once from a protected luacall registered with token.set_lua. \directlua{ userdata = userdata or {} function userdata.test() tex.print("\string\\numexpr 17\string\\relax") end } \tt \protected\def\test{\directlua{userdata.test()}} \meaning\test\par \edef\x{\test}\meaning\x\par \the\test \directlua{ local t = lua.get_functions_table() t[\string#t + 1] = userdata.test token.set_lua("test", userdata.test, \string#t, "protected") } \meaning\test\par \edef\x{\test}\meaning\x\par \the\test \bye However, the second \the\test fails with the following error: ! You can't use `luacall 0' after \the. l.21 \the\test Is this asymmetry between luacalls and regular macros intended? Is there a workaround where I can have a luacall that doesn't expand inside of \edef but still expands after \the? Cheers, Henri
On 1/26/2023 9:33 PM, Henri Menke via ntg-context wrote:
\directlua{ userdata = userdata or {} function userdata.test() tex.print("\string\\numexpr 17\string\\relax") end }
\tt \protected\def\test{\directlua{userdata.test()}} \meaning\test\par \edef\x{\test}\meaning\x\par \the\test
\directlua{ local t = lua.get_functions_table() t[\string#t + 1] = userdata.test token.set_lua("test", userdata.test, \string#t, "protected") } \meaning\test\par \edef\x{\test}\meaning\x\par \the\test from the code it looks like we're talking plain luatex ...
\the is very selective and doesn't work for all primitives or macros compare it to \number which accepts any number representation following it while \the doesn't work ok 123 that said, your set_lua is wrong and passes a function as well as an index \directlua{ lua.get_functions_table()[999] = function() tex.print("\string\\numexpr 17\string\\relax") end token.set_lua("test", 999) } \the \test % needs the \numexpr \number \test % doesn't \count0 \test % needs the \numexpr \count0=\test % doesn't etc Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Thu, 2023-01-26 at 23:07 +0100, Hans Hagen via ntg-context wrote:
On 1/26/2023 9:33 PM, Henri Menke via ntg-context wrote:
\directlua{ userdata = userdata or {} function userdata.test() tex.print("\string\\numexpr 17\string\\relax") end }
\tt \protected\def\test{\directlua{userdata.test()}} \meaning\test\par \edef\x{\test}\meaning\x\par \the\test
\directlua{ local t = lua.get_functions_table() t[\string#t + 1] = userdata.test token.set_lua("test", userdata.test, \string#t, "protected") } \meaning\test\par \edef\x{\test}\meaning\x\par \the\test from the code it looks like we're talking plain luatex ...
\the is very selective and doesn't work for all primitives or macros
compare it to \number which accepts any number representation following it while \the doesn't work ok 123
that said, your set_lua is wrong and passes a function as well as an index
\directlua{ lua.get_functions_table()[999] = function() tex.print("\string\\numexpr 17\string\\relax") end token.set_lua("test", 999) }
\the \test % needs the \numexpr
\number \test % doesn't
\count0 \test % needs the \numexpr
\count0=\test % doesn't
etc
Thanks for pointing out my mistake. Your example works fine but now the luacall is no longer protected and will therefore fully expand in \edef. So do I understand this correctly that there is no way to make the second \the\test work? \directlua{ lua.get_functions_table()[999] = function() tex.pri nt("\string\\numexpr 17\string\\relax") end } \tt \directlua{token.set_lua("test", 999)} \meaning\test\par % expandable luacall 999 \edef\x{\test}\meaning\x\par % macro:->\numexpr 17\relax \the\test % 17 \directlua{token.set_lua("test", 999, "protected")} \meaning\test\par % luacall 999 \edef\x{\test}\meaning\x\par % macro:->\test \the\test % ! You can't use `luacall 999' after \the. \bye Cheers, Henri
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
_____________________________________________________________________ ______________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context webpage : https://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : https://contextgarden.net _____________________________________________________________________ ______________
On 1/27/2023 10:13 AM, Henri Menke via ntg-context wrote:
On Thu, 2023-01-26 at 23:07 +0100, Hans Hagen via ntg-context wrote:
On 1/26/2023 9:33 PM, Henri Menke via ntg-context wrote:
\directlua{ userdata = userdata or {} function userdata.test() tex.print("\string\\numexpr 17\string\\relax") end }
\tt \protected\def\test{\directlua{userdata.test()}} \meaning\test\par \edef\x{\test}\meaning\x\par \the\test
\directlua{ local t = lua.get_functions_table() t[\string#t + 1] = userdata.test token.set_lua("test", userdata.test, \string#t, "protected") } \meaning\test\par \edef\x{\test}\meaning\x\par \the\test from the code it looks like we're talking plain luatex ...
\the is very selective and doesn't work for all primitives or macros
compare it to \number which accepts any number representation following it while \the doesn't work ok 123
that said, your set_lua is wrong and passes a function as well as an index
\directlua{ lua.get_functions_table()[999] = function() tex.print("\string\\numexpr 17\string\\relax") end token.set_lua("test", 999) }
\the \test % needs the \numexpr
\number \test % doesn't
\count0 \test % needs the \numexpr
\count0=\test % doesn't
etc
Thanks for pointing out my mistake. Your example works fine but now the luacall is no longer protected and will therefore fully expand in \edef. So do I understand this correctly that there is no way to make the second \the\test work?
\directlua{ lua.get_functions_table()[999] = function() tex.pri nt("\string\\numexpr 17\string\\relax") end }
\tt \directlua{token.set_lua("test", 999)} \meaning\test\par % expandable luacall 999 \edef\x{\test}\meaning\x\par % macro:->\numexpr 17\relax \the\test % 17
\directlua{token.set_lua("test", 999, "protected")} \meaning\test\par % luacall 999 \edef\x{\test}\meaning\x\par % macro:->\test \the\test % ! You can't use `luacall 999' after \the.
\bye you can do
\directlua{ function TestCmd() tex.print("\string\\numexpr 17\string\\relax") end } \protected\def\test{\directlua{TestCmd()}} or, more efficient: \directlua{ function TestCmd() tex.print("17") end } \protected\def\test{\numexpr\directlua{TestCmd()}\relax} Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Fri, 2023-01-27 at 10:30 +0100, Hans Hagen via ntg-context wrote:
On 1/27/2023 10:13 AM, Henri Menke via ntg-context wrote:
On Thu, 2023-01-26 at 23:07 +0100, Hans Hagen via ntg-context wrote:
On 1/26/2023 9:33 PM, Henri Menke via ntg-context wrote:
\directlua{ userdata = userdata or {} function userdata.test() tex.print("\string\\numexpr 17\string\\relax") end }
\tt \protected\def\test{\directlua{userdata.test()}} \meaning\test\par \edef\x{\test}\meaning\x\par \the\test
\directlua{ local t = lua.get_functions_table() t[\string#t + 1] = userdata.test token.set_lua("test", userdata.test, \string#t, "protected") } \meaning\test\par \edef\x{\test}\meaning\x\par \the\test from the code it looks like we're talking plain luatex ...
\the is very selective and doesn't work for all primitives or macros
compare it to \number which accepts any number representation following it while \the doesn't work ok 123
that said, your set_lua is wrong and passes a function as well as an index
\directlua{ lua.get_functions_table()[999] = function() tex.print("\string\\numexpr 17\string\\relax") end token.set_lua("test", 999) }
\the \test % needs the \numexpr
\number \test % doesn't
\count0 \test % needs the \numexpr
\count0=\test % doesn't
etc
Thanks for pointing out my mistake. Your example works fine but now the luacall is no longer protected and will therefore fully expand in \edef. So do I understand this correctly that there is no way to make the second \the\test work?
\directlua{ lua.get_functions_table()[999] = function() tex.pri nt("\string\\numexpr 17\string\\relax") end }
\tt \directlua{token.set_lua("test", 999)} \meaning\test\par % expandable luacall 999 \edef\x{\test}\meaning\x\par % macro:->\numexpr 17\relax \the\test % 17
\directlua{token.set_lua("test", 999, "protected")} \meaning\test\par % luacall 999 \edef\x{\test}\meaning\x\par % macro:->\test \the\test % ! You can't use `luacall 999' after \the.
\bye you can do
\directlua{ function TestCmd() tex.print("\string\\numexpr 17\string\\relax") end }
\protected\def\test{\directlua{TestCmd()}}
or, more efficient:
\directlua{ function TestCmd() tex.print("17") end }
\protected\def\test{\numexpr\directlua{TestCmd()}\relax}
Thanks, this is indeed the workaround that I currently use. However, the downside is that this has to tokenize the contents of \directlua every time (and is therefore susceptible to surrounding \catcode shenanigans) and it expands in at minimum two steps, whereas a luacall always expands in a single step. But I guess that's what I will have to live with. Cheers, Henri
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
_____________________________________________________________________ ______________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context webpage : https://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : https://contextgarden.net _____________________________________________________________________ ______________
On 1/27/2023 10:35 AM, Henri Menke via ntg-context wrote:
Thanks, this is indeed the workaround that I currently use. However, the downside is that this has to tokenize the contents of \directlua every time (and is therefore susceptible to surrounding \catcode shenanigans) and it expands in at minimum two steps, whereas a luacall always expands in a single step. But I guess that's what I will have to live with. on my 2018 laptop, with
function FoO() end this \directlua{FoO()}} takes 0.0000014 seconds so probably the least of your worries compared to other macro and lua magic Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Fri, 2023-01-27 at 11:01 +0100, Hans Hagen via ntg-context wrote:
On 1/27/2023 10:35 AM, Henri Menke via ntg-context wrote:
Thanks, this is indeed the workaround that I currently use. However, the downside is that this has to tokenize the contents of \directlua every time (and is therefore susceptible to surrounding \catcode shenanigans) and it expands in at minimum two steps, whereas a luacall always expands in a single step. But I guess that's what I will have to live with. on my 2018 laptop, with
function FoO() end
this
\directlua{FoO()}}
takes 0.0000014 seconds so probably the least of your worries compared to other macro and lua magic
True, but then there is still the problem with the number of expansions. With a nested \directlua I always need at least two steps to get the result from Lua. \directlua{ function TestCmd() tex.print("\string\\numexpr 17\string\\relax") end lua.get_functions_table()[999] = TestCmd } \let\:\expandafter % I'm lazy \tt \directlua{token.set_lua("test", 999)} \:\def\:\x\:{\test}\meaning\x % macro:->\numexpr 17\relax \protected\def\test{\directlua{TestCmd()}} \:\def\:\x\:{\test}\meaning\x\par % macro:->\directlua {TestCmd()} \:\:\:\def\:\:\:\x\:\:\:{\test}\meaning\x % macro:->\numexpr 17\relax \bye How much do you think would break if protected luacall were treated like protected macros after \the in the engine? That would probably need an extra branch in the switch(cur_cmd) in scan_something_internal. Currently protected luacall just goes to the default branch (You can't use ...). Cheers, Henri
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
_____________________________________________________________________ ______________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context webpage : https://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : https://contextgarden.net _____________________________________________________________________ ______________
On 1/27/2023 12:03 PM, Henri Menke via ntg-context wrote:
How much do you think would break if protected luacall were treated like protected macros after \the in the engine?
That kind of incompatibility is no option (I'm not going to analyze it for mkiv let along other macro package usage which is out of my scope).
That would probably need an extra branch in the switch(cur_cmd) in scan_something_internal. Currently protected luacall just goes to the default branch (You can't use ...). For various reasons luatex got frozen around version 1 and demand for stability (which also excluded experiments) is one of them. So, one has to work with what one gets (not much different from the rest of tex functionality). With careful coding the performance hit is not that large and machines still get a bit get faster so that compensates it anyway. Although some low level approaches can bring benefits it's never homogenious and largely depends on use cases.
Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
participants (2)
-
Hans Hagen -
Henri Menke