Snooping around in LMTX: Questions about Lua global variables
Hello again, While learning about how to drive TeX through Lua, I decided to recursively list all Lua global variables (actually this is traversing the _G table) in the LMTX environment, half to learn more Lua, half for getting to know ConTeXt better. I was quite surprised by the huge size of the environment, a file that contains the listing of all the globals is 42 MB long! I wonder if it would be possible to reduce the exposed globals by replacing some of them by getter and setter-like functions? That seems like it would be much nicer and less error prone - because Lua is so dynamic, polluting the global namespace seems even more dangerous than in C-like languages. I guess it could also be more performant, because Lua would conceivably spend less time managing huge tables. There were also some global variables with suspicious random variation in values between runs of ConTeXt: I ran my Lua script like so multiple times (attached, in case someone is interested in it): context s.lua rm s.tuc s.pdf s.log And I found that the values of some variables unpredictably and randomly vary: The variable resolvers.suffixmap.lua sometimes has the value "scripts", and sometimes "lua". I think this means that files with the file name suffix ".lua" are sometimes classified as general scripts and sometimes as Lua scripts. This seems like it could even be a bug? The variables storage.tofmodules and storage.toftables are also interesting: they vary from run to run like this: tofmodules: 0.175483 0.149536 0.150493 0.150005 toftables: 0.008407 0.008118 0.008395 0.008116 I'd like to know what is their purpose, if it's not to involved to explain? Thanks, Neven
Oops, I forgot to attach the scriptlet before.
[...] I guess it could also be more performant, because Lua would conceivably spend less time managing huge tables.
Now that I think about this some more, it doesn't actually make sense. However I'm still interested in whether it is really necessary to have that many globals exposed. Thanks, Neven
On 12/22/2020 6:57 PM, Neven Sajko wrote:
Oops, I forgot to attach the scriptlet before.
[...] I guess it could also be more performant, because Lua would conceivably spend less time managing huge tables.
Now that I think about this some more, it doesn't actually make sense. However I'm still interested in whether it is really necessary to have that many globals exposed. most of what you see in that generated file is either unicode data or font resources ... all needed (and geared for performance)
Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Tue, 22 Dec 2020 at 18:15, Hans Hagen
On 12/22/2020 6:57 PM, Neven Sajko wrote:
Oops, I forgot to attach the scriptlet before.
[...] I guess it could also be more performant, because Lua would conceivably spend less time managing huge tables.
Now that I think about this some more, it doesn't actually make sense. However I'm still interested in whether it is really necessary to have that many globals exposed. most of what you see in that generated file is either unicode data or font resources ... all needed (and geared for performance)
OK, now that I think just about the real global variables (instead of the "recursive" globals): would it make sense to transfer all the non-Lua-default globals into two tables, one for Lua(Meta)Tex, and another table for ConTeXt, so those would be the only two additional global variables? I'm not proposing you do it, since it seems like it could be a lot of work, I'm just wondering what you think about that, because it seems like things would be much tidier like that (less chance of accidentally accessing a global in Lua code, etc.). Thanks, Neven
On 12/22/2020 7:32 PM, Neven Sajko wrote:
On Tue, 22 Dec 2020 at 18:15, Hans Hagen
wrote: On 12/22/2020 6:57 PM, Neven Sajko wrote:
Oops, I forgot to attach the scriptlet before.
[...] I guess it could also be more performant, because Lua would conceivably spend less time managing huge tables.
Now that I think about this some more, it doesn't actually make sense. However I'm still interested in whether it is really necessary to have that many globals exposed. most of what you see in that generated file is either unicode data or font resources ... all needed (and geared for performance)
OK, now that I think just about the real global variables (instead of the "recursive" globals): would it make sense to transfer all the non-Lua-default globals into two tables, one for Lua(Meta)Tex, and another table for ConTeXt, so those would be the only two additional global variables?
well, and then if you load some library that would add some global again ... so it's a chicken egg problem.
I'm not proposing you do it, since it seems like it could be a lot of work, I'm just wondering what you think about that, because it seems like things would be much tidier like that (less chance of accidentally accessing a global in Lua code, etc.). You can create your own instance:
\definenamedlua[mylua] \startmyluacode global.context("USER 1") context.par() context("USER 2") context.par() if characters then context("ACCESS directly") elseif global.characters then context("ACCESS via global") else context("NO ACCESS at all") end context.par() if bogus then context("ACCESS directly") elseif global.bogus then context("ACCESS via global") else context("NO ACCESS at all") end context.par() \stopmyluacode I admit that I never run into conflicts. Ok, I never load libraries, mostly because we havne plenty of helpers on board. Changing the approach now would invalidate a lot of user code (and also mean a lot of work plus probably make us run into the > 200 locals issue due to aliasing). Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 12/22/2020 6:36 PM, Neven Sajko wrote:
Hello again,
While learning about how to drive TeX through Lua, I decided to recursively list all Lua global variables (actually this is traversing the _G table) in the LMTX environment, half to learn more Lua, half for getting to know ConTeXt better.
I was quite surprised by the huge size of the environment, a file that contains the listing of all the globals is 42 MB long! I wonder if it would be possible to reduce the exposed globals by replacing some of them by getter and setter-like functions? That seems like it would be
not sure what you refer to but even then you need to 'get' and 'set them someplace which then involves tables ... it's just a large system and that won't change also, when you run such tests, don't include the characters.* tables as most is data, and of you do it from a tex run you also see fonts and their data (which then means shared tables too)
much nicer and less error prone - because Lua is so dynamic, polluting the global namespace seems even more dangerous than in C-like languages. I guess it could also be more performant, because Lua would conceivably spend less time managing huge tables.
Wait, you traverse global tablers so their entries are *not* global. \starttext \startluacode context.starttabulate { "|T|T|" } for k, v in table.sortedhash(_G) do context.NC() context(type(v)) context.NC() context(k) context.NC() context.NR() end context.stoptabulate() \stopluacode \stoptext These are global. Many come from lua itself, then there are libraries than come with luametatex. The rest is context specific and again some are just helper modules. I notices some 6 stray locals that I fixed.
There were also some global variables with suspicious random variation in values between runs of ConTeXt: I ran my Lua script like so multiple times (attached, in case someone is interested in it):
context s.lua rm s.tuc s.pdf s.log
And I found that the values of some variables unpredictably and randomly vary:
Maybe weak tables?
The variable resolvers.suffixmap.lua sometimes has the value "scripts", and sometimes "lua". I think this means that files with the file name suffix ".lua" are sometimes classified as general scripts and sometimes as Lua scripts. This seems like it could even be a bug?
could be but probably more a side effect ... part of that resolver stuff is there for usage in tds and could be simplified in the meantime .. if there are hashes they can differ per document
The variables storage.tofmodules and storage.toftables are also interesting: they vary from run to run like this:
tofmodules: 0.175483 0.149536 0.150493 0.150005
toftables: 0.008407 0.008118 0.008395 0.008116
I'd like to know what is their purpose, if it's not to involved to explain? timers, so indeed they can differ per run
Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 12/22/2020 7:13 PM, Neven Sajko wrote:
Thank you very much for your answers!
you can run: s-inf-01.mkiv s-inf-03.mkiv s-inf-05.mkiv i need to update them but they show the picture Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
participants (2)
-
Hans Hagen
-
Neven Sajko