[NTG-context] lmtx goodies

Hans Hagen j.hagen at xs4all.nl
Thu Apr 15 10:26:51 CEST 2021


I'm checking out some old experiments (either use or drop 'm) and here's 
one that made it into lmtx in the end (not much code). We have a bunch 
of arrays in the engine: \uccode, \lccode, \sfcode, \efcode, catcodes 
etc and while traditional tex just has 256 slot arrays in luatex these 
are sparse: it saves memory because often these lists cover a small 
range of unicode.

Because this is kind of a general mechanism, I've added a lua interface 
to 'user sparse arrays' shown in the following examples (they can be 1, 
2 or 4 bytes numbers):

\setuplayout[tight] \setupbodyfont[tt,10pt]


         local d = io.loaddata(resolvers.findfile("tufte.tex"))
         local a = sparse.new()
         for v in string.utfvalues(d) do
          -- sparse.set(a,v,sparse.get(a,v)+1)
             a[v] = a[v] + 1
         for i, v in sparse.traverse(a) do
             if i > 32 and v > 0 then
                 context("%03i %U\\crlf",i,i)


         local a = { }
         local y = string.byte("1")
         local n = string.byte("0")
         for i=1,20 do
             a[i] = sparse.new(1,n) -- bytes (1|2|4) | default
         for i=8,15 do
             local ai = a[i]
             for j=10,15 do
              -- sparse.set(ai,j,y)
                 ai[j] = y
         for i=1,20 do
             a[i] = sparse.concat(a[i],1,20)



     {       \mycode 999 = 123} 000 = \the\mycode999\par
     {\global\mycode 999 = 124} 124 = \the\mycode999\par
     {       \mycode 999 = 125} 124 = \the\mycode999\par


Often using a lua array or hash is more efficient, but when you allocate 
many real large sets the sparse arrays might win. I can add more options 
at the tex end, like wipe, concat, prealloc, save, load, always global 
etc. but that depends on expected usage of which I have no clue (but as 
usual with tex features, at some point they will be used in unexpected 


                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl

More information about the ntg-context mailing list