[Dev-luatex] getnext et al.

Hans Hagen pragma at wxs.nl
Wed May 7 09:54:08 CEST 2014

On 5/7/2014 8:09 AM, Philipp Gesang wrote:
> ···<date: 2014-05-06, Tuesday>···<from: Hans Hagen>···
>> On 5/6/2014 7:39 AM, Philipp Gesang wrote:
>>>> reading the chapter 8.2 in the manual, I wonder how to call getnext and its ilk. Is this used like node.getnext()?
>>> Yes, but they operate on integer references which you obtain by
>>> calling node.direct.todirect() on the node you start out with:
>>       local l = tex.getbox(0).list
>>       print(node.getnext(l),node.direct.getnext(node.direct.todirect(l)))
>> so, node.getnext(n) works on 'normal nodes' (and can be faster than
>> n.next as it saves a metatable lookup + some internal speedup) ... if
>> it's really faster depends on the the use case (and using it also
>> relates to readability ... often n.next looks nicer)
>> never think of direct nodes as integers .. they're abstractions that
>> happen to be (positive) numbers
> Which also means you can do some weird kind of pointer arithmetic
> on them, e.g. to box all the glyph nodes that are among the lower
> 250 elements of the varmem region:
>      \directlua {
>        local hd, cur
>        for i = 1, 250 do
>          local chr = node.direct.getfield (i, "char")
>          if chr then
>            print (">>", i, chr, string.char (chr))
>            local new = node.direct.copy (i)
>            if cur then
>              node.direct.setfield (cur, "next", new)
>              node.direct.setfield (new, "prev", cur)
>            else
>              hd = new
>            end
>            cur = new
>          end
>        end
>        if hd then
>          local hbox = node.direct.hpack (hd)
>          node.direct.write (hbox)
>        end
>      }
> I can’t think of a serious use for that property, though,
> especially since there doesn’t appear to be a way to query the
> number of nodes allocated.

Indeed, so that's why the node (not direct) interface is the official 
user one and the direct subvariant for special cases ... in fact, those 
predefined ranges are not even official. It comes close to finding 
security holes and exploiting them. (One can equally well abuse the 
regular node interface.)

>>> @Hans: Considering that node.direct has been part of the master
>>> branch for some time and the new model is used in Context without
>>> users reporting segfaults, could you give an ETA on when the
>>> generic fontloader will benefit from it?
>> hard to say ... keep in mind that the gain is mostly in complex font
>> handling of manipulations and that is not always that generic (
>> btw, there are some more changes in the font related code (on my
>> machine, one of these days in the core ... given some testing by others)
>> ... one reason for keeping this separate from generic is that i don't
>> want to break that (esp not around tex live code freeze)
> I seem to recall that the generic font loader did actually ship
> with the nuts code for a short while until you undid that and
> reverted it to the previous implementation. Most of the bugs that
> occurred since were consequences of the code being out of sync
> with the rest of Context -- just wanted to point that out ;)

I cannot imagine that the official loader had nuts code (at least not 
intentionally) because it relies on some nut helpers and also assumes 
some consistency with other code... you may refer to the time when there 
was a hybrid i.e. function calls mimicking the direct interface that I 
experimented before nuts (which actually runs a few pct slower on some 

Concerning bugs: one reason to keep generic as a copy is that i don't 
want experimental new code (not that much btw the last year) to 
influence the generic code. Context users are normally willing to cq. 
can easily update often as we have the efficient garden sync, so there 
we can experiment more ... if the font code is going to change 
dramatically (unlikely) then for sure it will be in a private copy first.

Ok, there will be font-inj.lua soon after the tl code freeze (as drop in 
for node-inj.lua) but that one will be tagged 'experimental' for a while 
as I don't want to be the target of 'everything fails' mails. It works 
okay on my machine for a while so it's not untested.

Most of the direct code was tested on my and Luigi's machine for half a 
year (so I had to keep some 60 files in sync on my machine). The generic 
copy was therefore just the same one as the context one of the day 
before we made the switch.

>> the latest generic code + the latest luatex already should run somewhat
>> faster (something halfway) as the regular node code was also optimized
> Performance reached new heights during the last year, so much is
> certain.

You're joking now yes? The speedup was not that dramatic (unless you 
changed something at your end). It's not like old-time computers running 
twice as fast each year, Btw, this is one of the reasons for the 
speedups. I don't expect cpu's to become much faster.

The only thing about speed that I can say (as I only run context) is 
that on the average context has become faster but that's not due to font 
code which is accounts for only a relatively small part of the run time.


ps. At bachotex some folks told me that the perception is that luatex is 
slow. Apart from the expected lower speed due to 32 bit character 
handling and wide fonts, my impression is that (at least for context) we 
now run quite okay. Given that we do more complex things (not talking 
fonts now) mkiv runs faster than mkii or at least comes pretty close. My 
personal aim is that a edit-test cycle of a couple of pages should stay 
below .5 sec (all included). When I get 20-50 pages per second on 
average complex documents on my current laptop I'm fine.

ps. I just read in tugboat that the texbook on DEK's 3.2G xeon driven 
machine needs .3 seconds which is on such hardware twice the time 
context needs to produce a zero page document.

                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl

More information about the dev-luatex mailing list