[Dev-luatex] getnext et al.

Philipp Gesang philipp.gesang at alumni.uni-heidelberg.de
Wed May 7 20:14:11 CEST 2014

···<date: 2014-05-07, Wednesday>···<from: Hans Hagen>···

> On 5/7/2014 8:09 AM, Philipp Gesang wrote:
> > ···<date: 2014-05-06, Tuesday>···<from: Hans Hagen>···
> >> On 5/6/2014 7:39 AM, Philipp Gesang wrote:
> >>>> reading the chapter 8.2 in the manual, I wonder how to call getnext and its ilk. Is this used like node.getnext()?
> >>>
> >>> Yes, but they operate on integer references which you obtain by
> >>> calling node.direct.todirect() on the node you start out with:
> >>
> >>       local l = tex.getbox(0).list
> >>       print(node.getnext(l),node.direct.getnext(node.direct.todirect(l)))
> >>
> >> so, node.getnext(n) works on 'normal nodes' (and can be faster than
> >> n.next as it saves a metatable lookup + some internal speedup) ... if
> >> it's really faster depends on the the use case (and using it also
> >> relates to readability ... often n.next looks nicer)
> >>
> >> never think of direct nodes as integers .. they're abstractions that
> >> happen to be (positive) numbers
> >
> > Which also means you can do some weird kind of pointer arithmetic
> > on them, e.g. to box all the glyph nodes that are among the lower
> > 250 elements of the varmem region:
> >
> >      \directlua {
> >        local hd, cur
> >        for i = 1, 250 do
> >          local chr = node.direct.getfield (i, "char")
> >          if chr then
> >            print (">>", i, chr, string.char (chr))
> >            local new = node.direct.copy (i)
> >            if cur then
> >              node.direct.setfield (cur, "next", new)
> >              node.direct.setfield (new, "prev", cur)
> >            else
> >              hd = new
> >            end
> >            cur = new
> >          end
> >        end
> >
> >        if hd then
> >          local hbox = node.direct.hpack (hd)
> >          node.direct.write (hbox)
> >        end
> >      }
> >
> > I can’t think of a serious use for that property, though,
> > especially since there doesn’t appear to be a way to query the
> > number of nodes allocated.
> Indeed, so that's why the node (not direct) interface is the official 
> user one and the direct subvariant for special cases ... in fact, those 
> predefined ranges are not even official. It comes close to finding 
> security holes and exploiting them. (One can equally well abuse the 
> regular node interface.)
> >>> @Hans: Considering that node.direct has been part of the master
> >>> branch for some time and the new model is used in Context without
> >>> users reporting segfaults, could you give an ETA on when the
> >>> generic fontloader will benefit from it?
> >>
> >> hard to say ... keep in mind that the gain is mostly in complex font
> >> handling of manipulations and that is not always that generic (
> >>
> >> btw, there are some more changes in the font related code (on my
> >> machine, one of these days in the core ... given some testing by others)
> >> ... one reason for keeping this separate from generic is that i don't
> >> want to break that (esp not around tex live code freeze)
> >
> > I seem to recall that the generic font loader did actually ship
> > with the nuts code for a short while until you undid that and
> > reverted it to the previous implementation. Most of the bugs that
> > occurred since were consequences of the code being out of sync
> > with the rest of Context -- just wanted to point that out ;)
> I cannot imagine that the official loader had nuts code

I grepped the revision history and it looks like you’re right.

>                                                         (at least not 
> intentionally) because it relies on some nut helpers and also assumes 
> some consistency with other code... you may refer to the time when there 
> was a hybrid i.e. function calls mimicking the direct interface that I 
> experimented before nuts (which actually runs a few pct slower on some 
> cases).

That might be it. Actually I think I confuse this with the freeze
of font-otn in the generic code [0]. The frozen luatex-fonts-otn
differed from the then current font-otn from the outset. Compare
both files in commit ff54944.. of Marius’ mirror: Context
switched to the node.*() accessors whereas the generic code is
stuck with the earlier version since. (I recall vividly studying
font-otn.lua in search of a bug that wasn’t there because the
file wasn’t part of the merged package anymore.)

[0] http://repo.or.cz/w/context.git/commitdiff/ff54944f72aa8a402a330a82e847c9c19fba5f24

> Ok, there will be font-inj.lua soon after the tl code freeze (as drop in 
> for node-inj.lua) but that one will be tagged 'experimental' for a while 
> as I don't want to be the target of 'everything fails' mails.

Haha. Still, users haven’t yet learned to report directly to the
Context list so I don’t expect the complaint rate to rise

>                                                               It works 
> okay on my machine for a while so it's not untested.
> Most of the direct code was tested on my and Luigi's machine for half a 
> year (so I had to keep some 60 files in sync on my machine). The generic 
> copy was therefore just the same one as the context one of the day 
> before we made the switch.
> >> the latest generic code + the latest luatex already should run somewhat
> >> faster (something halfway) as the regular node code was also optimized
> >
> > Performance reached new heights during the last year, so much is
> > certain.
> You're joking now yes? The speedup was not that dramatic (unless you 
> changed something at your end).

No joke, just a different perspective: I’m primarily measuring
the font loading time which has been decimated in the last year.
Since i don’t require high throughput and find immediate visual
feedback distracting i’m not overly concerned with execution time
of a tex run.

>                                 It's not like old-time computers running 
> twice as fast each year, Btw, this is one of the reasons for the 
> speedups. I don't expect cpu's to become much faster.

Luigi did investigate the potential for parallelism, though, didn’t he?

> The only thing about speed that I can say (as I only run context) is 
> that on the average context has become faster but that's not due to font 
> code which is accounts for only a relatively small part of the run time.
> Hans
> ps. At bachotex some folks told me that the perception is that luatex is 
> slow. Apart from the expected lower speed due to 32 bit character 
> handling and wide fonts, my impression is that (at least for context) we 
> now run quite okay. Given that we do more complex things (not talking 
> fonts now) mkiv runs faster than mkii or at least comes pretty close.

Even if mkiv was noticably slower I’d never trade the extra
flexibility for it. Anyways, I never even used mkii so I only
witnessed Context getting faster and faster B-)

>                                                                       My 
> personal aim is that a edit-test cycle of a couple of pages should stay 
> below .5 sec (all included). When I get 20-50 pages per second on 
> average complex documents on my current laptop I'm fine.
> ps. I just read in tugboat that the texbook on DEK's 3.2G xeon driven 
> machine needs .3 seconds which is on such hardware twice the time 
> context needs to produce a zero page document.

Startup time should be slower due to the immense amount of code
that is loaded and executed before \starttext. Don’t you like
maintain one of the largest Lua code bases out there?


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://www.ntg.nl/pipermail/dev-luatex/attachments/20140507/7a1d7641/attachment.pgp>

More information about the dev-luatex mailing list