local allocation in ConTeXt

27 Dec 2016

      Hi!

I'm the author of a PGF/TikZ-based LaTeX package Forest for drawing 
(linguistic) trees. I have received several requests to support ConTexT 
and I'm trying to do that now.

Forest uses a lot of \boxes. Actually, the number of boxes used cannot 
be known in advance, as it depends on the document. Here's why: Forest's 
main job is to place the nodes of a tree so that they don't overlap and 
to do that, it needs to first typeset the individual nodes and measure 
them. The typeset nodes are kept around and later used when typesetting 
the entire tree.

There are three reasons for keeping the typeset nodes, and not 
typesetting them again as part of the entire tree. First (at least 
historically): performance. Second: there is no guarantee that running 
the same code twice will result in the same box. Third: changing this 
now would break compatibility.

The typeset nodes are kept in "local" boxes. They are created using 
\locbox, which is defined either by plain eTeX format or package 
elocalloc.sty for LaTeX >2015. (Prior to LaTeX 2015, \locbox was defined 
in etex.sty.) elocalloc.sty's \locbox is somewhat broken, but still 
works, mainly because it seems no other package is using local 
allocation. ;-)

Using normal, "global" boxes, created by \newbox, would be bad. If a 
\newbox was simply executed for each new node, the number of used boxes 
would remain increased even after Forest was done with a tree, so in a 
document with lots of big trees, the upper limit of 32000+ could 
actually be hit. An alternative approach would be to track and reuse 
boxes which were created by \newbox by are not used anymore. But 
implementing such a tracking mechanism seems quite complicated, and 
besides, it would not be perfect: after exiting the TeX group where the 
tree is drawn, the allocated boxes would persist. (So, from the outside 
perspective, Forest's box usage in the entire document would equal the 
box usage of the largest tree.)

The approach taken in the implementation of \locbox seems quite 
appealing in its simplicity: The local boxes (counts etc) are allocated 
"from the top" (starting at 32767 and going down) and all the \chardefs 
and allocation register changes are local (i.e. not \global). A bit of 
care to prevent clashes with global allocation "from the bottom", and 
that's it.

I've studied the allocation mechanism of ConTeXt --- but note that I'm 
new to ConTeXt! ---  and it seems that I'd know how to implement 
\loc<thing>s using "from the top" approach, but there are two, or rather 
three, problems. (I took a look at both MKII and MKIV, but the changes 
seemed mostly cosmetic, so I concentrated on MKIV.)

1. I'm not sure how happy ConTeXt were if I changed the value of 
\c_syst_max_allocated_register.  The name seems to imply it should be 
constant.

2. Simply decreasing \c_syst_max_allocated_register would while doing a 
\locbox would also decrease the upper limit for, say, \loccount, as they 
use the same upper limit, i.e. \c_syst_max_allocated_register. A proper 
way would be to declare \c_syst_max_allocated_box_register etc. Again, I 
have no idea how such a change would impact other parts of the system. 
(Btw, elocalloc.sty's implementation, unlike the plain eTeX one, suffers 
from precisely this problem.)

3. It seems that, unlike in the messy LaTeX world, ConTeXt people like 
to do things The Right Way. ;-)  As making a change like this in a 
module is surely not The Right Way, I hope that ConTeXt developers can 
help me out here, either by implementing the local allocation (I'm happy 
to assist in any way) or by maybe suggesting an alternative approach. 
(Btw, if it helps, I guess a MKIV-only solution would be acceptable to 
most users.)

Thanks in advance,
Sašo Živanović

Sašo Živanović

Aditya Mahajan

Sašo Živanović

Hans Hagen

Sašo Živanović

tags

participants (3)