Hi again, David Kastrup wrote:
And the same is true for buffer size limit problems. You have to figure out a way to with \scantokens, or you may as well forget about writing any code at all.
I see absolutely no point in solving buffer overflows for files when we keep getting unrecoverable errors for \scantokens and luatex's tex.print().
Do you actually expect megabyte lines from \scantokens and tex.print()? That question is serious and will affect the design.
Yes, that is definately possible (and even likely).
There is one construct for which I have to keep a hard limit. \csname ... \endcsname and, probably more important, \ifcsname ... \endcsname. Reducing the total input buffer size to something more reasonable will _definitely_ affect them, since they must fit the buffer completely. The same goes for \somecontrolsequencename: this must also fit.
I agree. Control sequence names longer than say 50 characters are unwieldy in practise anway, and defining csnames for the sake of hashing can better be done using lua strings. If you arrange for a \csname has to fit inside one of your 1K windows, that should be fine. Best, Taco