Taco Hoekwater
Hi again,
David Kastrup wrote:
And the same is true for buffer size limit problems. You have to figure out a way to with \scantokens, or you may as well forget about writing any code at all.
I see absolutely no point in solving buffer overflows for files when we keep getting unrecoverable errors for \scantokens and luatex's tex.print().
Do you actually expect megabyte lines from \scantokens and tex.print()? That question is serious and will affect the design.
Yes, that is definately possible (and even likely).
There is one construct for which I have to keep a hard limit. \csname ... \endcsname and, probably more important, \ifcsname ... \endcsname. Reducing the total input buffer size to something more reasonable will _definitely_ affect them, since they must fit the buffer completely. The same goes for \somecontrolsequencename: this must also fit.
I agree. Control sequence names longer than say 50 characters are unwieldy in practise anway, and defining csnames for the sake of hashing can better be done using lua strings. If you arrange for a \csname has to fit inside one of your 1K windows, that should be fine.
suffix.sty does something like ... \futurelet\a\b} \def\b{\ifcsname xxx@\meaning\a\endcsname ...} and expects to see things like "the character *" or so in \meaning. If instead the meaning is a macro containing a few thousand characters, TeX will panic. Since \WithSuffix is used for optional arguments, and those are supposed to be followed by "an opening brace {" or similar easily described things, this is not likely to cause trouble as long as one keeps braces around one's argument. So this is an application which could cause TeX to bomb out on certain input that is rather bold about what it feeds to \ifcsname, relying on the guarantee that it will not impact the hash space. suffix.sty is my own package. I know of no other package doing similarly reckless things, and I think it unlikely people will manage to trigger the problematic cases. Just wanted to mention it. And one will always get by with increasing the buffer size. -- David Kastrup