# [Dev-luatex] buffer size musings

David Kastrup dak at gnu.org
Thu Apr 5 16:41:01 CEST 2007

Taco Hoekwater <taco at elvenkind.com> writes:

> Hi again,
>
> David Kastrup wrote:
>>
>>> And the same is true for buffer size limit problems. You have to
>>> figure out a way to with \scantokens, or you may as well forget
>>> about writing any code at all.
>>>
>>> I see absolutely no point in solving buffer overflows for files when
>>> we keep getting unrecoverable errors for \scantokens and luatex's
>>> tex.print().
>>
>> Do you actually expect megabyte lines from \scantokens and
>> tex.print()?  That question is serious and will affect the design.
>
> Yes, that is definately possible (and even likely).
>
>> There is one construct for which I have to keep a hard limit.  \csname
>> ... \endcsname and, probably more important, \ifcsname ... \endcsname.
>> Reducing the total input buffer size to something more reasonable will
>> _definitely_ affect them, since they must fit the buffer completely.
>> The same goes for \somecontrolsequencename: this must also fit.
>
> I agree. Control sequence names longer than say 50 characters are
> unwieldy in practise anway, and defining csnames for the sake of
> hashing can better be done using lua strings. If you arrange for
> a \csname has to fit inside one of your 1K windows, that should be
> fine.

suffix.sty does something like

... \futurelet\a\b}

\def\b{\ifcsname xxx@\meaning\a\endcsname ...}

and expects to see things like
"the character *"
or so in \meaning.

If instead the meaning is a macro containing a few thousand
characters, TeX will panic.

Since \WithSuffix is used for optional arguments, and those are
supposed to be followed by "an opening brace {" or similar easily
described things, this is not likely to cause trouble as long as one
keeps braces around one's argument.

So this is an application which could cause TeX to bomb out on certain
input that is rather bold about what it feeds to \ifcsname, relying on
the guarantee that it will not impact the hash space.

suffix.sty is my own package.  I know of no other package doing
similarly reckless things, and I think it unlikely people will manage
to trigger the problematic cases.

Just wanted to mention it.  And one will always get by with increasing
the buffer size.

--
David Kastrup