Hi,
I recently noticed some cases where luametatex behaved in unexpected
ways:
- The "Extra \fi" error isn't triggered, instead an extra `\fi`
freezes luametatex. (Can be reproduced by compiling a document which
only consists of a single \fi)
- token.new can only create some `data` tokens, but it doesn't apply
bound checking on it's arguments:
Take
```
\directlua{
t = token.new(0x200000, token.command_id'data')
print(t.cmdname, t.command, t.mode)
}
```
which prints
register 102 0
The issue does not seem to be that such tokens do not exists because
\letdatacode\somedata="200000
\directlua{
local t = token.create'somedata'
print(t.cmdname, t.command, t.index)
}
does print
data 101 2097152
Also for all other commands LuaTeX seems to apply range-checks to
ensure that such overflows don't happen, even if invalid values are
passed as firstargument.
- There is token.primitives(). My assumption is that the returned
table is meant to indicate the command is, mode and name
corresponding to every primitive. (I think it is awesome that such a
table is made available in luametatex) But especially the mode
field sometimes has values which do not correspond to the mode of
the actual primitives:
I tried running the following in an almost iniTeX setting where all
primitives aside from \shipout and \Umathcodenum have their default
definitions:
```
\catcode`\%=12
\catcode`\~=12
\directlua{
local sorted = token.primitives()
table.sort(sorted, function(a,b) return a[1]