Re: [Dev-luatex] Fix for pdf_literal Lua string / token list conversions

5 Nov 2022

      On Sat Aug 28, 2021 at 9:14 PM CEST, Michal Vlasák wrote:
...
Hello,
take this plain LuaTeX example:
\setbox0=\hbox{\pdfextension literal{0 g}}
% 1)
    \directlua{
        local literal = tex.getbox(0).head
        texio.write_nl("log", "literal.data="..literal.data)
    }
    \showbox0
% 2)
    \directlua{
        local literal = tex.getbox(0).head
        literal.data = "test"
        literal.token = "toks"
        texio.write_nl("log", "literal.token="..(literal.token or "nil"))
        }
    \showbox0
% 3) patch test
    \directlua{
        tex.set("everyjob", "asd")
        texio.write_nl("log", tex.get("everyjob"))
    }
\bye
Expected log output (abridged):
1)
    literal.data=0 g
    .\pdfliteral origin{0 g}
2)
    literal.token=toks
    .\pdfliteral origin{toks}
Actual output:
1)
    literal.data=data
    .\pdfliteral origin{0 g}
2)
    literal.token=characters
    .\pdfliteral origin 
In the first case the Lua accessor returns a value which happens to be
on top of the stack (the key "data" itself). In the second case an index
into TeX memory is misinterpreted as a Lua registry index, so the
returned data is essentially garbage.
Patch for both issues is attached. I also extended `tokenlist_from_lua`
(which is what `nodelib_gettoks` is defined as) to allow an index
argument. The previous version used the value on top of the stack, which
probably worked for every use currently in LuaTeX, but seemed rather
dangerous and subtle.
The patch is from git, though it can be applied normally with
patch -Np1 < pdf_literal.patch
Kind regards,
Michal Vlasák
Are there any news on this?

I must admit I don't remember the details very well, but I used the
patch without issues. Though I would appriciate review from someone,
because I could have botched the patch in some subtle way.

Thanks in advance!

Michal