On Sat Aug 28, 2021 at 9:14 PM CEST, Michal Vlasák wrote:

Hello,

take this plain LuaTeX example:

\setbox0=\hbox{\pdfextension literal{0 g}}

% 1)
\directlua{
    local literal = tex.getbox(0).head
    texio.write_nl("log", "literal.data="..literal.data)
}
\showbox0

% 2)
\directlua{
    local literal = tex.getbox(0).head
    literal.data = "test"
    literal.token = "toks"
    texio.write_nl("log", "literal.token="..(literal.token or "nil"))
    }
\showbox0

% 3) patch test
\directlua{
    tex.set("everyjob", "asd")
    texio.write_nl("log", tex.get("everyjob"))
}

\bye

Expected log output (abridged):

1. literal.data=0 g .\pdfliteral origin{0 g}

2. literal.token=toks .\pdfliteral origin{toks}

Actual output:

1. literal.data=data .\pdfliteral origin{0 g}

2. literal.token=characters .\pdfliteral origin <lua data reference 153>

In the first case the Lua accessor returns a value which happens to be on top of the stack (the key "data" itself). In the second case an index into TeX memory is misinterpreted as a Lua registry index, so the returned data is essentially garbage.

Patch for both issues is attached. I also extended `tokenlist_from_lua` (which is what `nodelib_gettoks` is defined as) to allow an index argument. The previous version used the value on top of the stack, which probably worked for every use currently in LuaTeX, but seemed rather dangerous and subtle.

The patch is from git, though it can be applied normally with

patch -Np1 < pdf_literal.patch

Kind regards, Michal Vlasák

Hello,

seems like the issue was rediscovered and partly fixed by Phelype and Hans:

https://gitlab.lisn.upsaclay.fr/texlive/luatex/-/commit/c1909e4a4c5311197a25...

Though the fix omits one important line that I had in my patch (first change below). Also IMO the second change below is also worthwhile, as it makes Lua stack use more obvious.

--- a/source/texk/web2c/luatexdir/lua/lnodelib.c +++ b/source/texk/web2c/luatexdir/lua/lnodelib.c @@ -1566,7 +1566,7 @@ static int lua_nodelib_direct_setleader(lua_State * L) #define get_pdf_literal_direct_value(L,n) do { \ if (pdf_literal_type(n) == lua_refid_literal) { \ lua_rawgeti(L, LUA_REGISTRYINDEX, pdf_literal_data(n)); \ - } else if (pdf_literal_type(n) == lua_refid_literal) { \ + } else if (pdf_literal_type(n) == normal) { \ tokenlist_to_luastring(L, pdf_literal_data(n)); \ } \ } while (0) --- a/source/texk/web2c/luatexdir/lua/ltexlib.c +++ b/source/texk/web2c/luatexdir/lua/ltexlib.c @@ -1790,7 +1790,7 @@ static int settex(lua_State * L) } } else if (is_toks_assign(cur_cmd1)) { if (lua_type(L,i) == LUA_TSTRING) { - j = tokenlist_from_lua(L, -1); /* uses stack -1 */ + j = tokenlist_from_lua(L, i); assign_internal_value((isglobal ? 4 : 0), equiv(cur_cs1), j);

} else {

New patch attached. Same test case & patch instructions apply.

(For reference, the message I reply to: https://mailman.ntg.nl/pipermail/dev-luatex/2021-August/006542.html)

Best, Michal