[Dev-luatex] whatsit (subtype:write) bug?

[Sigitas Tolusis]

On 24 Oct 2019, at 13:10, Linas Stonys <lstonys at vtex.lt> wrote:
> 
> Hi,
> I noticed that whatsit subtype "write" has changed "data" field type.
> Documentation says "data -> table representing the token list to be written"
> but when I print it I see it as a string with LuaTeX, Version 1.10.0 (TeX Live 2019):
> HERE    <node    277 <     82 >    nil : whatsit 1>     \foo\fee
> 
> With LuaTeX, Version 1.07.0 (TeX Live 2018/W32TeX) is ok:
> HERE    <node    276 <     82 >    nil : whatsit 1>     table: 02D87EC8
> 
> 
> Minimal example:
> \write1{\string\foo \unexpanded{\fee}}%
> \directlua{
> local head = tex.nest[tex.nest.ptr].head
> local tl = node.tail(head)
> print ("HERE", tl, tl.data)
> }
> \bye
> 
> 
> The real world example will be:
> \documentclass{article}
> \usepackage[color=no]{nodetree}
> \nodetreeoption[channel]{log}
> \nodetreeregister{preout}        %  pre_output will rise an error trying to print whatsits data field
> 
> \begin{document}
> \makeatletter
> \protected at write \@auxout {}{\string\@gobble\string\fee}    %% this is ok
> \protected at write \@auxout {}{\string\@gobble\protect\foo}   %% but this will rise \foo undefined error
> \makeatother
> \end{document}
> 
> And this is very strange that accessing whatsits (subtype "write") "data" field executes it.

The reason is that function get_write_direct_value (lnodelib.c:1652) provides expanded tokenlist version 
by function expand_macros_in_tokenlist (lnodelib.c:1652; defined in extensions.c:512). 

Maybe a bit changed behaviour with providing unexpanded tokenlist be more useful and flexible working with 
whatsit data from lua side. In such case minimal changes could be: to use by get_write_direct_value a modified
version of  expand_macros_in_tokenlist  with different token scan changed from  scan_toks(false, true) to 
 scan_toks(false, false) (extensions.c:541).

Regards,
Sigitas