Hello, take the following PlainTeX file: %&luatex \def\recatcode#1{% \directlua0{% tex.sprint('\luaescapestring{\unexpanded{#1}}')% texio.write_nl('\luaescapestring{\unexpanded{#1}}')% }% } \recatcode{% \def\test#1{`#1'}% } \bye This results in This is LuaTeX, Version snapshot-0.30.0-2008093018 (Web2C 7.5.7) (MacroParameterCharacter.tex \def \test ##1{`##1'} ! Parameters must be numbered consecutively. <to be read again> ## l.1 \def \test ## 1{`##1'} \recatcode ...uaescapestring {\unexpanded {#1}}')} l.12 } ? x TeX interrupted. No pages of output. Transcript written on MacroParameterCharacter.log. So `#' is transformed into `##' when passed from TeX to Lua. Is this a bug or a feature? Jonathan
"Jonathan Sauer"
take the following PlainTeX file:
%&luatex \def\recatcode#1{% \directlua0{% tex.sprint('\luaescapestring{\unexpanded{#1}}')% texio.write_nl('\luaescapestring{\unexpanded{#1}}')% }% }
\recatcode{% \def\test#1{`#1'}% }
\bye
This results in
This is LuaTeX, Version snapshot-0.30.0-2008093018 (Web2C 7.5.7) (MacroParameterCharacter.tex \def \test ##1{`##1'} ! Parameters must be numbered consecutively. <to be read again> ## l.1 \def \test ## 1{`##1'} \recatcode ...uaescapestring {\unexpanded {#1}}')}
l.12 }
? x TeX interrupted. No pages of output. Transcript written on MacroParameterCharacter.log.
So `#' is transformed into `##' when passed from TeX to Lua.
No, when TeX prints.
Is this a bug or a feature?
Feature. Quite an annoying one in my opinion, but there you are. -- David Kastrup
David Kastrup a écrit :
So `#' is transformed into `##' when passed from TeX to Lua.
No, when TeX prints.
Is this a bug or a feature?
Feature. Quite an annoying one in my opinion, but there you are.
By the way, using LaTeX, the "ted" package has a command \ShowTokens which is meant to be handy to examine token lists: is doesn't have this "funny feature" of doubling ## (nor of addind spaces after some cs names) and moreover, lets you know about the catcodes and distinguishes between cs names and plain stings (unlike \show, \showtokens, etc.) Manuel.
David Kastrup wrote:
So `#' is transformed into `##' when passed from TeX to Lua.
No, when TeX prints.
You are saying the same thing. The tokenlist that results from the <general text> that is read after the \directlua is internally converted to a string before the lua interpreter gets a look at it. This currently uses TeX's tokenlist print function, but I could make it behave differently, no problem (after all, this generated string is only 'seen' by the lua interpreter). Best wishes, Taco
Hello,
So `#' is transformed into `##' when passed from TeX to Lua. [...] This currently uses TeX's tokenlist print function, but I could make it behave differently, no problem (after all, this generated string is only 'seen' by the lua interpreter).
In that case: Please make it so.
Best wishes, Taco
Thanks in advance, Jonathan
"Jonathan Sauer"
Hello,
So `#' is transformed into `##' when passed from TeX to Lua. [...] This currently uses TeX's tokenlist print function, but I could make it behave differently, no problem (after all, this generated string is only 'seen' by the lua interpreter).
In that case: Please make it so.
It is consistent with TeX's printing mechanism. You could deal with this problem by placing the resulting string inside of a macro body (where ## will get collapsed to # while the body is being collected) and then use it from there. Disadvantage: this is not an expandible method. -- David Kastrup
Hello,
So `#' is transformed into `##' when passed from TeX to Lua. [...] This currently uses TeX's tokenlist print function, but I could make it behave differently, no problem (after all, this generated string is only 'seen' by the lua interpreter).
In that case: Please make it so.
It is consistent with TeX's printing mechanism.
Sure. But I don't think that this consistency is necessary. Also, I would guess that "#" in a token register is not doubled when the token register is accessed from Lua, so it would be consistent with how "#" between TeX and Lua is handled there.
You could deal with this problem by placing the resulting string inside of a macro body (where ## will get collapsed to # while the body is being collected) and then use it from there.
Disadvantage: this is not an expandible method.
Or I could use gsub in the Lua code. Disadvantage: It is expandable, but slower. Also, I need two functions in Lua: One to be called from TeX, and one to be called from other Lua functions. Jonathan
Jonathan Sauer wrote:
Hello,
So `#' is transformed into `##' when passed from TeX to Lua. [...] This currently uses TeX's tokenlist print function, but I could make it behave differently, no problem (after all, this generated string is only 'seen' by the lua interpreter). In that case: Please make it so. It is consistent with TeX's printing mechanism.
Sure. But I don't think that this consistency is necessary. Also, I would guess that "#" in a token register is not doubled when the token register is accessed from Lua, so it would be consistent with how "#" between TeX and Lua is handled there.
Actually the problem was with \luaescapestring. Commit #1544 changes the behaviour of \luaescapestring so that it no longer duplicates characters with catcode 6. Please verify that that does what you want. Best wishes, Taco
Hello,
Actually the problem was with \luaescapestring. Commit #1544 changes the behaviour of \luaescapestring so that it no longer duplicates characters with catcode 6. Please verify that that does what you want.
Thank you very much! I tried it with the following (simpler) example: %&luatex \directlua0{% texio.write_nl('\luaescapestring{\unexpanded{#}}')% texio.write_nl('\luaescapestring{#}')% texio.write_nl('#')% % texio.write_nl(#_VERSION)% A } \bye This is LuaTeX, Version snapshot-0.30.0-2008100718 (Web2C 7.5.7) (MacroParameterCharacter2.tex # # ## ) No pages of output. Transcript written on MacroParameterCharacter2.log. So with \luaescapestring, the `#' is not doubled anymore, without it still is. Although most of the time, strings will be passed to Lua through \luaescapestring, this behaviour still creates problems when `#' is used inside Lua code (as in the commented line A above, where I try to print the length of the _VERSION string); also it is a bit inconsistent. What do you think?
Best wishes, Taco
Jonathan
Jonathan Sauer wrote:
So with \luaescapestring, the `#' is not doubled anymore, without it still is. Although most of the time, strings will be passed to Lua through \luaescapestring, this behaviour still creates problems when `#' is used inside Lua code (as in the commented line A above, where I try to print the length of the _VERSION string); also it is a bit inconsistent.
What do you think?
#1546 Best wishes, Taco
Hello,
So with \luaescapestring, the `#' is not doubled anymore, without it still is. Although most of the time, strings will be passed to Lua through \luaescapestring, this behaviour still creates problems when `#' is used inside Lua code (as in the commented line A above, where I try to print the length of the _VERSION string); also it is a bit inconsistent.
What do you think?
#1546
Thank you!
Best wishes, Taco
Jonathan
participants (4)
-
David Kastrup -
Jonathan Sauer -
Manuel Pégourié-Gonnard -
Taco Hoekwater