[Dev-luatex] [PATCH] Allow literal # in argument text by escaping

[Hans Hagen]

On 5/27/2019 6:49 PM, Marcel Fabian Krüger wrote:

> either, but I will just mention it anyway: Would it maybe be a option to
> provide access to the token list of macros instead? Meaning some variant
> of `get_macro/set_macro` which allows setting/getting the token list as
> returned by token.scan_toks(true, -)? This wouldn't affect any old code
> and not have any speciall effect for the TeX side, but would make
> working with macros from Lua much easier, especially if they contain too
> inconsistant (or unknown) catcodes to make going through a string
> representation a viable option. It would e.g. allow doing this `#` stuff
> through Lua, atomic replacements in existing macros and creating macros
> with more "creative" catcodes from Lua code without requiring having to
> go through odd `\def` constructions. Giving that the internal argument
> list representation is exposed through scan_toks anyway, it looks like a
> quite small and consistant change.
Hm, not in the short term I think, but i'll keep it in mind. There's all 
kind of stuff going on like triggers and sentinels for arguments and ref 
counting and so. I also think that going too wild with lua is not a good 
idea. The node interface is already adding quite some fragility. The 
get/setmacro are actually kind of strange. One can do a lot at the tex 
end and if it can't be done it's probabbly also not well suited for tex. 
Messing with creative catcodes is no option as there is just a limited 
set with prescribed and often hard coded behaviour.

The whole define and expand machinery is very optimized and cannot be 
made much better in terms of performance, only worse and that is no 
option. Writing for instance that kind of stuff in lua (or adding 
callbacks) makes not much sense because when performance drops 
significantly tex basically becomes unuseable.

fwiw: I've done all kind of experiments the last decade, also like the 
ones you suggest above, and rejected mosty because of messyness, 
weirdness and bordercaseness. At some point one (i can't speak for all 
of course) decides that there is not that much more to gain. In fact, if 
we remove (or bypass) some of the peculiar tex macro things one 
basically makes tex into something different ... some of its charm is in 
its weirdness ... i would have quit using it already long ago if that 
weren't the case. Of course I play with possible new things but it has 
to prove itself in usage and looking-kind-of-normaal (I admit that in 
luatex some things were added that don't honor that principle but 
removing that now is no option I think, although I have no problems 
kicking out strange stuff in the follow up that I experiment with in 
context.)

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------