On 5/27/2019 6:49 PM, Marcel Fabian Krüger wrote:
either, but I will just mention it anyway: Would it maybe be a option to provide access to the token list of macros instead? Meaning some variant of `get_macro/set_macro` which allows setting/getting the token list as returned by token.scan_toks(true, -)? This wouldn't affect any old code and not have any speciall effect for the TeX side, but would make working with macros from Lua much easier, especially if they contain too inconsistant (or unknown) catcodes to make going through a string representation a viable option. It would e.g. allow doing this `#` stuff through Lua, atomic replacements in existing macros and creating macros with more "creative" catcodes from Lua code without requiring having to go through odd `\def` constructions. Giving that the internal argument list representation is exposed through scan_toks anyway, it looks like a quite small and consistant change. Hm, not in the short term I think, but i'll keep it in mind. There's all kind of stuff going on like triggers and sentinels for arguments and ref counting and so. I also think that going too wild with lua is not a good idea. The node interface is already adding quite some fragility. The get/setmacro are actually kind of strange. One can do a lot at the tex end and if it can't be done it's probabbly also not well suited for tex. Messing with creative catcodes is no option as there is just a limited set with prescribed and often hard coded behaviour.
The whole define and expand machinery is very optimized and cannot be made much better in terms of performance, only worse and that is no option. Writing for instance that kind of stuff in lua (or adding callbacks) makes not much sense because when performance drops significantly tex basically becomes unuseable. fwiw: I've done all kind of experiments the last decade, also like the ones you suggest above, and rejected mosty because of messyness, weirdness and bordercaseness. At some point one (i can't speak for all of course) decides that there is not that much more to gain. In fact, if we remove (or bypass) some of the peculiar tex macro things one basically makes tex into something different ... some of its charm is in its weirdness ... i would have quit using it already long ago if that weren't the case. Of course I play with possible new things but it has to prove itself in usage and looking-kind-of-normaal (I admit that in luatex some things were added that don't honor that principle but removing that now is no option I think, although I have no problems kicking out strange stuff in the follow up that I experiment with in context.) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------