[NTG-context] \scantokens in luametatex

[Hans Hagen]

On 8/13/2020 2:31 PM, Marcel Fabian Krüger wrote:

> in the current luametatex upload, \scantokens (and also \scantextokens)
> behave odd: They seem to act like \detokenize, except that spaces get
> catcode other instead of catcode space:
> 
> \starttext
> \edef\abc{\scantokens{\relax}}
> \abc
> \edef\abc{\scantextokens{\relax}}
> \abc
> \stoptext
> 
> used to result in an empty output file but with luametatex, it  writes \relax \relax to the output.
i'll look at it (i probably forget to set the catcodetable)

here is the generalized case:

     \def\foo{!}

     \edef\abc{\tokenized catcodetable \catcodetable {a\relax \foo}} \abc
     \edef\abc{\tokenized catcodetable -1            {a\relax \foo}} \abc

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------