[Dev-luatex] .BEILMOPS or how I stopped worrying and love Open Source

David Kastrup dak at gnu.org
Wed Dec 12 10:26:53 CET 2007

"Jonathan Sauer" <Jonathan.Sauer at silverstroke.com> writes:

> Hello,
>> >> when using lpeg an dmatching, keep in mind that \unexpanded has the
>> >> side effect of introducing spaces
>> >
>> > Oh. I used \unexpanded, because sometime someone on this mailing list 
>> > noted that it was faster than \detokenize.
>> Should make no difference, I guess.  \detokenize should 
>> introduce the same spaces, using the same algorithm.
> Well, currently it makes a difference, three actually:
> 1.	\unexpanded introduces "IMPOSSIBLE."


> 2.	\unexpanded introduces spaces after control sequences.

Why wouldn't \detokenize do the same?  Wait, it does:

dak at lola:~$ etex
This is pdfTeXk, Version 3.141592-1.40.3 (Web2C 7.5.6)
 %&-line parsing enabled.
**\message{\detokenize{\a b}}
entering extended mode
\a b
*\message{\unexpanded{\a b}}
\a b
No pages of output.
Transcript written on texput.log.

> 3.	\unexpanded ignores \par's, while \detokenize does not. So empty
> 	lines in \directlua are no problem with \unexpanded, but result
> 	in a (Lua) parse error when \detokenize is used.
> At least in the context of \directlua.

I guess that this may be the real reason.

> Also, in terms of speed: \unexpanded only marks all tokens as
> unexpandable, while \detokenize creates new character tokens. So the
> latter should be somewhat slower.

No question about that.

>> >> btw, if you change \unexpanded by \detokenize you get the desired 
>> >> result
>> >
>> > I know.
>> Oops.  Why would that be?
> Because of a bug? :-)

I should think so.  Not sure whether this behavioral difference is
intended, but it certainly feels wrong to me.

David Kastrup, Kriemhildstr. 15, 44793 Bochum

More information about the dev-luatex mailing list