# [Dev-luatex] token processing

Taco Hoekwater taco at elvenkind.com
Sun Jan 21 12:17:33 CET 2007

Hi all,

The text below started on a different list, then continued in private
email. Since it is probably interesting for other people on this list
as well, it is better to continue the discussion here.

Chris Rowley wrote:
> Taco
>
> Although I did a lot of work on the ltx3 langauge about 14 years ago I
> have not seen much point in trying to play thus with TeX' for amny
> decades now.  So feel free to say what you like (I know you would
> anyway:-).

Yes, I tend to do that :-)

>>You may disagree, but I personally think the actual ltx3 syntax is
>>absolutely awful.
>
> That it has to exist is awful: if you think it does not need to do so #
> (for current TeX, please tell us ... well Frank).

> The actual concrete syntax will be whatever the users want, as far as I know.
> The csname naming conventon is just that, nothing to do with the semantics.

As far as I can remember from the presentations, the important points
are that arguments have types, macros behave like procedures, and the
tail of their names function as reminder for the arguments they accept.

Is that correct? Is there some documentation I should be reading?

>>IMO, it requires a token processing layer
>
> Meaning?  You mean processing other than by expansion?

An example: I shiver when I see something like this (xparse.dtx):

a)
\xparse_error:x {
Command~name~\token_to_string:N #1'~ already~defined!
}
}

in side TeX, you almost certainly have to do it like this, but
imo it would look much better like this:

b)
xparse_error("Command name %N' already defined!", cname);
}

To go from b) to a), you could use a preprocesso. Even better
would be to allow b), or a variation thereof, as input to the
executable.

>>on top of it just to make writing any non-trivial code bearable
>>(yes, you could do that in luaTeX, using Lua, but as I am a ConTexT
>>user ...)
>
> What does Context give you for such a token processing layer?

ConTeXt does not try to force typing upon an untyped language, so
it does not have the problems of ltx3.

What I meant with my statement about being a ConTeXt user is that
as I have no use for LaTeX code, I am not going to write a token
processing layer for b) myself. Even though that could be done in
luatex right now.

> Yes, it is all for stuff that can be done in lua but that is not what
> it was intended for.

A concise definition of what it _was_ intended for would be helpful.

> What you use' (Con or La) makes no difference as a token list programmer',
> does it?

Not for LaTeX2e vs. ConTeXt II (the current versions of both). But
ltx3 looks like it is aiming for something completely different.

Best, Taco

`