On Thu, 10 Oct 2024 at 09:47, Hans Hagen <j.hagen@xs4all.nl> wrote:
On 10/10/2024 8:29 AM, Petr Olsak wrote:
> Hello,
>
> my question: is there a reason why \toksapp assignment cannot be set
> expandable by \immediateassigned or \immediateassignment?
>
> We can compare:
>
>     \immediateassigned{\toks0={abc} \toks0\ea{\the\toks0 xyz}}
>
> it works. On the other hand:
>
>     \immediateassigned{\toks0={abc} \toksapp0{xyz}}
>
> fails with error:
>
> ! This can't happen (prefix).
> l.2 \immediateassigned{\toks0={abc} \toksapp
>                                            0{xyz}}
>
> Where is a difference?

Most assignmentd are handled by the 'prefix' routine (\global etc) but
not these token list combiners (has to do with memory management and such).

> It should be great if \toksapp can be set expandable. I use it in my
> macro \replstring. If very large strings are modified by this macro at
> many points then there is significantly better calculation speed with
> \toksapp than \toks\ea{\the\toks} (approximately ten times better). So
> \toksapp is optimal but if it is used in a macro then the macro cannot
> be set expandable.

I did a quick test with a patch to also handle these in the immediate
assign and it seems to work. No guarantees, but we'll consider it anyway.

Hans



commit fc89b39b1cbe9b420a767f59adb1092e3ec70afc
Author: Luigi Scarso <luigi.scarso@gmail.com>
Date:   Sat Oct 12 10:04:14 2024 +0200

    Handle token combiners in immediate assignments (H.Hagen). LuaTeX 1.19.0

Also into TeXLive repo.

--
luigi