[NTG-context] lmts status

[Hans Hagen]

Hi,

There haven't been lmtx updates for while now but there is a reason for 
it. On my machine I have luametatex 2.08 (so we jump number) which is in 
terms of typesetting the 20201010 version. That version has some 
reorganized internals, most have to do with consistency in the token 
interface at the Lua end. Of course there are some new and improved 
things too but those will be revealed in articles or manuals or the 
development status documents (already on my machine but they need checking).

However, around that date I also decided to play a bit with some pending 
features: extended macro definitions and frozen macros. The first has to 
do with extensions to the macro definition code. Some are already 
present for a while, others have been added. The challenge there is to 
make defining macros somewhat more convenient and at the same time get 
rid of often too confusing error messages when a user does something bad 
with arguments (those foo_bar_whatever kind of things which might look 
impressive but often is pretty ugly). Of course that only makes sense 
when we keep the current performance: only gain, no pain. But, it also 
means that we need to adapt quite some code in order to benefit from it 
and when I started doing that one quickly ends up in a chain reaction: 
from a dozen files to over 100 and fives times that amount pending. But 
it's actually kind of fun (also for users I think). And hopefully 
Wolfgang will find my many errors.

In parallel I played with the other feature: some kind of protection 
against macro abuse. This is actually related to the above because when 
you want to see macros with readable names in error messages, you cannot 
protect them with underscores and other ugly tricks that at the same 
time make stuff unreadable and ugly. We have a chicken and egg issue there.

Anyway, the reason for this mail is that I want to warn users that the 
next lmtx release will use several protection mechanisms. Checking will 
for now be off by default but one can bump the level of protection at 
runtime. Just to give you an idea: we have primitives that can get a 
heavy protection against overload. We also have macro package system 
commands and these can get a pretty hard protection too. Commands 
defined by users (using \define...) can be treated less strict. Of 
course we also have all kind of runtime (temporary) macros that we have 
to protect (freeze) but also be able to redefine when we want to (think 
of variables). Then we have properties that are not primitives but 
nevertheless need the same strong protection, think of predefined 
counters, dimensions etc. So, you can imagine that there is some work 
involved in getting all that up and running. A side effect of that is 
that occasionally I notice some undesired hack or something that needs 
attention. The good news is that this probably can be pulled off without 
  a real hit on performance.

However, one side effect is that when users enable this (off by default) 
mechanism the might run into messages (when only warnings are enables) 
or errors (when they go for the full package). Now, in order to get this 
working well (as intended) I'd like users who then run into issues to 
make small examples that we we add to the test suite (unless of course 
they decide to fix the style, simply because they figure that they did 
something wrong). You're warned.

So, when the next lmtx hits the web, make sure that you keep the old 
tree because one never knows where I mess up. I don't know when I'll 
upload. After all Wolfgang and I need to come to the conclusion that 
it's the way to go.

More later,

Hans

PS. When doing this one runs into quite interesting conceptual issues, 
even in a system like context that is rather structured. But this is 
also why the final solution needs somewhat more work that one thinks 
when one starts with it. A macro language like tex is quite different 
from some regular programming language (and making it look like one 
doesn't help).

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------