Assignments in runtoks

29 Aug 2019

Hello,
in current versions of LuaTeX, `runtoks` does not preserve token lists
of ongoing assignments. Therefore running `runtoks` inside of `\edef`
or similar replace the definition with the defined value of the last
inner assignment.
For example:
\toks0{\def\innermacro{inner}}
\edef\outermacro{outer\directlua{tex.runtoks(0)}}
\show\outermacro
\bye
in plain TeX should show
...
\outermacro=macro:
->outer.
but currently does show
...
\outermacro=macro:
->inner.
instead.
(As a side effect the reference count isn't updated, so for example
\toks0{\def\innermacro{inner}\let\innermacro\undefined}
\edef\outermacro{outer\directlua{tex.runtoks(0)}}
\show\outermacro
\bye
starts printing hyphenation patterns on my system.)
This can be fixed by saving scanner_status, warning_index and def_ref
when entering local_control:

diff --git a/source/texk/web2c/luatexdir/tex/maincontrol.c b/source/texk/web2c/luatexdir/tex/maincontrol.c
index b01931dd1..284fb0ae3 100644
--- a/source/texk/web2c/luatexdir/tex/maincontrol.c
+++ b/source/texk/web2c/luatexdir/tex/maincontrol.c
@@ -1055,6 +1055,10 @@ extern void local_control_message(const char *s)
 void local_control(void)
 {
     int ll = local_level;
+    int save_scanner_status = scanner_status;
+    halfword save_def_ref = def_ref;
+    halfword save_warning_index = warning_index;
+
     main_control_state = goto_next;
     local_level += 1;
     while (1) {
@@ -1077,14 +1081,17 @@ void local_control(void)
             if (tracing_nesting_par > 2) {
                 local_control_message("leaving due to level change");
             }
-            return ;
+            break;
         } else if (main_control_state == goto_return) {
             if (tracing_nesting_par > 2) {
                 local_control_message("leaving due to triggering");
             }
-            return;
+            break;
         }
     }
+    scanner_status = save_scanner_status;
+    def_ref = save_def_ref;
+    warning_index = save_warning_index;
     return;
 }

    

done

permit nested runtoks in more complex situations of nesting (HH)

done

permit nested runtoks in more complex situations of nesting (HH)