Currently LuaTeX ignores "null paragraphs" (see the manual at the end of section 3.3.2 "How it works"), but it is actually doing so only for paragraphs created with end_graf. enter_display_math ignores paragraphs if they only contain local_par nodes, but not if they also contain dir nodes. This is somewhat inconsistent, so it would be nice if enter_display_math would discard paragraphs like end_graf. I attached a patch. It is almost identical to the change in https://gitlab.lisn.upsaclay.fr/texlive/luatex/-/commit/908e51adaad4b4048fed... but it is even smaller, since enter_display_math already discard paragraphs containing only local_par nodes. Even though this change is less disruptive, there might be more concerns about backwards compatibility today than three years ago, if this is the case a new value to \matheqdirmode can be added. Udi
On Sun, Feb 2, 2025 at 11:56 AM Udicoudco
but it is even smaller, since enter_display_math already discard paragraphs containing only local_par nodes.
Oh, end_graf was also flushing a paragraph with only local_par nodes, at the time so the change is the same. Udi
On 2/2/2025 10:56 AM, Udicoudco wrote:
Currently LuaTeX ignores "null paragraphs" (see the manual at the end of section 3.3.2 "How it works"), but it is actually doing so only for paragraphs created with end_graf. enter_display_math ignores paragraphs if they only contain local_par nodes, but not if they also contain dir nodes. This is somewhat inconsistent, so it would be nice if enter_display_math would discard paragraphs like end_graf.
I attached a patch. It is almost identical to the change in https://gitlab.lisn.upsaclay.fr/texlive/luatex/-/commit/908e51adaad4b4048fed... but it is even smaller, since enter_display_math already discard paragraphs containing only local_par nodes.
Even though this change is less disruptive, there might be more concerns about backwards compatibility today than three years ago, if this is the case a new value to \matheqdirmode can be added.
This kind of behavior is not going to change .. adding / changing things wrt luatex waS frozen for a reason so only real bugs get fixed. One can use callbacks to handle whatever comes in the way (in addition to wrapping solutions in macros which in the case of mixed direction typesetting makes a lot of sense). Keep in mind that there a lot of macro code around thta depends on the engine not changing. (Of course one can have a local version of luatex that has whatever patches one wants.) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
Hello Hans,
On Sun, Feb 2, 2025 at 1:35 PM Hans Hagen
This kind of behavior is not going to change .. adding / changing things wrt luatex waS frozen for a reason so only real bugs get fixed.
Then why was it changed *after* LuaTeX was declared as frozen (LuaTeX 1.10) for paragraphs not created by math displays? (It was you who changed it :) To me, it seems like it was decided it is a bug... and simply display math was not considered at the time.
Keep in mind that there a lot of macro code around thta depends on the engine not changing.
This is why I suggested to add a new possible value to one of the *mode primitives, if you think this is a breaking change (although it was not done three years ago)
(Of course one can have a local version of luatex that has whatever patches one wants.)
True, but I don't think I'm the only one interested in that: https://tex.stackexchange.com/questions/716621/bidi-document-grouping-list-e... Udi
On 2/2/2025 12:52 PM, Udicoudco wrote:
Hello Hans,
On Sun, Feb 2, 2025 at 1:35 PM Hans Hagen
wrote: This kind of behavior is not going to change .. adding / changing things wrt luatex waS frozen for a reason so only real bugs get fixed.
Then why was it changed *after* LuaTeX was declared as frozen (LuaTeX 1.10) for paragraphs not created by math displays?
well, 1.10 was kind of an official release where we decided that this-is-it also because of pressure to freeze (before that we only had sort of stable development snapshots for TL) but realy freezing happened bit later with it now being deep-freeze
(It was you who changed it :)
indeed kind of a bug
To me, it seems like it was decided it is a bug... and simply display math was not considered at the time.
because displaymmath is used in various scenarios: as a pargaraph of its own, inside a paragraph (where it is assumed to take 3 lines), in order to catch a previous line width (kind of an abuse), etc we also don't remove dir nodes from hboxes when there's only dirs (so \ifvoid is seeing them)
Keep in mind that there a lot of macro code around thta depends on the engine not changing.
This is why I suggested to add a new possible value to one of the *mode primitives, if you think this is a breaking change (although it was not done three years ago)
indeed something like that has to be under mode primitive control; for an empty paragraph one can argue that the textdir is a side effect but for a construct like display math it's a bit different, so maybe mode control is indeed needed here (\mathemptydisplaymode) as we can't know if it won't break something where it is assumed so can you add a mode ? (i have no time for doing that now) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Sun, Feb 2, 2025 at 3:18 PM Hans Hagen
well, 1.10 was kind of an official release where we decided that this-is-it also because of pressure to freeze (before that we only had sort of stable development snapshots for TL) but realy freezing happened bit later with it now being deep-freeze
Oh, I see. Thanks for the clarification.
because displaymmath is used in various scenarios: as a pargaraph of its own, inside a paragraph (where it is assumed to take 3 lines), in order to catch a previous line width (kind of an abuse), etc
we also don't remove dir nodes from hboxes when there's only dirs (so \ifvoid is seeing them)
Interesting, I've never thought about it.
indeed something like that has to be under mode primitive control; for an empty paragraph one can argue that the textdir is a side effect but for a construct like display math it's a bit different, so maybe mode control is indeed needed here (\mathemptydisplaymode) as we can't know if it won't break something where it is assumed
so can you add a mode ? (i have no time for doing that now)
Absolutely! Any preferences on what values should it take? Positive/Negative like \matheqdirmode or simply 1 like \breakafterdirmode? Another option, to avoid introducing a new primitive, is that if \mathdisplayskipmode is negative (-1, -2 or -3) then null pars will be flushed (kind of make sense as it also controls white space), and for the skip around the display the absolute value will be taken into account. Udi
On 2/2/2025 3:13 PM, Udicoudco wrote:
Absolutely! Any preferences on what values should it take? Positive/Negative like \matheqdirmode or simply 1 like \breakafterdirmode?
\mathbreakafterdirmode
Another option, to avoid introducing a new primitive, is that if \mathdisplayskipmode is negative (-1, -2 or -3) then null pars will be flushed (kind of make sense as it also controls white space), and for the skip around the display the absolute value will be taken into account.
one never know if someone already sets it to a negative value (maybe as signal) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Sun, 2 Feb 2025 at 15:22, Hans Hagen
On 2/2/2025 3:13 PM, Udicoudco wrote:
Absolutely! Any preferences on what values should it take? Positive/Negative like \matheqdirmode or simply 1 like \breakafterdirmode?
\mathbreakafterdirmode
Another option, to avoid introducing a new primitive, is that if \mathdisplayskipmode is negative (-1, -2 or -3) then null pars will be flushed (kind of make sense as it also controls white space), and for the skip around the display the absolute value will be taken into account.
one never know if someone already sets it to a negative value (maybe as signal)
Hi all, this is the plan for TeX Live 2025 at https://www.tug.org/texlive: 1feb: candidate/final sources committed, test builds begin. 8feb: TL pretest starts, CTAN updates continue there and in tlnet. 22feb: code freeze for final build, major bug fixes only. 1mar: final updates from CTAN, final doc tweaks. 8mar: release TL and MacTeX. and yes, we are a little late -- and I guess that in this case we have to update the manual too. -- luigi
On 2/2/2025 10:56 AM, Udicoudco wrote:
Even though this change is less disruptive, there might be more concerns about backwards compatibility today than three years ago, if this is the case a new value to \matheqdirmode can be added.
I checked how I'd do it in luametatex and noticed a potential issue with the patch: because there can be more than one node (par plus plenty dir) you need to flush the list starting with vlink(head), otherwise you leak nodes. (Because pop_nest is kind of dumb that should be enough but in other situations one should also set tail to head and the next node of head to null.) (I actually added a bit more clever variant so that i can test some scenarios, but right now I can't forecast side effects as i have no clue about intended usage in macro packages as we don't need it in context.) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Sun, Feb 2, 2025 at 7:52 PM Hans Hagen
On 2/2/2025 10:56 AM, Udicoudco wrote:
Even though this change is less disruptive, there might be more concerns about backwards compatibility today than three years ago, if this is the case a new value to \matheqdirmode can be added.
I checked how I'd do it in luametatex and noticed a potential issue with the patch: because there can be more than one node (par plus plenty dir) you need to flush the list starting with vlink(head), otherwise you leak nodes.
(Because pop_nest is kind of dumb that should be enough but in other situations one should also set tail to head and the next node of head to null.)
(I actually added a bit more clever variant so that i can test some scenarios, but right now I can't forecast side effects as i have no clue about intended usage in macro packages as we don't need it in context.)
Oh, yes, a total oversight on my part... I intended to copy what is done in end_graf (which also suits the current behaviour that tests that vlink(head)==tail and flushes the tail, so flushing vlink(head) is the same). wrong copy paste that worked for my too simplistic tests. I'll test some more and send a new patch with the primitive and doc in a couple of hours. Thanks, Udi
On Sun, Feb 2, 2025 at 9:38 PM Udicoudco
On Sun, Feb 2, 2025 at 7:52 PM Hans Hagen
wrote: On 2/2/2025 10:56 AM, Udicoudco wrote:
Even though this change is less disruptive, there might be more concerns about backwards compatibility today than three years ago, if this is the case a new value to \matheqdirmode can be added.
I checked how I'd do it in luametatex and noticed a potential issue with the patch: because there can be more than one node (par plus plenty dir) you need to flush the list starting with vlink(head), otherwise you leak nodes.
(Because pop_nest is kind of dumb that should be enough but in other situations one should also set tail to head and the next node of head to null.)
(I actually added a bit more clever variant so that i can test some scenarios, but right now I can't forecast side effects as i have no clue about intended usage in macro packages as we don't need it in context.)
Oh, yes, a total oversight on my part... I intended to copy what is done in end_graf (which also suits the current behaviour that tests that vlink(head)==tail and flushes the tail, so flushing vlink(head) is the same).
wrong copy paste that worked for my too simplistic tests.
I'll test some more and send a new patch with the primitive and doc in a couple of hours.
I attached the patch with the new primitive, documentation, and the fix pointed by Hans (thanks for catching that early). Udi
On Sun, 2 Feb 2025 at 18:51, Hans Hagen
On 2/2/2025 10:56 AM, Udicoudco wrote:
Even though this change is less disruptive, there might be more concerns about backwards compatibility today than three years ago, if this is the case a new value to \matheqdirmode can be added.
I checked how I'd do it in luametatex and noticed a potential issue with the patch: because there can be more than one node (par plus plenty dir) you need to flush the list starting with vlink(head), otherwise you leak nodes.
(Because pop_nest is kind of dumb that should be enough but in other situations one should also set tail to head and the next node of head to null.)
(I actually added a bit more clever variant so that i can test some scenarios, but right now I can't forecast side effects as i have no clue about intended usage in macro packages as we don't need it in context.)
ok, finally done.
commit ba8a0978018c3215144c77ffb0c4f14638a3c1e1 (HEAD -> master,
origin/master, origin/HEAD)
Author: Luigi Scarso
participants (3)
-
Hans Hagen
-
luigi scarso
-
Udicoudco