Broken \adjdemerits on Linux due to unspecified enumeration type
Hi, current versions of LuaTeX from TeXLive for Linux (Windows is not affected) always add `\adjdemerits` when a line is more loose than the previous one, even if it is only one step more loose: Take \tracingparagraphs1 \tracingonline1 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa \bye This gives @firstpass []\tenrm aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @ via @0 b=96 p=0 d=1011236 @@1: line 1.1 t=21236 -> @0 aaaaaa @ via @0 b=83 p=0 d=8649 @@2: line 1.3 t=8649 -> @0 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aa aaaa aaaaaa @ via @1 b=3 p=0 d=169 @@3: line 2.2 t=1011405 -> @1 aaaaaa @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @\par via @3 b=0 p=-10000 d=100 @\par via @4 b=0 p=-10000 d=100 @@5: line 3.2- t=1008918 -> @4 Here, the relevant part is @@2: line 1.3 t=8649 -> @0 ... @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 So breakpoint @2 is .3 aka. "tight", while @4 is .2 aka. "decent". This break gives demerits d=100169=1000000+(10+3)^2. So \adjdemerits are inserted which is wrong because tight and decent lines are "visually compatible" as defined in the TeXbook. The problem is in linebreak.c. There the demerits are inserted by if (abs(fit_class - fitness(r)) > 1) d = d + adj_demerits; Both fit_class and fitness(r) have type fitness_value which is an enum with four entries. These enums have an implementation defined type and the bug is triggered if LuaTeX is compiled with a compiler which uses an unsigned type: Then `fit_class - fitness(r)` is a difference of unsigned values and therefore unsigned. If fitness(r) is bigger than fit_class, the attempt to store a negative difference in a unsigned value wraps around to a high positive value which is bigger than 1. So if fitness(r) is bigger than fit_class, the condition is always true. This can be fixed by explicitly casting fit_class and fitness(r) to a signed type. A possible patch is attached. Best regards, Marcel
On Wed, Oct 30, 2019 at 02:32:35AM +0100, Marcel Fabian Krüger wrote:
This can be fixed by explicitly casting fit_class and fitness(r) to a signed type. A possible patch is attached.
A second attempt to attach the patch.
Best regards,
Marcel _______________________________________________ dev-luatex mailing list dev-luatex@ntg.nl https://mailman.ntg.nl/mailman/listinfo/dev-luatex
On Wed, Oct 30, 2019 at 2:33 AM Marcel Fabian Krüger
Hi,
current versions of LuaTeX from TeXLive for Linux (Windows is not affected) always add `\adjdemerits` when a line is more loose than the previous one, even if it is only one step more loose:
Take
\tracingparagraphs1 \tracingonline1
aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa \bye
This gives
@firstpass []\tenrm aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @ via @0 b=96 p=0 d=1011236 @@1: line 1.1 t=21236 -> @0 aaaaaa @ via @0 b=83 p=0 d=8649 @@2: line 1.3 t=8649 -> @0 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aa aaaa aaaaaa @ via @1 b=3 p=0 d=169 @@3: line 2.2 t=1011405 -> @1 aaaaaa @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @\par via @3 b=0 p=-10000 d=100 @\par via @4 b=0 p=-10000 d=100 @@5: line 3.2- t=1008918 -> @4
Here, the relevant part is
@@2: line 1.3 t=8649 -> @0 ... @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2
So breakpoint @2 is .3 aka. "tight", while @4 is .2 aka. "decent". This break gives demerits d=100169=1000000+(10+3)^2. So \adjdemerits are inserted which is wrong because tight and decent lines are "visually compatible" as defined in the TeXbook.
The problem is in linebreak.c. There the demerits are inserted by
if (abs(fit_class - fitness(r)) > 1) d = d + adj_demerits;
Both fit_class and fitness(r) have type fitness_value which is an enum with four entries. These enums have an implementation defined type and the bug is triggered if LuaTeX is compiled with a compiler which uses an unsigned type: Then `fit_class - fitness(r)` is a difference of unsigned values and therefore unsigned. If fitness(r) is bigger than fit_class, the attempt to store a negative difference in a unsigned value wraps around to a high positive value which is bigger than 1. So if fitness(r) is bigger than fit_class, the condition is always true.
This can be fixed by explicitly casting fit_class and fitness(r) to a signed type. A possible patch is attached.
hm. I will see it. -- luigi
On Wed, Oct 30, 2019 at 2:33 AM Marcel Fabian Krüger
Hi,
current versions of LuaTeX from TeXLive for Linux (Windows is not affected) always add `\adjdemerits` when a line is more loose than the previous one, even if it is only one step more loose:
Take
\tracingparagraphs1 \tracingonline1
aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa \bye
This gives
@firstpass []\tenrm aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @ via @0 b=96 p=0 d=1011236 @@1: line 1.1 t=21236 -> @0 aaaaaa @ via @0 b=83 p=0 d=8649 @@2: line 1.3 t=8649 -> @0 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aa aaaa aaaaaa @ via @1 b=3 p=0 d=169 @@3: line 2.2 t=1011405 -> @1 aaaaaa @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @\par via @3 b=0 p=-10000 d=100 @\par via @4 b=0 p=-10000 d=100 @@5: line 3.2- t=1008918 -> @4
Here, the relevant part is
@@2: line 1.3 t=8649 -> @0 ... @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2
So breakpoint @2 is .3 aka. "tight", while @4 is .2 aka. "decent". This break gives demerits d=100169=1000000+(10+3)^2. So \adjdemerits are inserted which is wrong because tight and decent lines are "visually compatible" as defined in the TeXbook.
The problem is in linebreak.c. There the demerits are inserted by
if (abs(fit_class - fitness(r)) > 1) d = d + adj_demerits;
Both fit_class and fitness(r) have type fitness_value which is an enum with four entries. These enums have an implementation defined type and the bug is triggered if LuaTeX is compiled with a compiler which uses an unsigned type: Then `fit_class - fitness(r)` is a difference of unsigned values and therefore unsigned. If fitness(r) is bigger than fit_class, the attempt to store a negative difference in a unsigned value wraps around to a high positive value which is bigger than 1. So if fitness(r) is bigger than fit_class, the condition is always true.
This can be fixed by explicitly casting fit_class and fitness(r) to a signed type. A possible patch is attached.
Best regards,
Marcel _______________________________________________ dev-luatex mailing list dev-luatex@ntg.nl https://mailman.ntg.nl/mailman/listinfo/dev-luatex
$> mtxrun --script plain test.tex ( using lmroman10-regular.otf ) here is my log @firstpass []\tenrm aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @ via @0 b=97 p=0 d=21449 @@1: line 1.1 t=21449 -> @0 aaaaaa @ via @0 b=82 p=0 d=8464 @@2: line 1.3 t=8464 -> @0 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aa aaaa aaaaaa @ via @1 b=3 p=0 d=169 @@3: line 2.2 t=21618 -> @1 aaaaaa @ via @2 b=3 p=0 d=10169 @@4: line 2.2 t=18633 -> @2 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @\par via @3 b=0 p=-10000 d=100 @\par via @4 b=0 p=-10000 d=100 @@5: line 3.2- t=18733 -> @4 -- luigi
30. Oktober 2019 03:18, "luigi scarso"
$> mtxrun --script plain test.tex
( using lmroman10-regular.otf )
here is my log @firstpass []\tenrm aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @ via @0 b=97 p=0 d=21449 @@1: line 1.1 t=21449 -> @0 aaaaaa @ via @0 b=82 p=0 d=8464 @@2: line 1.3 t=8464 -> @0 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aa aaaa aaaaaa @ via @1 b=3 p=0 d=169 @@3: line 2.2 t=21618 -> @1 aaaaaa @ via @2 b=3 p=0 d=10169 @@4: line 2.2 t=18633 -> @2 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @\par via @3 b=0 p=-10000 d=100 @\par via @4 b=0 p=-10000 d=100 @@5: line 3.2- t=18733 -> @4
--luigi
Oh, I got a bit tired last night and copied the wrong version of the test file: You get the same values if you add \adjdemerits=1000000 at the beginning of test.tex I added this to make it more obvious that the demerits are really caused by adjdemerits and not some other kind of demerits. -- Marcel
participants (3)
-
luigi scarso -
Marcel Fabian Krüger -
Marcel Krüger