unexplained difference between luatex and luahbtex
Hello I am testing new luahbtex and comparing it with luatex because I am considering to set OpTeX with luahbtex engine instead luatex in TeXlive 2025. I have found a difference between lautex and luahbtex behavior which I am unable to explain why. The difference is shown only when huge number of fonts are downloaded by luaotfload and only with selected fonts from internet saved to ~/texmf/fonts. And it manifests only as a little delay (with luahbtex) during dlownloading the fonts to PDF at the end of the document visible if you are watching at the terminal. The PDF result is equal, but I am worried that there is a hidden bug in luahbtex because the difference is unexpected. I teseted this with OpTeX macros, they never use mode=harf. So I expect the same behavior with both engines. The example for testing is somewhat more complicated and it is exactly described at https://tex.stackexchange.com/questions/735071/ The main question is: why luatex behaves correctly (with no delay during downolading fonts) but luahbtex presents a delay in given cases? Where is the difference? Can the luatex developers explain this? Thank you Petr Olsak
On Sun, 19 Jan 2025 at 11:37, Petr Olsak
Hello
I am testing new luahbtex and comparing it with luatex because I am considering to set OpTeX with luahbtex engine instead luatex in TeXlive 2025. I have found a difference between lautex and luahbtex behavior which I am unable to explain why. The difference is shown only when huge number of fonts are downloaded by luaotfload and only with selected fonts from internet saved to ~/texmf/fonts. And it manifests only as a little delay (with luahbtex) during dlownloading the fonts to PDF at the end of the document visible if you are watching at the terminal. The PDF result is equal, but I am worried that there is a hidden bug in luahbtex because the difference is unexpected.
I teseted this with OpTeX macros, they never use mode=harf. So I expect the same behavior with both engines. The example for testing is somewhat more complicated and it is exactly described at
https://tex.stackexchange.com/questions/735071/
The main question is: why luatex behaves correctly (with no delay during downolading fonts) but luahbtex presents a delay in given cases? Where is the difference? Can the luatex developers explain this?
Thank you
Petr Olsak
under linux /usr/bin/time always reports luahbtex slower of about 5seconds, but I was not able to find why /where. -- luigi
On Sun, 19 Jan 2025 at 19:32, luigi scarso
On Sun, 19 Jan 2025 at 11:37, Petr Olsak
wrote: Hello
I am testing new luahbtex and comparing it with luatex because I am considering to set OpTeX with luahbtex engine instead luatex in TeXlive 2025. I have found a difference between lautex and luahbtex behavior which I am unable to explain why. The difference is shown only when huge number of fonts are downloaded by luaotfload and only with selected fonts from internet saved to ~/texmf/fonts. And it manifests only as a little delay (with luahbtex) during dlownloading the fonts to PDF at the end of the document visible if you are watching at the terminal. The PDF result is equal, but I am worried that there is a hidden bug in luahbtex because the difference is unexpected.
I teseted this with OpTeX macros, they never use mode=harf. So I expect the same behavior with both engines. The example for testing is somewhat more complicated and it is exactly described at
https://tex.stackexchange.com/questions/735071/
The main question is: why luatex behaves correctly (with no delay during downolading fonts) but luahbtex presents a delay in given cases? Where is the difference? Can the luatex developers explain this?
Thank you
Petr Olsak
under linux /usr/bin/time always reports luahbtex slower of about 5seconds, but I was not able to find why /where.
-- luigi
Using gprofng https://sourceware.org/binutils/wiki/gprofng I can see that luahbtex spends a relevant time on sweeplist, which means that the lua gc plays an important role here. -- luigi
Thank you very much for testing. I didn't use such tools. I am watching the terminal when fonts are downloaded (at the end of the document) and I notice the delay per each Besley fonts downloaded from ~/texmf. Sure, it is observable. It looks like waiting for some kind of time limit. Unfortunately, questions remains: why behavior of Lua garbage collector differs in luatex and luahbtex when the programs have to do the same? When we copy the fonts from ~/texmf/fonts to current directory: cp ~/texmf/fonts/opentype/besley/* . and run the test, then the unwanted delay with luahbtex disappears. It seems that this has something to do with file searching from TeXmf tree. Garbage collector behavior differs when files are in in different locations? Best regards Petr Olsak On 1/20/25 11:12, luigi scarso wrote:
On Sun, 19 Jan 2025 at 19:32, luigi scarso
wrote: On Sun, 19 Jan 2025 at 11:37, Petr Olsak
wrote: Hello
I am testing new luahbtex and comparing it with luatex because I am considering to set OpTeX with luahbtex engine instead luatex in TeXlive 2025. I have found a difference between lautex and luahbtex behavior which I am unable to explain why. The difference is shown only when huge number of fonts are downloaded by luaotfload and only with selected fonts from internet saved to ~/texmf/fonts. And it manifests only as a little delay (with luahbtex) during dlownloading the fonts to PDF at the end of the document visible if you are watching at the terminal. The PDF result is equal, but I am worried that there is a hidden bug in luahbtex because the difference is unexpected.
I teseted this with OpTeX macros, they never use mode=harf. So I expect the same behavior with both engines. The example for testing is somewhat more complicated and it is exactly described at
https://tex.stackexchange.com/questions/735071/
The main question is: why luatex behaves correctly (with no delay during downolading fonts) but luahbtex presents a delay in given cases? Where is the difference? Can the luatex developers explain this?
Thank you
Petr Olsak
under linux /usr/bin/time always reports luahbtex slower of about 5seconds, but I was not able to find why /where.
-- luigi
Using gprofng https://sourceware.org/binutils/wiki/gprofng I can see that luahbtex spends a relevant time on sweeplist, which means that the lua gc plays an important role here.
-- luigi
On 2025-01-20 at 15:26:47 +0100, Petr Olsak wrote:
Thank you very much for testing.
I didn't use such tools. I am watching the terminal when fonts are downloaded (at the end of the document) and I notice the delay per each Besley fonts downloaded from ~/texmf. Sure, it is observable. It looks like waiting for some kind of time limit.
Unfortunately, questions remains: why behavior of Lua garbage collector differs in luatex and luahbtex when the programs have to do the same?
When we copy the fonts from ~/texmf/fonts to current directory:
cp ~/texmf/fonts/opentype/besley/* .
and run the test, then the unwanted delay with luahbtex disappears. It seems that this has something to do with file searching from TeXmf tree. Garbage collector behavior differs when files are in in different locations?
To trace file searching you can set the environment variable KPATHSEA_DEBUG=32 Regards, Reinhard -- ------------------------------------------------------------------ Reinhard Kotucha Phone: +49-511-3373112 Marschnerstr. 25 D-30167 Hannover mailto:reinhard.kotucha@gmx.de ------------------------------------------------------------------
On Mon, 20 Jan 2025 at 23:39, Reinhard Kotucha
On 2025-01-20 at 15:26:47 +0100, Petr Olsak wrote:
Thank you very much for testing.
I didn't use such tools. I am watching the terminal when fonts are downloaded (at the end of the document) and I notice the delay per each Besley fonts downloaded from ~/texmf. Sure, it is observable. It looks like waiting for some kind of time limit.
Unfortunately, questions remains: why behavior of Lua garbage collector differs in luatex and luahbtex when the programs have to do the same?
When we copy the fonts from ~/texmf/fonts to current directory:
cp ~/texmf/fonts/opentype/besley/* .
and run the test, then the unwanted delay with luahbtex disappears. It seems that this has something to do with file searching from TeXmf tree. Garbage collector behavior differs when files are in in different locations?
To trace file searching you can set the environment variable
KPATHSEA_DEBUG=32
We can use the debug library \directlua{local oldgc=collectgarbage; collectgarbage("stop"); function collectgarbage(...); print(...); print(debug.traceback()); end} \fontfam[catalog] \bye but the debug library must be enabled at command line: luahbtex --luadebug One message looks suspicious, because it's "harf" stack traceback: [\directlua]:1: in function 'collectgarbage' ...mf-dist/tex/luatex/luaotfload/luaotfload-harf-define.lua:542: in function <...mf-dist/tex/luatex/luaotfload/luaotfload-harf-define.lua:539> (...tail calls...) luaotfload-harf-define.lua 539 luatexbase.add_to_callback('glyph_stream_provider', function(fid, cid, kind, ocid) 540 if cid == 0 then -- Always the first call for a font 541 glyph_stream_data, extents_hbfont = nil 542 collectgarbage() 543 local fontdir = font.getfont(fid) 544 if fontdir and fontdir.hb then 545 if kind == 3 then and if we comment line 542, we have the same as luatex. -- luigi
One answer by Michal Vlasak was given at https://tex.stackexchange.com/questions/735071/ This answer implies that the problem sits in luaotfload code, no in luahbtex binary. Thank you everyone for addressing this issue. Petr Olsak On 1/20/25 11:12, luigi scarso wrote:
On Sun, 19 Jan 2025 at 19:32, luigi scarso
wrote: On Sun, 19 Jan 2025 at 11:37, Petr Olsak
wrote: Hello
I am testing new luahbtex and comparing it with luatex because I am considering to set OpTeX with luahbtex engine instead luatex in TeXlive 2025. I have found a difference between lautex and luahbtex behavior which I am unable to explain why. The difference is shown only when huge number of fonts are downloaded by luaotfload and only with selected fonts from internet saved to ~/texmf/fonts. And it manifests only as a little delay (with luahbtex) during dlownloading the fonts to PDF at the end of the document visible if you are watching at the terminal. The PDF result is equal, but I am worried that there is a hidden bug in luahbtex because the difference is unexpected.
I teseted this with OpTeX macros, they never use mode=harf. So I expect the same behavior with both engines. The example for testing is somewhat more complicated and it is exactly described at
https://tex.stackexchange.com/questions/735071/
The main question is: why luatex behaves correctly (with no delay during downolading fonts) but luahbtex presents a delay in given cases? Where is the difference? Can the luatex developers explain this?
Thank you
Petr Olsak
under linux /usr/bin/time always reports luahbtex slower of about 5seconds, but I was not able to find why /where.
-- luigi
Using gprofng https://sourceware.org/binutils/wiki/gprofng I can see that luahbtex spends a relevant time on sweeplist, which means that the lua gc plays an important role here.
-- luigi
participants (3)
-
luigi scarso
-
Petr Olsak
-
Reinhard Kotucha