------------------------------------------------------------------------ r4742 | luigi | 2014-01-11 12:26:06 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /trunk/source/texk/web2c/luatexdir/luafontloader/fontforge/fontforge/parsettfatt.c unwanted side effects: font loading takes way longer due to linear checks and it makes no sense to do that for a bugged font; this can better be doen at the lua end if needed at all (hans) ------------------------------------------------------------------------ r4743 | luigi | 2014-01-11 12:29:30 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /branches/experimental/source/texk/web2c/luatexdir/luafontloader/fontforge/fontforge/parsettfatt.c sync experimental with trunk ------------------------------------------------------------------------ r4744 | luigi | 2014-01-11 12:42:36 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /trunk/source/texk/web2c/luatexdir/luafontloader/src/luafflib.c update do_ff_info ------------------------------------------------------------------------ r4745 | luigi | 2014-01-11 12:43:37 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /branches/experimental/source/texk/web2c/luatexdir/luafontloader/src/luafflib.c sync experimental with trunk ------------------------------------------------------------------------
On Sat, Jan 11, 2014 at 04:30:19PM +0100, Cron Daemon wrote:
------------------------------------------------------------------------ r4742 | luigi | 2014-01-11 12:26:06 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /trunk/source/texk/web2c/luatexdir/luafontloader/fontforge/fontforge/parsettfatt.c
unwanted side effects: font loading takes way longer due to linear checks and it makes no sense to do that for a bugged font; this can better be doen at the lua end if needed at all (hans)
Then you are better dropping glyph name guessing code completely because it is known to produce duplicate glyph names, having generic "glyphXXXX" names is better than having more than one glyph with the same name (the fonts are not bugged, but the original was). Regards, Khaled
Dear Khaled, since you were not included in the recent discussion I only wanted to briefly explain my part of the story of recent changes that took place in ConTeXt and in LuaTeX. They might be relevant for luaotfload as well, but if details are needed, Taco and Hans can probably explain better than I can. I first figured out that font database generation (which actually stores just the font name, filename and a few basic font properties, no actual info about glyphs or kerns or any other kind of huge data) took "forever". I admit that I'm "guilty" of not having used the latest ConTeXt for a very long time already and in TL 2013 it behaved acceptable, so I only noticed it about a week or two ago (but many Mac users kept asking about ways to disable system fonts on the ConTeXt mailing list recently). I'm not sure how long that "forever" processing of LastResort.ttf was because I gave up after about half an hour, but it could have been 40-60 minutes or so (I think the font generation job actually finished once in the background). It turned out that something weird was going on with the font LastResort.ttf (which is also blacklisted in luaotfload, I think, based on some comments on stackexchange claiming that processing that font takes a very long time). Then both Hans and Taco fixed a few bits both in the LuaTeX engine and in ConTeXt. This reduced the time to deal with this font to about 4 seconds I think. Next I found out that extracting the name from Arial Unicode alone took about 30 seconds. Luigi and Hans objected that my computer was ultra-slow and that doing the same on their machines with SSD took 3 seconds. Hans sent me a minimal lua script to mimick the basic operations done in font generation database and it turned out that the slow part of the code was opening and closing the font when trying to extract a few parameters. And just opening the Arial font without doing anything else took that 30 seconds, while doing the same with TeX Live 2013 took just 0.15 seconds or so. Then Luigi figured out which commit was responsible for that enormous time waste and this is exactly the reverted commit. (I don't know what the proper way to deal with this is, but ending up with 30 seconds time to solely open the font seems like an overkill to me. I didn't try to understand the code, but I strongly suspect that an O(n^2) algorithm was used where an O(n) could be written if the reverted code still seems valuable/usable. I mean: glyphNameExists could easily be implemented with O(1) or O(log n), there's no need for O(n).) I was still dissatisfied with 2-3 minutes font database generation time, given that ConTeXt spent about 3-4 seconds per certain Korean fonts (Mac ships with 24 of those) and some others, so LuaTeX was further modified to provide a few extra bits of meta font information, so that ConTeXt no longer had to open any font to get sufficient information. All in all this lead to reduction of time spent on font generation database from (probably) 40-60 minutes to merely 3 seconds on my so-called "old and slow" notebook from 2009. I'm not sure who takes care of luaotfload, but I believe this is something worth fixing there as well. Mojca On Sun, Jan 12, 2014 at 11:51 PM, Khaled Hosny wrote:
On Sat, Jan 11, 2014 at 04:30:19PM +0100, Cron Daemon wrote:
------------------------------------------------------------------------ r4742 | luigi | 2014-01-11 12:26:06 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /trunk/source/texk/web2c/luatexdir/luafontloader/fontforge/fontforge/parsettfatt.c
unwanted side effects: font loading takes way longer due to linear checks and it makes no sense to do that for a bugged font; this can better be doen at the lua end if needed at all (hans)
Then you are better dropping glyph name guessing code completely because it is known to produce duplicate glyph names, having generic "glyphXXXX" names is better than having more than one glyph with the same name (the fonts are not bugged, but the original was).
Regards, Khaled
I was going to write a lengthy rant on how choosing FontForge as a base for LuaTeX font loader was its biggest mistakes and how font and text layout support in LuaTeX is years behind the rest of the world because of this, but then decided not to write it :) I’m unlikely to able to convince Hans with anything in this area. Regards, Khaled On Mon, Jan 13, 2014 at 12:32:17AM +0100, Mojca Miklavec wrote:
Dear Khaled,
since you were not included in the recent discussion I only wanted to briefly explain my part of the story of recent changes that took place in ConTeXt and in LuaTeX. They might be relevant for luaotfload as well, but if details are needed, Taco and Hans can probably explain better than I can.
I first figured out that font database generation (which actually stores just the font name, filename and a few basic font properties, no actual info about glyphs or kerns or any other kind of huge data) took "forever". I admit that I'm "guilty" of not having used the latest ConTeXt for a very long time already and in TL 2013 it behaved acceptable, so I only noticed it about a week or two ago (but many Mac users kept asking about ways to disable system fonts on the ConTeXt mailing list recently).
I'm not sure how long that "forever" processing of LastResort.ttf was because I gave up after about half an hour, but it could have been 40-60 minutes or so (I think the font generation job actually finished once in the background). It turned out that something weird was going on with the font LastResort.ttf (which is also blacklisted in luaotfload, I think, based on some comments on stackexchange claiming that processing that font takes a very long time). Then both Hans and Taco fixed a few bits both in the LuaTeX engine and in ConTeXt. This reduced the time to deal with this font to about 4 seconds I think.
Next I found out that extracting the name from Arial Unicode alone took about 30 seconds. Luigi and Hans objected that my computer was ultra-slow and that doing the same on their machines with SSD took 3 seconds. Hans sent me a minimal lua script to mimick the basic operations done in font generation database and it turned out that the slow part of the code was opening and closing the font when trying to extract a few parameters. And just opening the Arial font without doing anything else took that 30 seconds, while doing the same with TeX Live 2013 took just 0.15 seconds or so. Then Luigi figured out which commit was responsible for that enormous time waste and this is exactly the reverted commit. (I don't know what the proper way to deal with this is, but ending up with 30 seconds time to solely open the font seems like an overkill to me. I didn't try to understand the code, but I strongly suspect that an O(n^2) algorithm was used where an O(n) could be written if the reverted code still seems valuable/usable. I mean: glyphNameExists could easily be implemented with O(1) or O(log n), there's no need for O(n).)
I was still dissatisfied with 2-3 minutes font database generation time, given that ConTeXt spent about 3-4 seconds per certain Korean fonts (Mac ships with 24 of those) and some others, so LuaTeX was further modified to provide a few extra bits of meta font information, so that ConTeXt no longer had to open any font to get sufficient information.
All in all this lead to reduction of time spent on font generation database from (probably) 40-60 minutes to merely 3 seconds on my so-called "old and slow" notebook from 2009. I'm not sure who takes care of luaotfload, but I believe this is something worth fixing there as well.
Mojca
On Sun, Jan 12, 2014 at 11:51 PM, Khaled Hosny wrote:
On Sat, Jan 11, 2014 at 04:30:19PM +0100, Cron Daemon wrote:
------------------------------------------------------------------------ r4742 | luigi | 2014-01-11 12:26:06 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /trunk/source/texk/web2c/luatexdir/luafontloader/fontforge/fontforge/parsettfatt.c
unwanted side effects: font loading takes way longer due to linear checks and it makes no sense to do that for a bugged font; this can better be doen at the lua end if needed at all (hans)
Then you are better dropping glyph name guessing code completely because it is known to produce duplicate glyph names, having generic "glyphXXXX" names is better than having more than one glyph with the same name (the fonts are not bugged, but the original was).
Regards, Khaled
On Mon, Jan 13, 2014 at 4:42 PM, Khaled Hosny wrote:
I was going to write a lengthy rant on how choosing FontForge as a base for LuaTeX font loader was its biggest mistakes and how font and text layout support in LuaTeX is years behind the rest of the world because of this, but then decided not to write it :) I’m unlikely to able to convince Hans with anything in this area.
But then again you could regard this as a new opportunity. Given your skill set by now I can imagine you taking over a project of adding HarfBuzz (or whatever library exactly you had in mind) support to LuaTeX. Years back (maybe in 2006 or 2007) we had discussions about whether LuaTeX and XeTeX were destined to be merged once in the future. Mojca
On Mon, Jan 13, 2014 at 10:05:50PM +0100, Mojca Miklavec wrote:
On Mon, Jan 13, 2014 at 4:42 PM, Khaled Hosny wrote:
I was going to write a lengthy rant on how choosing FontForge as a base for LuaTeX font loader was its biggest mistakes and how font and text layout support in LuaTeX is years behind the rest of the world because of this, but then decided not to write it :) I’m unlikely to able to convince Hans with anything in this area.
But then again you could regard this as a new opportunity.
Given your skill set by now I can imagine you taking over a project of adding HarfBuzz (or whatever library exactly you had in mind) support to LuaTeX.
I’d love to, but 1) I don’t have enough time for it right now 2) If ConTeXt is not going to use it, then it is just as useless to me. Regards, Khaled
On 1/12/2014 11:51 PM, Khaled Hosny wrote:
On Sat, Jan 11, 2014 at 04:30:19PM +0100, Cron Daemon wrote:
------------------------------------------------------------------------ r4742 | luigi | 2014-01-11 12:26:06 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /trunk/source/texk/web2c/luatexdir/luafontloader/fontforge/fontforge/parsettfatt.c
unwanted side effects: font loading takes way longer due to linear checks and it makes no sense to do that for a bugged font; this can better be doen at the lua end if needed at all (hans)
Then you are better dropping glyph name guessing code completely because it is known to produce duplicate glyph names, having generic "glyphXXXX" names is better than having more than one glyph with the same name (the fonts are not bugged, but the original was).
we'll have a look at it (using some bianry search instead); the old patch was quitting and not assigning a generic indexXXXX name either which was also wrong i think; in that case a_b_c_1 (first available subname) is better Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On 01/12/2014 11:51 PM, Khaled Hosny wrote:
On Sat, Jan 11, 2014 at 04:30:19PM +0100, Cron Daemon wrote:
------------------------------------------------------------------------ r4742 | luigi | 2014-01-11 12:26:06 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /trunk/source/texk/web2c/luatexdir/luafontloader/fontforge/fontforge/parsettfatt.c
unwanted side effects: font loading takes way longer due to linear checks and it makes no sense to do that for a bugged font; this can better be doen at the lua end if needed at all (hans)
Then you are better dropping glyph name guessing code completely because it is known to produce duplicate glyph names, having generic "glyphXXXX" names is better than having more than one glyph with the same name (the fonts are not bugged, but the original was).
Luatex is now back to how it behaved earlier (buggy, but with known workarounds) with a ticket that explains what happens and also why it is not patched right now: http://tracker.luatex.org/view.php?id=875 Switching to glyphXXXX seems best to me, but it would be a shame if that means loosing the ability to cut&paste from the PDF. Best wishes, Taco
On Mon, Jan 13, 2014 at 02:25:37PM +0100, Taco Hoekwater wrote:
On 01/12/2014 11:51 PM, Khaled Hosny wrote:
On Sat, Jan 11, 2014 at 04:30:19PM +0100, Cron Daemon wrote:
------------------------------------------------------------------------ r4742 | luigi | 2014-01-11 12:26:06 +0100 (Sat, 11 Jan 2014) | 1 line Changed paths: M /trunk/source/texk/web2c/luatexdir/luafontloader/fontforge/fontforge/parsettfatt.c
unwanted side effects: font loading takes way longer due to linear checks and it makes no sense to do that for a bugged font; this can better be doen at the lua end if needed at all (hans)
Then you are better dropping glyph name guessing code completely because it is known to produce duplicate glyph names, having generic "glyphXXXX" names is better than having more than one glyph with the same name (the fonts are not bugged, but the original was).
Luatex is now back to how it behaved earlier (buggy, but with known workarounds) with a ticket that explains what happens and also why it is not patched right now:
Since the part after the period is irrelevant (text extracting tools guessing code point from glyph name strip everything starting from the first period found), we can just generate a unique prefix each time regardless of the feature tag. Regards, Khaled
participants (5)
-
Hans Hagen -
Khaled Hosny -
Mojca Miklavec -
root＠mail.boekplan.nl -
Taco Hoekwater