Why searching fonts in xetex is so slow?

Yue Wang

4 Mar 2009 4 Mar '09

5:24 a.m.

Hi, When I use XeTeX, font searching through texmf- tree is very slow. search font through fontconfig is very fast. plain TeX or LaTeX does not have that problem. why is that? anyway to improve it? Yue Wang

Show replies by date

Hans Hagen

4 Mar 4 Mar

8:04 a.m.

Yue Wang wrote:

...

Hi,

When I use XeTeX, font searching through texmf- tree is very slow. search font through fontconfig is very fast. plain TeX or LaTeX does not have that problem. why is that? anyway to improve it?

afaik xetex perfers to look in the database first and then on the file system context on the other hand prefers to look for a file first if you don't know what "whatever" stands for, prefixing with file: or name: might help, as in file:whatever Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Yue Wang

9 Mar 9 Mar

2:14 a.m.

...

Exit code: 0

(1 run) On Wed, Mar 4, 2009 at 4:04 PM, Hans Hagen wrote:

...

Yue Wang wrote:

...
Hi,

When I use XeTeX, font searching through texmf- tree is very slow. search font through fontconfig is very fast. plain TeX or LaTeX does not have that problem. why is that? anyway to improve it?

afaik xetex perfers to look in the database first and then on the file system

context on the other hand prefers to look for a file first

if you don't know what "whatever" stands for, prefixing with file: or name: might help, as in file:whatever

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Hans Hagen

8:28 a.m.

Yue Wang wrote:

...

is that normal?

%engine=xetex \usetypescript[palatino] \setupbodyfont[palatino] \starttext $$\int_0^t$$ \stoptext

uses

(see the transcript file for additional information) Output written on test.pdf (1 page).

i get (c:/data/develop/context/sources/sort-def.tex) (c:/data/develop/context/sources/sort-lan.tex) [1.1]stdin -> test.pdf [1] 4770 bytes written systems : end file test at line 8 ) (see the transcript file for additional information) Output written on test.pdf (1 page). it's indeed not so fast, maybe because one of the used files cannot be found fast eventually we might end up with extending the xtx typescript Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Yue Wang

24 Apr 24 Apr

11:33 a.m.

Hi, Hans Now I know why XeTeX uses 1-2 minutes to compile a simple document: Each time first-setup.bat runs, it erase the user fontforge cache. So if I compile a document right after updating ConTeXt minimals, XeTeX will automatically run fc-cache to generate the font cache. Today I test a little bit, the time used for fc-cache -r is approximately the xetex's first running time. Next compilation uses about 7-8 seconds, which is reasonable (it first searches the fc cache, then the texmf tree) btw, can we change the type-otf.tex to force mkii to search fonts like iwona and tex gyre in the texmf tree first? Then the compilation will run much faster than before. Yue Wang On Mon, Mar 9, 2009 at 4:28 PM, Hans Hagen wrote:

...

Yue Wang wrote:

...
is that normal?

%engine=xetex \usetypescript[palatino] \setupbodyfont[palatino] \starttext $$\int_0^t$$ \stoptext

uses

(see the transcript file for additional information) Output written on test.pdf (1 page).

i get

(c:/data/develop/context/sources/sort-def.tex) (c:/data/develop/context/sources/sort-lan.tex) [1.1]stdin -> test.pdf [1] 4770 bytes written

systems : end file test at line 8 ) (see the transcript file for additional information) Output written on test.pdf (1 page).

it's indeed not so fast, maybe because one of the used files cannot be found fast

eventually we might end up with extending the xtx typescript

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Mojca Miklavec

8:27 p.m.

On Fri, Apr 24, 2009 at 13:33, Yue Wang wrote:

...

Hi, Hans

Now I know why XeTeX uses 1-2 minutes to compile a simple document: Each time first-setup.bat runs, it erase the user fontforge cache. So if I compile a document right after updating ConTeXt minimals, XeTeX will automatically run fc-cache to generate the font cache.

Today I test a little bit, the time used for fc-cache -r is approximately the xetex's first running time.

True, XeTeX runs fc-cache every now and then, but I didn't manage to figure out when that happens. I would suggest to ask about that on the XeTeX mailing list. ConTeXt has zero influence on that behaviour, except that it could avoid searching via fc.

...

Next compilation uses about 7-8 seconds, which is reasonable (it first searches the fc cache, then the texmf tree) btw, can we change the type-otf.tex to force mkii to search fonts like iwona and tex gyre in the texmf tree first?

Iwona? There are lines like \definefontsynonym [Iwona-Regular] [file:Iwona-Regular] [features=default] which means that it first searches on texmf tree, but I suspect that math fonts are being searched via fontconfig first. The way font searching is implemented now ([rm-iwonar] will look for \font\something="rm-iwonar" instead of \font\something=rm-iwona => this means searching via fontconfig first; the font is not found there, so it searches in texmf tree further) it is not possible to convince ConTeXt to search in texmf tree first unless Hans changes core macros (the part that has been broken for almost a year). Mojca

Hans Hagen

8:57 p.m.

Mojca Miklavec wrote:

...

On Fri, Apr 24, 2009 at 13:33, Yue Wang wrote:

...
Hi, Hans

Now I know why XeTeX uses 1-2 minutes to compile a simple document: Each time first-setup.bat runs, it erase the user fontforge cache. So if I compile a document right after updating ConTeXt minimals, XeTeX will automatically run fc-cache to generate the font cache.

Today I test a little bit, the time used for fc-cache -r is approximately the xetex's first running time.

True, XeTeX runs fc-cache every now and then, but I didn't manage to figure out when that happens. I would suggest to ask about that on the XeTeX mailing list. ConTeXt has zero influence on that behaviour, except that it could avoid searching via fc.

...
Next compilation uses about 7-8 seconds, which is reasonable (it first searches the fc cache, then the texmf tree) btw, can we change the type-otf.tex to force mkii to search fonts like iwona and tex gyre in the texmf tree first?

Iwona? There are lines like \definefontsynonym [Iwona-Regular] [file:Iwona-Regular] [features=default] which means that it first searches on texmf tree, but I suspect that math fonts are being searched via fontconfig first. The way font searching is implemented now ([rm-iwonar] will look for \font\something="rm-iwonar" instead of \font\something=rm-iwona => this means searching via fontconfig first; the font is not found there, so it searches in texmf tree further) it is not possible to convince ConTeXt to search in texmf tree first unless Hans changes core macros (the part that has been broken for almost a year).

broken? well. the " is there because fonts can have spaces and i'm bot going to parse for that Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Mojca Miklavec

9:17 p.m.

On Fri, Apr 24, 2009 at 22:57, Hans Hagen wrote:

...

Mojca Miklavec wrote:

...
Iwona? There are lines like \definefontsynonym [Iwona-Regular] [file:Iwona-Regular] [features=default] which means that it first searches on texmf tree, but I suspect that math fonts are being searched via fontconfig first. The way font searching is implemented now ([rm-iwonar] will look for \font\something="rm-iwonar" instead of \font\something=rm-iwona => this means searching via fontconfig first; the font is not found there, so it searches in texmf tree further) it is not possible to convince ConTeXt to search in texmf tree first unless Hans changes core macros (the part that has been broken for almost a year).

broken?

Sorry, wrong tense. I meant "had been broken" (several years ago, risky to touch again :). Mojca

Yue Wang

25 Apr 25 Apr

1:44 a.m.

Hi, Mojca: you are absolutely right! When load iwona with the following two typeface, the loading time takes up 7-8 seconds. TeXExec | runtime: 8.296 \definetypeface [iwona] [ss] [sans] [iwona] [default] [encoding=ec] \definetypeface [iwona] [mm] [math] [iwona] [default] [encoding=ec] \setupbodyfont[iwona,ss,12pt] \starttext \input zapf $$\int_a^b f(x) d x = 0$$ \stoptext However, when comment the second line (\definetypeface [iwona] [mm] [math] [iwona] [default] [encoding=ec]), TeXExec | runtime: 1.828 total compile time is reduced to only 2 seconds. (of course, equations are typeset in latin modern). How can I speed up math font loading? change add file: suffix (like [file:rm-iwonar]) won't help :( Yue Wang On Sat, Apr 25, 2009 at 4:27 AM, Mojca Miklavec wrote:

...

On Fri, Apr 24, 2009 at 13:33, Yue Wang wrote:

...
Hi, Hans

Now I know why XeTeX uses 1-2 minutes to compile a simple document: Each time first-setup.bat runs, it erase the user fontforge cache. So if I compile a document right after updating ConTeXt minimals, XeTeX will automatically run fc-cache to generate the font cache.

Today I test a little bit, the time used for fc-cache -r is approximately the xetex's first running time.

True, XeTeX runs fc-cache every now and then, but I didn't manage to figure out when that happens. I would suggest to ask about that on the XeTeX mailing list. ConTeXt has zero influence on that behaviour, except that it could avoid searching via fc.

...
Next compilation uses about 7-8 seconds, which is reasonable (it first searches the fc cache, then the texmf tree) btw, can we change the type-otf.tex to force mkii to search fonts like iwona and tex gyre in the texmf tree first?

Iwona? There are lines like \definefontsynonym [Iwona-Regular] [file:Iwona-Regular] [features=default] which means that it first searches on texmf tree, but I suspect that math fonts are being searched via fontconfig first. The way font searching is implemented now ([rm-iwonar] will look for \font\something="rm-iwonar" instead of \font\something=rm-iwona => this means searching via fontconfig first; the font is not found there, so it searches in texmf tree further) it is not possible to convince ConTeXt to search in texmf tree first unless Hans changes core macros (the part that has been broken for almost a year).

Mojca ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Mojca Miklavec

9:39 a.m.

On Sat, Apr 25, 2009 at 03:44, Yue Wang wrote:

...

Hi, Mojca:

However, when comment the second line (\definetypeface [iwona] [mm] [math] [iwona] [default] [encoding=ec]), TeXExec | runtime: 1.828 total compile time is reduced to only 2 seconds. (of course, equations are typeset in latin modern).

But LM should suffer from the same problem.

...

How can I speed up math font loading? change add file: suffix (like [file:rm-iwonar]) won't help :(

Core macros need to be changed somehow if you want to achieve that. Hans doesn't use XeTeX, so you need strong arguments to convince him to modify XeTeX-related code. Hans wanted to implement \definefontsynonym [a] [b] to automatically work whether b is a font name, font filename or the old good tfm font. Even though the idea sounds OK and works well in LuaTeX, there's a big problem with it, namely, it becomes so annoyingly slow that it's basically useless (just for fun, you can try to remove file: in front of LM fonts and observe the consequences). Usually filename and font name don't match anyway, so you definitely need to specify which one you want to use. But there's one further problem. You can now specify file: or name: prefix. In the first case this translates to \font\a=[b] and in the second case it translates to \font\a="b" For tfm fonts you however need \font\a=b The second form with "b" works as well, but in that case XeTeX does internal search by first asking fc and only if that fails it asks kpathsea. If you don't provide either file: or name: then ConTeXt needs to check all the three different options in the following order: \font\a="b" \font\a=b \font\a=[b] I forgot whether the first one is already accepted (maybe it is), but it any case it takes a lot of time before the right font is found. Ideally ConTeXt should at least switch the order of searching for fonts and should try kpatsea before fc, but then one runs into problems with font names with spaces if one doesn't explicitely provide the name: prefix. The current clever scheme causes more problems & longer load times than it brings advantages, but unless Hans changes core macros, there's not much that you can do about it to speed up the loading time. Mojca

Yue Wang

11:18 a.m.

Hi, On Sat, Apr 25, 2009 at 5:39 PM, Mojca Miklavec wrote:

...

On Sat, Apr 25, 2009 at 03:44, Yue Wang wrote:

...
Hi, Mojca:

However, when comment the second line (\definetypeface [iwona] [mm] [math] [iwona] [default] [encoding=ec]), TeXExec | runtime: 1.828 total compile time is reduced to only 2 seconds. (of course, equations are typeset in latin modern).

But LM should suffer from the same problem.

I think LM tfm/otf metrics information are dumped to fmt statically. See the TeX82 source code (don't know xetex's situation, but i am sure tex82 works like that). Yue Wang

Mojca Miklavec

11:49 a.m.

On Sat, Apr 25, 2009 at 13:18, Yue Wang wrote:

...

...
But LM should suffer from the same problem.

I think LM tfm/otf metrics information are dumped to fmt statically. See the TeX82 source code (don't know xetex's situation, but i am sure tex82 works like that).

That's possible. But if you remove file: prefix in LM font definitions then it will take forever to compile a hello-world file. (I cannot explain that, but it doesn't really matter.) Mojca

Hans Hagen

12:43 p.m.

Yue Wang wrote:

...

Hi,

On Sat, Apr 25, 2009 at 5:39 PM, Mojca Miklavec wrote:

...
On Sat, Apr 25, 2009 at 03:44, Yue Wang wrote:

...
Hi, Mojca:

However, when comment the second line (\definetypeface [iwona] [mm] [math] [iwona] [default] [encoding=ec]), TeXExec | runtime: 1.828 total compile time is reduced to only 2 seconds. (of course, equations are typeset in latin modern). But LM should suffer from the same problem.

I think LM tfm/otf metrics information are dumped to fmt statically. See the TeX82 source code (don't know xetex's situation, but i am sure tex82 works like that).

context never dumps fonts in the format Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Yue Wang

1:33 p.m.

I see, no font is defined when dumping the formats... pretty unlike plain TeX... then why so much time is used for iwona, while only 1.2 sec is used to load lm roman? On Sat, Apr 25, 2009 at 8:43 PM, Hans Hagen wrote:

...

Yue Wang wrote:

...
Hi,

On Sat, Apr 25, 2009 at 5:39 PM, Mojca Miklavec wrote:

...
On Sat, Apr 25, 2009 at 03:44, Yue Wang wrote:

...
Hi, Mojca:

However, when comment the second line (\definetypeface [iwona] [mm] [math] [iwona] [default] [encoding=ec]), TeXExec | runtime: 1.828 total compile time is reduced to only 2 seconds. (of course, equations are typeset in latin modern).

But LM should suffer from the same problem.

I think LM tfm/otf metrics information are dumped to fmt statically. See the TeX82 source code (don't know xetex's situation, but i am sure tex82 works like that).

context never dumps fonts in the format

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Hans Hagen

1:41 p.m.

Yue Wang wrote:

...

I see, no font is defined when dumping the formats... pretty unlike plain TeX... then why so much time is used for iwona, while only 1.2 sec is used to load lm roman?

see previous mail ... because xetex rebuilds the fc cache each time; so it's not related to context as all name lookups suffer from the same Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Yue Wang

2:07 p.m.

Hi, Hans On Sat, Apr 25, 2009 at 9:41 PM, Hans Hagen wrote:

...

Yue Wang wrote:

...
I see, no font is defined when dumping the formats... pretty unlike plain TeX... then why so much time is used for iwona, while only 1.2 sec is used to load lm roman?

see previous mail ... because xetex rebuilds the fc cache each time; so it's not related to context as all name lookups suffer from the same

then if it tries to load lm, it will build fc cache too, why is the loading time too short? in short, can you please explain to us why %engine=xetex \usetypescript[modern] [ec] \switchtobodyfont[modern, 10pt] \starttext \input zapf \stoptext uses TeXExec | runtime: 1.577 while %engine=xetex \usetypescript[palatino] [ec] \switchtobodyfont[palatino, 10pt] \starttext \input zapf \stoptext uses TeXExec | runtime: 6.469 (most time) and even sometime uses TeXExec | runtime: 40.806 (disk is busy, maybe due to font caching ...) what's the difference between lm and palatino? (I think lm contains more fonts.....) Yue Wang

...

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Mojca Miklavec

2:09 p.m.

...

in short, can you please explain to us why

%engine=xetex

\usetypescript[modern] [ec]

Once you explain why you use [ec] ... Mojca

Mojca Miklavec

2:13 p.m.

On Sat, Apr 25, 2009 at 16:09, Mojca Miklavec wrote:

...

...
in short, can you please explain to us why

%engine=xetex

\usetypescript[modern] [ec]

Once you explain why you use [ec] ...

I thought it made a difference, but I indeed don't see any timing difference when [ec] is added, but you need to bear in mind that LM is loaded no matter whether you use it or if you use some other font. I suspect that 40 seconds indeed come from fc-cache, but you need to ask Jonathan or Akira about details. Running fc-cache is completely out of ConTeXt's control. Mojca

Yue Wang

2:13 p.m.

Hi, Mojca:

...

Once you explain why you use [ec] ...

Hans uses [ec] in the previous mail. anyway, %engine=xetex \usetypescript[palatino] \setupbodyfont[palatino, 10pt] \starttext \input zapf \stoptext uses TeXExec | runtime: 6.516 while %engine=xetex \usetypescript[modern] \setupbodyfont[modern, 10pt] \starttext \input zapf \stoptext uses TeXExec | runtime: 1.531 Yue Wang

Hans Hagen

4:18 p.m.

Mojca Miklavec wrote:

...

...
in short, can you please explain to us why

%engine=xetex

\usetypescript[modern] [ec]

Once you explain why you use [ec] ...

because i also run the test for pdftex -) ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Mojca Miklavec

2:15 p.m.

On Sat, Apr 25, 2009 at 16:07, Yue Wang wrote:

...

Hi, Hans

in short, can you please explain to us why

%engine=xetex

\usetypescript[modern] [ec] \switchtobodyfont[modern, 10pt] \starttext \input zapf \stoptext

uses TeXExec | runtime: 1.577

while

%engine=xetex \usetypescript[palatino] [ec] \switchtobodyfont[palatino, 10pt] \starttext \input zapf \stoptext

uses TeXExec | runtime: 6.469 (most time)

and even sometime uses TeXExec | runtime: 40.806 (disk is busy, maybe due to font caching ...)

what's the difference between lm and palatino? (I think lm contains more fonts.....)

Out of curiosity: does timing change if you delete everything but iwona from type-otf.mkii? Mojca

Yue Wang

2:31 p.m.

Hi,

...

...
what's the difference between lm and palatino? (I think lm contains more fonts.....)

Out of curiosity: does timing change if you delete everything but iwona from type-otf.mkii?

maybe just set cont-sys.rme, and enable palatino there, then the palatino loading time can be very short. Yue Wang

Hans Hagen

4:19 p.m.

Yue Wang wrote:

...

Hi,

...
...
what's the difference between lm and palatino? (I think lm contains more fonts.....) Out of curiosity: does timing change if you delete everything but iwona from type-otf.mkii?

maybe just set cont-sys.rme, and enable palatino there, then the palatino loading time can be very short.

loaded at runtime anyway so unrelated as mentioned several times before now ... it has nothing to do with the speed of loading a font metric file or parsing typescripts but all with the font cache ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Mojca Miklavec

6:06 p.m.

On Sat, Apr 25, 2009 at 18:19, Hans Hagen wrote:

...

Yue Wang wrote:

...
maybe just set cont-sys.rme, and enable palatino there, then the palatino loading time can be very short.

loaded at runtime anyway so unrelated

as mentioned several times before now ... it has nothing to do with the speed of loading a font metric file or parsing typescripts but all with the font cache

There was indeed a difference here. Total runtime when loading palatino has decreased from 3 to 1.3 seconds if I made the change. But I cannot tell if during those two additional seconds fc cache has been regenerated. Mojca

Hans Hagen

6:17 p.m.

Mojca Miklavec wrote:

...

On Sat, Apr 25, 2009 at 18:19, Hans Hagen wrote:

...
Yue Wang wrote:

...
maybe just set cont-sys.rme, and enable palatino there, then the palatino loading time can be very short. loaded at runtime anyway so unrelated

as mentioned several times before now ... it has nothing to do with the speed of loading a font metric file or parsing typescripts but all with the font cache

There was indeed a difference here. Total runtime when loading palatino has decreased from 3 to 1.3 seconds if I made the change.

But I cannot tell if during those two additional seconds fc cache has been regenerated.

it also depends on how ths OS caches files in mem (if you have enough of it) and the size of the tree (quit esome time is spent on the many real huge fonts that i have on my machine) ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Hans Hagen

4:18 p.m.

Mojca Miklavec wrote:

...

Out of curiosity: does timing change if you delete everything but iwona from type-otf.mkii?

Hans Hagen

4:17 p.m.

Yue Wang wrote:

...

Hi, Hans

On Sat, Apr 25, 2009 at 9:41 PM, Hans Hagen wrote:

...
Yue Wang wrote:

...
I see, no font is defined when dumping the formats... pretty unlike plain TeX... then why so much time is used for iwona, while only 1.2 sec is used to load lm roman? see previous mail ... because xetex rebuilds the fc cache each time; so it's not related to context as all name lookups suffer from the same

then if it tries to load lm, it will build fc cache too, why is the loading time too short?

in short, can you please explain to us why

name: uses the font database with file system fallback file: uses the file system with fontname fallback when the font cache is ok there is no difference in speed; ha snothign to do with the font or context (in mkiv there is a bit more clever searching going on) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Mojca Miklavec

2:08 p.m.

On Sat, Apr 25, 2009 at 15:33, Yue Wang wrote:

...

I see, no font is defined when dumping the formats... pretty unlike plain TeX... then why so much time is used for iwona, while only 1.2 sec is used to load lm roman?

I have a bit of a problem to test all that since running fc-cache only takes two seconds. A run with iwona takes 2.1 seconds (2.4 when some math equations are added) and a run with LM takes 1.1 seconds. The timings refer to subsequent runs, not to the first run. Maybe I should try to experiment on Windows. I have no idea when XeTeX forces reload of fc-cache, but I suspect that any other application does the same if font database becomes outdated. I didn't notice subsequent runs of fc-cache though. Once fc-cache is through, it doesn't run for a while. Mojca

Hans Hagen

4:15 p.m.

Mojca Miklavec wrote:

...

On Sat, Apr 25, 2009 at 15:33, Yue Wang wrote:

...
I see, no font is defined when dumping the formats... pretty unlike plain TeX... then why so much time is used for iwona, while only 1.2 sec is used to load lm roman?

I have a bit of a problem to test all that since running fc-cache only takes two seconds.

here fc-cache takes minutes

...

A run with iwona takes 2.1 seconds (2.4 when some math equations are added) and a run with LM takes 1.1 seconds. The timings refer to subsequent runs, not to the first run.

lm is faster because the definitions are preloaded (not the fonts themselves but the typescript defs)

...

Maybe I should try to experiment on Windows. I have no idea when XeTeX forces reload of fc-cache, but I suspect that any other application does the same if font database becomes outdated. I didn't notice subsequent runs of fc-cache though. Once fc-cache is through, it doesn't run for a while.

here it runs *each* xetex run ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Yue Wang

2:41 p.m.

Hi,

...

context never dumps fonts in the format

there is one line in cont-en.tex: \setupencoding[default=ec] \usetypescript[fallback][\defaultencoding] \setupbodyfont[rm,12pt] also in context.tex: \setupencoding[default=ec] \usetypescript[fallback][\defaultencoding] \setupbodyfont[rm,12pt] the fallback typescript are defined in type-otf.tex: \starttypescript [fallback] \definetypeface [] [rm] [serif] [modern] [computer-modern] \definetypeface [] [ss] [sans] [modern] [computer-modern] \definetypeface [] [tt] [mono] [modern] [computer-modern] \definetypeface [] [mm] [math] [modern] [computer-modern] \quittypescriptscanning \stoptypescript so when the formats are dumped, all lm fonts are loaded? (I don't know whether at this stage the actual tfms are loaded...) Yue Wang

Hans Hagen

4:21 p.m.

Yue Wang wrote:

...

Hi,

...
context never dumps fonts in the format

there is one line in cont-en.tex: \setupencoding[default=ec] \usetypescript[fallback][\defaultencoding] \setupbodyfont[rm,12pt]

also in context.tex: \setupencoding[default=ec] \usetypescript[fallback][\defaultencoding] \setupbodyfont[rm,12pt]

the fallback typescript are defined in type-otf.tex: \starttypescript [fallback] \definetypeface [] [rm] [serif] [modern] [computer-modern] \definetypeface [] [ss] [sans] [modern] [computer-modern] \definetypeface [] [tt] [mono] [modern] [computer-modern] \definetypeface [] [mm] [math] [modern] [computer-modern] \quittypescriptscanning \stoptypescript

so when the formats are dumped, all lm fonts are loaded?

(I don't know whether at this stage the actual tfms are loaded...)

only font definitions are in the format (for the modern typeface) but no fonts are loaded; context loads on demands ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Taco Hoekwater

12:14 p.m.

Mojca Miklavec wrote:

...

The current clever scheme causes more problems & longer load times than it brings advantages, but unless Hans changes core macros, there's not much that you can do about it to speed up the loading time.

I assume Context could add support for an explicit tfm: prefix, that would likely solve most issues. But one could also ask Jonathan to make XeTeX accept a less terse and idiosyncratic font syntax. As it stands you can't feed an arbitrary font name (which may contain spaces) into the non quoted syntax (because then the quoted syntax is actually required for tfm loading also), but by adding the quotes the system fonts are suddenly searched first. This makes it harder than needed to support both old and new font formats at the same time. Best wishes, Taco

Hans Hagen

1:27 p.m.

Mojca Miklavec wrote:

...

Hans wanted to implement \definefontsynonym [a] [b] to automatically work whether b is a font name, font filename or the old good tfm font.

Even though the idea sounds OK and works well in LuaTeX, there's a big problem with it, namely, it becomes so annoyingly slow that it's basically useless (just for fun, you can try to remove file: in front

huh? resolving synonyms is pretty fast \starttext \definefontsynonym[mojca][is wrong] \testfeatureonce{10000}{\xdef\test{\truefontname{mojca}}} mojca \test \stoptext and the overhead in the loop probably takes most time

...

I forgot whether the first one is already accepted (maybe it is), but it any case it takes a lot of time before the right font is found.

i think that when xetex had no tested we were forced to use batchmode which then resulted in many logs messages but that's long ago

...

The current clever scheme causes more problems & longer load times than it brings advantages, but unless Hans changes core macros, there's not much that you can do about it to speed up the loading time.

well, the problem is that xetex uses this mixture of [whatever] and "whatever" and "[whatever]" syntax which might be nice for latex but as context uses [] for optional args it complicates things an option is to make special typescript files for xetex (which is not a bad idea because then we can optimize the otf typescripts for luatex usage) so, it must be some other problem ... \starttext \switchtobodyfont[10pt] \input tufte \endgraf \switchtobodyfont[11pt] \input tufte \endgraf \switchtobodyfont[12pt] \input tufte \endgraf \stoptext gives and a texexec run of 2.5 seconds and \dorecurse{1000}{ \switchtobodyfont[10pt] mojca \switchtobodyfont[11pt] is \switchtobodyfont[12pt] wrong } takes 10 seconds ok it takes 3.5 seconds in mkiv but there we have a more optimal font system due to less fonts needed; pdftex takes 9 seconds, so the difference between mkii pdftex/xetex is 1 second which is mostly due to the fact that xetex is unicode, uses larger fonts, does otf etc etc now, the timing for \usetypescript[iwona] [ec] \usetypescript[palatino] [ec] \usetypescript[postscript][ec] \usetypescript[modern] [ec] \starttext \switchtobodyfont[iwona, 10pt] mojca \switchtobodyfont[iwona, 11pt] is \switchtobodyfont[iwona, 12pt] wrong \switchtobodyfont[palatino, 10pt] mojca \switchtobodyfont[palatino, 11pt] is \switchtobodyfont[palatino, 12pt] wrong \switchtobodyfont[postscript,10pt] mojca \switchtobodyfont[postscript,11pt] is \switchtobodyfont[postscript,12pt] wrong \switchtobodyfont[modern, 10pt] mojca \switchtobodyfont[modern, 11pt] is \switchtobodyfont[modern, 12pt] wrong \stoptext is as follows: pdftex: 2.4s, luatex 4.3s, xetex 11.7s now, interesting is that if we only load one of them xetex needs 9.5 seconds, so there seems to be quite some initialization time involved so, i looked into this caching stuff and found out that fc-cache -v first deletes an invalid cache and on a next run tries to make a cache and fails which then is the reason why *each* xetex run needs to build it sown cache (which is quite visible with some filehandle tracing tool) i noticed that there is also a failed message when caching c:\windows\fonts so maybe fc-cache is messed up anyhow, it's not a context issue at all, just caching Hans Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Mojca Miklavec

24 Apr 24 Apr

8:29 p.m.

On Fri, Apr 24, 2009 at 13:33, Yue Wang wrote:

...

Hi, Hans

Each time first-setup.bat runs, it erase the user fontforge cache.

Where is that cache located? (Are you talking about windows?) Maybe we could prevent deleting the font cache. This might solve your problem. Mojca

Yue Wang

25 Apr 25 Apr

1:08 a.m.

On Sat, Apr 25, 2009 at 4:29 AM, Mojca Miklavec wrote:

...

On Fri, Apr 24, 2009 at 13:33, Yue Wang wrote:

...
Hi, Hans

Each time first-setup.bat runs, it erase the user fontforge cache.

Where is that cache located? (Are you talking about windows?) Maybe we could prevent deleting the font cache. This might solve your problem.

I think it is in the C:\context\tex\texmf-mswin\fonts\cache\

...

Mojca ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Mojca Miklavec

11:45 a.m.

On Sat, Apr 25, 2009 at 03:08, Yue Wang wrote:

...

On Sat, Apr 25, 2009 at 4:29 AM, Mojca Miklavec wrote:

...
Where is that cache located? (Are you talking about windows?) Maybe we could prevent deleting the font cache. This might solve your problem.

I think it is in the C:\context\tex\texmf-mswin\fonts\cache\

Oh, right! It seemed weird to me when you were talking about font cache being deleted since I thouht you were using FreeBSD. What you can do temporary is to add --keep option to first-setup.bat, but I would like to fix the issue anyway. Hans, can you please apply the following fix: local normalflags = states.get("rsync.flags.normal") local deleteflags = "" if (destination:find("texmf$") or destination:find("context$")) and (not environment.argument("keep")) then deleteflags = states.get("rsync.flags.delete") end command = format("%s %s %s %s'%s' '%s'", bin, normalflags, deleteflags, url, archives, destination) logs.report("mtx update", format("running command: %s",command)) Mojca

5720

Age (days ago)

5772

Last active (days ago)

List overview

Download

35 comments

4 participants

participants (4)

Hans Hagen
Mojca Miklavec
Taco Hoekwater
Yue Wang

Why searching fonts in xetex is so slow?

tags

participants (4)