Can we make font loading much faster?
Hi: The following cases all use second run of context. So there is no font cache time. I use a new installed Windows XP SP3 on a Core 2 Duo 1.68GHz with 1GB mem machine. Only TeXWorks and ConTeXt are installed and running on the machine. I close all pre-installed software (like MSN, IE, etc.) and leave all extra memory for LuaTeX. Only 3 big chinese font, AdobeSongStd-Light, AdobeHeitiStd-Regular, AdobeKaitiStd-Regular are loaded. However, different approaches have different runtime, here, two cases are tested (zhfonts is attached in the mail): \usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext 你好 {\it 你好} {\bf 你好} \stoptext MtxRun | total runtime: 194.156 \definefontfeature[zh][mode=node, script=hang, lang=zhs] \definefontsynonym[song][AdobeSongStd-Light][features=zh] \definefont[song][song at 12pt] \definefontsynonym[hei][AdobeHeitiStd-Regular][features=zh] \definefont[hei][hei at 12pt] \definefontsynonym[kai][AdobeKaitiStd-Regular][features=zh] \definefont[kai][kai at 12pt] \starttext {\song 你好}{\hei 你好}{\kai 你好} \stoptext MtxRun | total runtime: 13.797 This also applies to file which only one big font, AdobeSongStd-Light, is used, two cases are tested: \definefontfeature[zh][mode=node, script=hang, lang=zhs] \definefontsynonym[song][AdobeSongStd-Light][features=zh] \definefont[song][song at 12pt] \definefontsynonym[hei][AdobeHeitiStd-Regular][features=zh] \definefont[hei][hei at 12pt] \definefontsynonym[kai][AdobeKaitiStd-Regular][features=zh] \definefont[kai][kai at 12pt] \starttext {\song 你好} \stoptext MtxRun | total runtime: 5.609 \usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext 你好 \stoptext MtxRun | total runtime: 135.125 If no Chinese fonts are used, three cases are tested: \usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext Hello \stoptext MtxRun | total runtime: 89.938 \starttext Hello \stoptext MtxRun | total runtime: 2.594 \definefontfeature[zh][mode=node, script=hang, lang=zhs] \definefontsynonym[song][AdobeSongStd-Light][features=zh] \definefont[song][song at 12pt] \definefontsynonym[hei][AdobeHeitiStd-Regular][features=zh] \definefont[hei][hei at 12pt] \definefontsynonym[kai][AdobeKaitiStd-Regular][features=zh] \definefont[kai][kai at 12pt] \starttext Hello \stoptext MtxRun | total runtime: 2.734 In a typical chinese science book, usually 4 to 10 chinese truetype/opentype font are used. Magazines might use twenty or more. So if we want to use the standard way (use chinese font as body font, use virtual fonts mechanism to mix the font with latin fonts, use different alternatives for single chinese font) as the zhfonts.tex is presented, the running time here is usually intolerable (more than 3 minutes' loading time with very simple text for three fonts). A simple XeLaTeX file with three font embeding using XeCJK or zhspacing for interchartoken font switching usually runs less than 8 seconds. In this case, all the family (rm, ss, tt) parts (like bf, it, etc) in NFSS are defined. So, I think there should be a way to improve the font loading procedure. Maybe we had made deep copy to every fontsynonym defined here which made the loading time terribly long? Are there some way to mirror all the unchanged part of a new loading font B from a loaded time A and only leave the modified table when A and B points to same physical font? Thanks. Yue Wang
Am 12.09.2008 um 16:30 schrieb Yue Wang:
Hi:
The following cases all use second run of context. So there is no font cache time. I use a new installed Windows XP SP3 on a Core 2 Duo 1.68GHz with 1GB mem machine. Only TeXWorks and ConTeXt are installed and running on the machine. I close all pre-installed software (like MSN, IE, etc.) and leave all extra memory for LuaTeX.
Only 3 big chinese font, AdobeSongStd-Light, AdobeHeitiStd-Regular, AdobeKaitiStd-Regular are loaded. However, different approaches have different runtime, here, two cases are tested (zhfonts is attached in the mail):
All times for a normal ConTeXt run (2 runs) without intermediate from a former run on a MacBook (2GB mem, 2.4 GHz).
\usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext 你好 {\it 你好} {\bf 你好} \stoptext MtxRun | total runtime: 194.156
MtxRun | total runtime: 17.996
\definefontfeature[zh][mode=node, script=hang, lang=zhs] \definefontsynonym[song][AdobeSongStd-Light][features=zh] \definefont[song][song at 12pt] \definefontsynonym[hei][AdobeHeitiStd-Regular][features=zh] \definefont[hei][hei at 12pt] \definefontsynonym[kai][AdobeKaitiStd-Regular][features=zh] \definefont[kai][kai at 12pt] \starttext {\song 你好}{\hei 你好}{\kai 你好} \stoptext MtxRun | total runtime: 13.797
MtxRun | total runtime: 8.930
This also applies to file which only one big font, AdobeSongStd-Light, is used, two cases are tested:
\definefontfeature[zh][mode=node, script=hang, lang=zhs] \definefontsynonym[song][AdobeSongStd-Light][features=zh] \definefont[song][song at 12pt] \definefontsynonym[hei][AdobeHeitiStd-Regular][features=zh] \definefont[hei][hei at 12pt] \definefontsynonym[kai][AdobeKaitiStd-Regular][features=zh] \definefont[kai][kai at 12pt] \starttext {\song 你好} \stoptext MtxRun | total runtime: 5.609
MtxRun | total runtime: 4.820
\usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext 你好 \stoptext MtxRun | total runtime: 135.125
MtxRun | total runtime: 16.326
If no Chinese fonts are used, three cases are tested: \usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext Hello \stoptext MtxRun | total runtime: 89.938
MtxRun | total runtime: 14.940
\starttext Hello \stoptext MtxRun | total runtime: 2.594
MtxRun | total runtime: 2.902
\definefontfeature[zh][mode=node, script=hang, lang=zhs] \definefontsynonym[song][AdobeSongStd-Light][features=zh] \definefont[song][song at 12pt] \definefontsynonym[hei][AdobeHeitiStd-Regular][features=zh] \definefont[hei][hei at 12pt] \definefontsynonym[kai][AdobeKaitiStd-Regular][features=zh] \definefont[kai][kai at 12pt] \starttext Hello \stoptext MtxRun | total runtime: 2.734
MtxRun | total runtime: 2.892
In a typical chinese science book, usually 4 to 10 chinese truetype/opentype font are used. Magazines might use twenty or more. So if we want to use the standard way (use chinese font as body font, use virtual fonts mechanism to mix the font with latin fonts, use different alternatives for single chinese font) as the zhfonts.tex is presented, the running time here is usually intolerable (more than 3 minutes' loading time with very simple text for three fonts).
Enable \preloadtypescript in cont-sys.tex.
A simple XeLaTeX file with three font embeding using XeCJK or zhspacing for interchartoken font switching usually runs less than 8 seconds. In this case, all the family (rm, ss, tt) parts (like bf, it, etc) in NFSS are defined.
So, I think there should be a way to improve the font loading procedure. Maybe we had made deep copy to every fontsynonym defined here which made the loading time terribly long? Are there some way to mirror all the unchanged part of a new loading font B from a loaded time A and only leave the modified table when A and B points to same physical font? Thanks.
Yue Wang
Wolfgang
Yue Wang wrote:
Hi:
The following cases all use second run of context. So there is no font cache time. I use a new installed Windows XP SP3 on a Core 2 Duo 1.68GHz with 1GB mem machine. Only TeXWorks and ConTeXt are installed and running on the machine. I close all pre-installed software (like MSN, IE, etc.) and leave all extra memory for LuaTeX.
Only 3 big chinese font, AdobeSongStd-Light, AdobeHeitiStd-Regular, AdobeKaitiStd-Regular are loaded. However, different approaches have different runtime, here, two cases are tested (zhfonts is attached in the mail):
\usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext \dorecurse{100}{ 你好 {\it 你好} {\bf 你好} \par} \stoptext MtxRun | total runtime: 194.156
19.625 seconds, 100 processed pages, 100 shipped pages, 5.096 pages/second btw, the AdobeKai is not on my system (so i took AdobeMyungjoStd-Medium) ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Oh, My God. What's wrong on my system? Umm.......
Can you give me a hint to find what's wrong here?
I use the minimals GUI installer, and after installation,
I copy the fonts to texmf-fonts/fonts/opentype.
Is there a profile program in ConTeXt available? (I know there is a
Memory Performance chart module available).
On Sat, Sep 13, 2008 at 12:02 AM, Hans Hagen
Yue Wang wrote:
Hi:
The following cases all use second run of context. So there is no font cache time. I use a new installed Windows XP SP3 on a Core 2 Duo 1.68GHz with 1GB mem machine. Only TeXWorks and ConTeXt are installed and running on the machine. I close all pre-installed software (like MSN, IE, etc.) and leave all extra memory for LuaTeX.
Only 3 big chinese font, AdobeSongStd-Light, AdobeHeitiStd-Regular, AdobeKaitiStd-Regular are loaded. However, different approaches have different runtime, here, two cases are tested (zhfonts is attached in the mail):
\usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext \dorecurse{100}{ 你好 {\it 你好} {\bf 你好} \par} \stoptext MtxRun | total runtime: 194.156
19.625 seconds, 100 processed pages, 100 shipped pages, 5.096 pages/second
btw, the AdobeKai is not on my system (so i took AdobeMyungjoStd-Medium)
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________
Yue Wang wrote:
Oh, My God. What's wrong on my system? Umm....... Can you give me a hint to find what's wrong here? I use the minimals GUI installer, and after installation, I copy the fonts to texmf-fonts/fonts/opentype.
Is there a profile program in ConTeXt available? (I know there is a Memory Performance chart module available).
lua garbage collection on xp is somehow quite slow, when i switched to vista, the luatex test files (mk) that took forever on xp, ran smoothly on vista Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hi:
lua garbage collection on xp is somehow quite slow, when i switched to vista, the luatex test files (mk) that took forever on xp, ran smoothly on vista OK. Today I got a copy of windows vista, so I installed it on the same machine, and test the same tex file.
\usetypescript[myfont] \setupbodyfont[myfont,rm,12pt] \starttext 汉语 {\bf 汉语} {\tt 汉语} \stoptext mkiv lua stats : used config path - C:/context/tex/texmf/web2c/texmf.cn f mkiv lua stats : used cache path - C:/context/tex/texmf-cache mkiv lua stats : modules/dumps/instances - 90/24/1 mkiv lua stats : input load time - 0.062 seconds mkiv lua stats : fonts load time - 5.632 seconds mkiv lua stats : xml load time - 0.000 seconds, lpath calls: 0, cach ed calls: 0 mkiv lua stats : lxml load time - 0.000 seconds preparation, backrefe rences: 0 mkiv lua stats : mps conversion time - 0.000 seconds mkiv lua stats : node processing time - 0.000 seconds including kernel mkiv lua stats : kernel processing time - 0.000 seconds mkiv lua stats : attribute processing time - 0.000 seconds mkiv lua stats : language load time - 0.031 seconds, n=2 mkiv lua stats : graphics processing time - 0.000 seconds including tex, n=0 mkiv lua stats : metapost processing time - 0.000 seconds, loading: 0.000 secon ds, execution: 0.000 seconds, n: 0 mkiv lua stats : current memory usage - 499448604 bytes mkiv lua stats : cleaned up reserved nodes - 19 nodes, 8 lists of 100 mkiv lua stats : node memory usage - 20 glue_spec, 2 dir mkiv lua stats : loaded patterns - us:us:pat:exc:2 mkiv lua stats : loaded fonts - AdobeHeitiStd-Regular*features=zh,f allbacks=serifbolditalicwhatever:otf AdobeHeitiStd-Regular*features=zh,fallbacks =serifboldwhatever:otf AdobeKaitiStd-Regular*features=zh,fallbacks=monowhatever: otf AdobeKaitiStd-Regular*features=zh,fallbacks=serifitalicwhatever:otf AdobeSon gStd-Light*features=zh,fallbacks=serifwhatever:otf file:lmroman10-bolditalic*fea tures=default,:otf file:lmroman10-italic*features=default,:otf file:lmroman12-bo ld*features=default,:otf file:lmroman12-italic*features=default,:otf file:lmroma n12-regular*features=default,:otf file:lmroman7-bold*features=default,:otf file: lmroman7-regular*features=default,:otf file:lmroman9-bold*features=default,:otf file:lmroman9-italic*features=default,:otf file:lmroman9-regular*features=defaul t,:otf file:lmromancaps10-regular*features=default,:otf file:lmromanslant10-bold *features=default,:otf file:lmromanslant10-regular*features=default,:otf file:lm romanslant12-regular*features=default,:otf file:lmromanslant9-regular*features=d efault,:otf lmex10:tfm lmmi12:tfm lmmi7:tfm lmmi9:tfm lmmono10-regular:otf lmrom an10-bold:otf lmroman10-bolditalic:otf lmroman10-italic:otf lmroman10-regular:ot f lmsy10:tfm lmsy7:tfm lmsy9:tfm msam10:tfm msam7:tfm msbm10:tfm msbm7:tfm rm-lm r12:tfm rm-lmr7:tfm rm-lmr9:tfm mkiv lua stats : runtime - 59.904 seconds, 1 processed pages, 1 shipped pages, 0.017 pages/second MtxRun | total runtime: 67.548 well, a bit faster than XP..... Yue Wang
Yue Wang wrote:
MtxRun | total runtime: 67.548
first run? or when the windows cache is nicely filled? how much memory? (vista needs some 2 gig at least) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Yue Wang wrote:
Oh, My God. What's wrong on my system? Umm....... Can you give me a hint to find what's wrong here? I use the minimals GUI installer, and after installation, I copy the fonts to texmf-fonts/fonts/opentype.
when you set fonts.trace=true you will see that many fonts are defined at some point we might make this an option: no \tfa..d etc at all since in styles one might prefer to use hard coded fonts for titles or just switch the bodyfont (is relatively fast with typefaces); the reason for these extra sizes are convenience and speed (at least that was the case 10 years ago) for the moment, a way out of your problem is (please wikify this) \synchronizemathfontsfalse or less intrusive \let\synchronizetext\relax you should keep in mind that the context font mechanisms evolved over time and that they support rather complex mixted usage of fonts (i.e. mixing complete sets of different typefaces and such); this comes at a price (normally goes unnoticed) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hi:
for the moment, a way out of your problem is (please wikify this)
\synchronizemathfontsfalse
Woo... It is much faster! Thanks for your magic. 你好 {\it 你好} {\bf 你好} use 24.750s, 你好 use 11.766s, Hello World use 6.438s. I will wikify this together with a short introduction for chinese typesetting in MKIV.
lua garbage collection on xp is somehow quite slow, when i switched to vista, the luatex test files (mk) that took forever on xp, ran smoothly on vista
@@ buying a copy of vista is not on my budget.... too expansive. I will test this on GNU/Linux later. Yue Wang
Hi:
@@ buying a copy of vista is not on my budget.... too expansive. I will test this on GNU/Linux later.
The result on the same machine running 32bit GNU/Linux. \usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext 你好 {\it 你好} {\bf 你好} \stoptext MtxRun | total runtime: 15.854 \synchronizemathfontsfalse \usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext 你好 {\it 你好} {\bf 你好} \stoptext MtxRun | total runtime: 7.066 You are right, Hans. This is a XP specific problem. (Of course, 15.854s runtime is still a bit too long, but that's tolerable) Yue Wang
Yue Wang wrote:
(Of course, 15.854s runtime is still a bit too long, but that's tolerable)
you should not take a one page document as benchmark, since then there's always the dominant overhead of loading fonts and writing resources to the pdf file ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
I usually compose short documents (paper, homework, report etc.) which
usually runs 8-10 pages.
There will be considerable difference between traditional TeX (CJK for
LaTeX, or pdftex MKII using subfonts) and MKIV.
That means, every MKIV Chinese document I compile will spend 10
seconds loading the fonts and extra 5 seconds for compiling the
actual document.
For longer documents (more than 100 pages), I seldom compile them all.
ConTeXt user usually split the book into components. I compiled each
component (usually 10 pages) until I am satisfy in all components,
then I will compile the whole book. So, efficient on small document is
very important.
On Sat, Sep 13, 2008 at 6:14 PM, Hans Hagen
Yue Wang wrote:
(Of course, 15.854s runtime is still a bit too long, but that's tolerable)
you should not take a one page document as benchmark, since then there's always the dominant overhead of loading fonts and writing resources to the pdf file
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________
Yue Wang wrote:
\usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext 你好 {\it 你好} {\bf 你好} \stoptext MtxRun | total runtime: 194.156
If no Chinese fonts are used, three cases are tested: \usetypescriptfile[zhfonts] \usetypescript[myfont] \setupbodyfont[myfont,rm,11pt] \starttext Hello \stoptext MtxRun | total runtime: 89.938
the reason for this is that including the files in the pdf files takes long; we suspect that there might be a memory issue there (not freed) so that on your system you end up with swapping (it is on the agenda to look into this)
In a typical chinese science book, usually 4 to 10 chinese truetype/opentype font are used. Magazines might use twenty or more. So if we want to use the standard way (use chinese font as body font, use virtual fonts mechanism to mix the font with latin fonts, use different alternatives for single chinese font)
it's not virtual fonts at all, these fonts are used in parallel and fallbacks are applied by node list parsing; this also means that the fallback fonts have feature support (independent of the main font)
as the zhfonts.tex is presented, the running time here is usually intolerable (more than 3 minutes' loading time with very simple text for three fonts).
A simple XeLaTeX file with three font embeding using XeCJK or zhspacing for interchartoken font switching usually runs less than 8 seconds. In this case, all the family (rm, ss, tt) parts (like bf, it, etc) in NFSS are defined.
So, I think there should be a way to improve the font loading procedure. Maybe we had made deep copy to every fontsynonym defined here which made the loading time terribly long? Are there some way to mirror all the unchanged part of a new loading font B from a loaded time A and only leave the modified table when A and B points to same physical font? Thanks.
at the lua end there is already a lot of sharing going on but eventually the font has to be fed into tex itself where there is no such sharing anyhow, performance might improve over time Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
I've also been wondering why LuaTeX is considerably slower than XeTeX. Has anyone been doing any research into LuaJIT?
Mohamed Bana wrote:
I've also been wondering why LuaTeX is considerably slower than XeTeX. Has anyone been doing any research into LuaJIT?
(1) luajit is non portable (not on all architectures) so for the moment it's no option (2) luatex will always be slower, because we do many things in lua ... it's the price of flexibility and opening up (3) although we do our best to speed up luatex and mkiv as much as possible it's not our intention to compete in speed with pdftex (only type 1, frozen funtionality, etc) and xetex (depencency of libraries, closed engine, etc) (4) at some point (and there are already a few areas where this si true) luatex/mkiv will be faster than a traditional tex engine where all happens in macros (or is hadcodes) (5) machines become faster and mem cheaper (6) you're free to use the engine you like most [our benchmark docs are not that much slower than for instance pdftex esp when one consideres the extra functionality available] Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On Sep 12, 2008, at 10:15 PM, Hans Hagen wrote:
(4) at some point (and there are already a few areas where this si true) luatex/mkiv will be faster than a traditional tex engine where all happens in macros (or is hadcodes)
Just a very short remark: I'm testing my presentation stuff extensively these days, and there's a lot of metapost graphics involved. mkiv is amazingly fast, while both mkii and xetex take very long to compile my test files (and clutter the directory with all the intermediate files). Font loading tends to be long at first run and quite efficient thereafter. Thomas
participants (5)
-
Hans Hagen -
Mohamed Bana -
Thomas A. Schmitz -
Wolfgang Schuster -
Yue Wang