here is the xelatex result given by a chinese user: \documentclass{minimal} \usepackage{xeCJK} \setCJKfamilyfont{song}{SimSun} \begin{document} {hello {\CJKfamily{song}你好}}\par % 10000 lines \end{document} got 209 pages, three seconds to run. note he is switching families 20000 times. (\family{foo} in latex == \switchbodyfont[foo] in context) On Tue, May 12, 2009 at 5:30 PM, Hans Hagen wrote:

Yue Wang wrote:

...

Hi,

Sum up: define a font usng typescripts takes 64.797 seconds. define a font like \definefont[a][file:SimSun.ttf] takes 55.548 seconds define a font like \definefont[a][file:SimSun] takes 4.329 seconds define a font like \font\a="[simsun]" takes 1.812 seconds

Jonathan and Jin-Hwan, sorry for the noise. I am now sure that it is a context bug, not xetex's.

and

...

update: change the second example, remove the ".otf" suffix , and texexec --xtx gives: TeXExec | runtime: 5.766 Obviously, this is a parsing bug in ConTeXt.

context is not parsing for .otf or .ttf and has no reason for it either

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Yue Wang wrote:

here is the xelatex result given by a chinese user:

\documentclass{minimal} \usepackage{xeCJK} \setCJKfamilyfont{song}{SimSun} \begin{document} {hello {\CJKfamily{song}你好}}\par % 10000 lines \end{document}

got 209 pages, three seconds to run. note he is switching families 20000 times. (\family{foo} in latex == \switchbodyfont[foo] in context)

hard to say ... it all depends on how a bodyfont (or typeface) is defined, how many real fontsa re involved etc; normally, when using a typeface definition a bodyfont switch is rather efficient as it only involves one namespace switch + one font switch but when extensive math definitions are also involved it might take a few more cycles anyway .. i cannot comment on runtimes as xetex on my windows box runs too slow (caching issue) and all examples are slow, no matter how i specify fonts so i will not look into the code till that gets fixed first Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On Tue, May 12, 2009 at 7:26 PM, Taco Hoekwater wrote:

Hans Hagen wrote:

...

...

Taco Hoekwater wrote:

Hans Hagen wrote:

...

anyway .. i cannot comment on runtimes as xetex on my windows box runs too slow (caching issue) and all examples are slow, no matter how i specify fonts so i will not look into the code till that gets fixed first

The example with the typescript seems logical to me: a typescript does much work beforehand, so I assume SimSum is loaded multiple times.

I am trying the AdobeSong example as we speak, there surely seems something wrong there.

let's start with saying that I have no intention to optimize mkii for xetex as we decided some time ago (at the context meeting) that we would

For educational purposes: whether you have

\definefont[a][file:AdobeSongLight-Std] or \definefont[a][file:AdobeSongLight-Std.otf]

makes no difference to context at all.

The first makes \a expand into (simplified)

\font\internala="[AdobeSongLight-Std]" at 12pt\internala

the second into

\font\internala="[AdobeSongLight-Std.otf]" at 12pt\internala

but there are no other macro expansion changes at all. You can test this yourself by making two alternative files an adding \loggingall to them, then compare the two logs using diff.

As to the reason why one of the two is much slower than the other: I do not know for sure, but I suspect that XeTeX does not cache the (some of) the internals results of font file assignments with explicit extensions. The speed difference is most certainly *not* caused by anything the context macros do.

disagreed: you can test by yourself that \font\a="[simsun]" \starttext \dorecurse{10000}{ hello {\a 你好}\par} \stoptext runtime: TeXExec | runtime: 2.922 \font\a="[simsun.ttf]" \starttext \dorecurse{10000}{ hello {\a 你好}\par} \stoptext runtime: TeXExec | runtime: 2.938

Best wishes, Taco

PS As to why \definefont is itself slower than \font: it does not actually define a font \a at a specific size. Instead it defines a macro \a that defines and then selects a font with an internal, temporary name, using the local font size etc.

___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Taco Hoekwater wrote:

Yue Wang wrote:

...

\font\a="[simsun]" \starttext \dorecurse{10000}{ hello {\a 你好}\par} \stoptext

runtime: TeXExec | runtime: 2.922

\font\a="[simsun.ttf]" \starttext \dorecurse{10000}{ hello {\a 你好}\par} \stoptext

runtime: TeXExec | runtime: 2.938

Please try an otf font and/or actually compare the log files, as I said.

also, make sure that the *same* font is used as on my machine there is no ttf but a simsun.ttc file so it can well be that th efirst call loads a different font than the suffix call Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Yue Wang wrote:

...

...

\font\a="[simsun]" \starttext \dorecurse{10000}{ hello {\a 你好}\par} \stoptext

runtime: TeXExec | runtime: 2.922

\font\a="[simsun.ttf]" \starttext \dorecurse{10000}{ hello {\a 你好}\par} \stoptext

as taco already mentioned there's a problem with xetex caching fonts in a traditional tex engine (pdftex, xetex) one can say \font\xxx=blabla many times but blabla is loaded only once; however, it looks like in the following case \font\xxx=blabla.suffix no caching takes place and as a result you load the font 10000 times (can be seen when you enable a file tracer) however, memory consumption does grow only in small steps so some caching takes place (maybe only new id's) if you define a font grouped {\a ...} the font gets defined each time (as context has delayed font loading and supports local redefinitions which is quite handy when using SymbolicNames for fonts) and normally xetex's internal caching should make that efficient i would not be surprised if the (forced) name-lookup problems are kind of related to this so ... this suffix problem is not related to context at all Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Hi, On Tue, May 12, 2009 at 8:26 PM, Taco Hoekwater wrote:

Yue Wang wrote:

...

\font\a="[AdobeSongStd-Light]"

Why? I asked to use \definefont.

After reading Hans' explanation, I understand what you said now. \dorecurse{10000}{ hello {\font\a="[AdobeSongStd-Light]"\a 你好}\par} is fast while \dorecurse{10000}{ hello {\font\a="[AdobeSongStd-Light.otf]"\a 你好}\par} is slow Thank you. Will report that to Jonathan Kew. Yue Wang

___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Yue Wang wrote:

Thank you, Taco.

currently I found one way to speed up font loading:

\synchronizemathfontsfalse \let\synchronizetext\relax \starttypescript[serif] [myzhfont] \definefontsynonym [Serif] [file:SimSun] \stoptypescript \definetypeface [myzhfont] [rm] [serif][myzhfont] [default] \def\a{\myzhfont} \dorecurse{10000}{{hello {\a 你好}}\par} \bye

as wolfgang already pointed out, if you want fast fonts you should define them directly using \definefont if you disable bits and pieces of the setupbodyfont mechanism you will eventually run into problems, for instance because \tfd etc are not defined and your chapter titles etc will behave unexpected (sizes as well as when mixed with math) in a regular document you only set up the bodyfont once (not 10000 times) and then use \bf, \sl, \itc etc which are pretty fast

TeXExec | runtime: 10.172 much faster, but still far slower than \definefont.

this reminds me another possible bug in ConTeXt:

as mentioned ... we're no talking bugs here but 'features' and side effects of them

Test1:

\definetypeface[iwona][ss][sans] [iwona] [default] \definetypeface[iwona][mm][math] [iwona] [default][text=ss] \setupbodyfont[iwona] \starttext foobar \stoptext

runtime= 6.625. There is a 5 seconds pause after loading (c:/context/tex/texmf-context/tex/context/base/spec-xet.tex)

But I don't know what xetex is loading.

fonts (like the sans and more noticeably math (6 families * 3 sizes = 18 fonts at least)

Test 2:

\definetypeface[iwona][ss][sans] [iwona] [default] %\definetypeface[iwona][mm][math] [iwona] [default][text=ss] \setupbodyfont[iwona] \starttext foobar \stoptext

I only comment the math defination line. now runetime = 0.969

right .. and as context is supposed to do math (keep in mind that some symbols come from math anyway) this is expected

Test 3:

move the three lines to context.rme, \definetypeface[iwona][ss][sans] [iwona] [default] \definetypeface[iwona][mm][math] [iwona] [default][text=ss] \setupbodyfont[iwona] then rebuild the format. note we also load the math font. but one thing is clear: Hans said unlike plain TeX and LaTeX, font tfm is not dumped into ConTeXt format.

well, in this case you probably *did* force them in format; the normal default font (lm) is not in the format and loading is delayed

However, at this time we compile \starttext foobar \stoptext

runtime =0.875

I don't know the reason for this problem too.... I think XeTeX load math tfms quite fast. (it should be even faster than otf loading, but in xecontext, this is not true).

more needs to be set up ... not only fonts, + some testing + ... just do a \tracingall before setupbodyfont and you will see Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Yue Wang wrote:

moreover, can you tell me why pdftex load these fonts so fast? (also 6 families * 3 sizes)

i don't know; as the same code is used so it must be xetex itself then ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Hi, Hans: here is the solution to your great feature: change \def\defaultfontfile{lmmono10-regular} in font-ini.mkii to \def\defaultfontfile{"[lmmono10-regular]"} to disable that feature. Yue Wang On Tue, May 12, 2009 at 9:59 PM, Yue Wang wrote:

On Tue, May 12, 2009 at 8:54 PM, Hans Hagen wrote:

...

Yue Wang wrote:

...

moreover, can you tell me why pdftex load these fonts so fast? (also 6 families * 3 sizes)

i don't know; as the same code is used so it must be xetex itself then

Then let me tell you why: ConTeXt tries to load lmmono10-regular at least 6 times, failed after two testing, then succeed in the end. However, try to testing whether a loaded font is \nullfont or not is quite slow in XeTeX (Jonathan already mentioned that in \testFontIsAvailable). we should definately avoid that. But why pdftex is so fast? because it does not involve font testing.

So why XeTeX is spending 6 seconds aimlessly? since ConTeXt asked it to search for a non-existed font.

Yue Wang

...

-----------------------------------------------------------------                                          Hans Hagen | PRAGMA ADE              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com                                             | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net archive  : https://foundry.supelec.fr/projects/contextrev/ wiki     : http://contextgarden.net ___________________________________________________________________________________

On Tue, May 12, 2009 at 11:02 PM, Hans Hagen wrote:

Yue Wang wrote:

...

Hi, Hans:

here is the solution to your great feature:

huh? what great features? what's wrong with the features anyway? you want a special version that instead prints hundreds of error messages each and every run because some font is missing which in practice is harmless?

...

change  \def\defaultfontfile{lmmono10-regular} in font-ini.mkii to \def\defaultfontfile{"[lmmono10-regular]"} to disable that feature.

it should be \defaultfontfile{file:lmmono10-regular} (when i  played with that this morning it made not much difference on the test runs here but the file: prefix does not hurt)

There is great difference here (8 sec compare to 1.2 seconds per run)

don't use the "[ ]" syntax which is rather engine specific

But that line is enclosed in \ifXeTeX and \endif... So it will be fine.

i moved all xetex code to a separate file font-xtx now (not sure if that  has side effects due to definition order but don't want to mess up the mkii code)

I see.

but it cannot solve the suffix problem nor the name lookup problems due to caching

I already reported that feature/bug to Jonathan Kew

Hans

-----------------------------------------------------------------                                          Hans Hagen | PRAGMA ADE              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com                                             | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net archive  : https://foundry.supelec.fr/projects/contextrev/ wiki     : http://contextgarden.net ___________________________________________________________________________________

On Tue, May 12, 2009 at 11:22 PM, Hans Hagen wrote:

Yue Wang wrote:

...

On Tue, May 12, 2009 at 11:02 PM, Hans Hagen wrote:

...

Yue Wang wrote:

...

Hi, Hans:

here is the solution to your great feature:

huh? what great features? what's wrong with the features anyway? you want a special version that instead prints hundreds of error messages each and every run because some font is missing which in practice is harmless?

...

change \def\defaultfontfile{lmmono10-regular} in font-ini.mkii to \def\defaultfontfile{"[lmmono10-regular]"} to disable that feature.

it should be \defaultfontfile{file:lmmono10-regular} (when i played with that this morning it made not much difference on the test runs here but the file: prefix does not hurt)

There is great difference here (8 sec compare to 1.2 seconds per run)

makes no difference here for the big test

Well, I always split my projects into small pieces so... it really counts.

\setupbodyfont[myzhfont] \dorecurse{10000}{{hello {\switchtobodyfont[myzhfont] 你好}}\par}

so it's probably also an operating system issue (caching files in mem, disk access etc)

Well, that is possible. But in fact no new font is defined during the font switching, which is paradoxical. I will look into that tomorrow. Please fix the - lmmono defination - unicode-letters charclass definations - mpgraphics displacement bug (or feature...) - dvipdfmx focus=standard feature that I reported today. Thank you. Good night and good luck! Yue Wang

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

I can confirm that I am wrong. 2009/5/13 Yue Wang :

Hi, Hans:

On Tue, May 12, 2009 at 11:22 PM, Hans Hagen wrote:

...

\setupbodyfont[myzhfont] \dorecurse{10000}{{hello {\switchtobodyfont[myzhfont] 你好}}\par} so it's probably also an operating system issue (caching files in mem, disk access etc)

After debugging for half an hour in the morning, finally I know why this is so slow on both XeTeX and pdfTeX. This problem is not an operating system issue, But a ConTeXt "feature".

in font-mkii, you use the following to define a actual font (\definefontlocal and \definefontglobal): {\expandafter\xdef\csname#1\endcsname % ! ! ! ! not needed in luatex ! ! ! ! {\noexpand\csname#1:\endcsname \noexpand\reactivatefont{\somefontname}{\fontfile}}%

For example, in the previous example, it will define a lot of fonts, like \*myzhfont12ptmmexrm*: , is defined as \reactivatefont{cmex10}{lmex10}. When define that font, a macro \*myzhfont12ptmmexrm* is defined to select the \*myzhfont12ptmmexrm* : font. \*myzhfont12ptmmexrm* =\csname *myzhfont12ptmmexrm* :\endcsname \reactivefont{cmex10}{lmex10} TeX then expand your reactivatefont macros to lmex10 at 12.0pt. So the \*myzhfont12ptmmexrm*: font will assigned to lmex10 at 12.0pt. This is quite right for definition.

However, after switching the font, the problems occurs: it tries to call \*myzhfont12ptmmexrm* macro. the macro then expands to \csname *myzhfont12ptmmexrm* :\endcsname \reactivefont{cmex10}{lmex10} then to \*myzhfont12ptmmexrm* :\reactivefont{cmex10}{lmex10} and here TeX tries to do the following thing: TeX select the font \*myzhfont12ptmmexrm*:, which is lmex10 at 12pt, then *changing* into that font environment in order to *typeset* \reactivefont. it then expand the macro of reacitve font, but found nothing to typeset...

what's worse, this will occur on every real fonts in the definition. So in fact TeX will switching dozens of fonts into the *current* font for a "\switchtobodyfont" call, in order to typeset nothing. And that's why ConTeXt is so slow on typeface changing.

The solution to this bug (or feature?) is quite easy, just use \font to define the \*myzhfont12ptmmexrm*: font, but pay attention not to switch the current font to \*myzhfont12ptmmexrm*: . Since it is quite slow to switch to dozens of font for one \switchtobodyfont call even in Knuth TeX.

Yue Wang

...

so it's probably also an operating system issue (caching files in mem, disk access etc)

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Yue Wang wrote:

On Tue, May 12, 2009 at 11:02 PM, Hans Hagen wrote:

...

Yue Wang wrote:

...

Hi, Hans:

here is the solution to your great feature: huh? what great features? what's wrong with the features anyway? you want a special version that instead prints hundreds of error messages each and every run because some font is missing which in practice is harmless?

...

change \def\defaultfontfile{lmmono10-regular} in font-ini.mkii to \def\defaultfontfile{"[lmmono10-regular]"} to disable that feature. it should be \defaultfontfile{file:lmmono10-regular} (when i played with that this morning it made not much difference on the test runs here but the file: prefix does not hurt)

There is great difference here (8 sec compare to 1.2 seconds per run)

i uploaded a new zip to the website you can say: \settrue\tracexetexfonts as you will see, not that much fonts are loaded; once a font (in typefaces) is found, it is remembered; for xetex i now added an extra layer of remembering (the checks) so that misses in lookups might happen less (it might speed up the MathGamma cases a bit but i didn't notice much performance gain so it probably depends on other factors as well) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On Tue, May 12, 2009 at 10:50 PM, Hans Hagen wrote:

Yue Wang wrote:

...

On Tue, May 12, 2009 at 8:54 PM, Hans Hagen wrote:

...

Yue Wang wrote:

...

moreover, can you tell me why pdftex load these fonts so fast? (also 6 families * 3 sizes)

i don't know; as the same code is used so it must be xetex itself then

Then let me tell you why: ConTeXt tries to load lmmono10-regular at least 6 times, failed after two testing, then succeed in the end. However, try to testing whether a loaded font is \nullfont or not is quite slow in XeTeX (Jonathan already mentioned that in \testFontIsAvailable). we should definately avoid that. But why pdftex is so fast? because it does not involve font testing.

So why XeTeX is spending 6 seconds aimlessly? since ConTeXt asked it to search for a non-existed font.

well, even locating a font 6 times should be no big deal

loading 6 fonts, each fonts will be scanned for 3 times. so there are 18 searching. 12 of them are fc searches, and these are quite slow.

btw, you can try to change the following into

\def\defaultfontfile{file:lmmono10-regular}

After changing like this, XeTeX runs like a blink. (It wasted 6 seconds for each compile. now it won't)

but even then ... if that one is used then there is something else going on so best find out what happens ... in context we can have 4 extra math families and in most cases only two are used (MathAlpha and MathBeta) while (just in onder to catch errors) MathGamma etc then automatically will trigger the default font to be used (other approached would demand more definitions at the user end and/or a more low level implementation); the only optimization i can imagine is more clever sharing of the default font but as in other cases one expects the default to be properly scaled it not that simple; after all, users also expect proper error recovery (and in many cases missing some specific fonts is no real problem until it's used); so, you can hardly blame context for the fact that xetex has a certain logix on locating fonts that happens to be not that good a match for context

The fact that xetex uses this mixture of "" en [] does not help either as context uses [] itself so parsing is somewhat complicated (the file/name prefix was introduced to circumvent this problem);

Hans

-----------------------------------------------------------------                                          Hans Hagen | PRAGMA ADE              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com                                             | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net archive  : https://foundry.supelec.fr/projects/contextrev/ wiki     : http://contextgarden.net ___________________________________________________________________________________

Taco Hoekwater wrote:

Yue Wang wrote:

...

...

\def\defaultfontfile{file:lmmono10-regular} After changing like this, XeTeX runs like a blink. (It wasted 6 seconds for each compile. now it won't)

Does that mean that the near-endless font issues with xetex are now finally fixed?

no ... suffix and name lookup remain (but the file: before the fallback font saves such name lookups so we disguised the problem) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Yue Wang wrote:

loading 6 fonts, each fonts will be scanned for 3 times. so there are 18 searching. 12 of them are fc searches, and these are quite slow.

if so, then this fc mechanism is pretty messed up as it's just a hash lookup which cna be fast also, it does not explain why simple plain definitions also have a suffix and/or name lookup problem so you'd better make sure that that is fixed too (if only because we cannot be sure of users always using filenames) in the meantime i'll add some lookup caching in the xetex specific context code (costs a bit of extra hash mem but lets not bother about that) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

2009/5/12 Yue Wang:

here is the xelatex result given by a chinese user:

\documentclass{minimal} \usepackage{xeCJK} \setCJKfamilyfont{song}{SimSun} \begin{document} {hello {\CJKfamily{song}你好}}\par % 10000 lines \end{document}

got 209 pages, three seconds to run. note he is switching  families 20000 times. (\family{foo} in latex == \switchbodyfont[foo] in context)

I didn't test that, but I suspect that one thing could make an enormous time difference if you switch the font 10.000 times. I suspect that what LaTeX does is something similar to the following (in ConTeXt slang): \starttypescript[cjk] [zhfont] \definefontsynonym [CJK] [file:SimSun] \definefontsynonym [CJKBold] [file:SimSun] % you could artificially embolden the font \definefontsynonym [CJKItalic] [file:SimSun] % you could artificially slant the font \definefontsynonym [CJKBoldItalic] [file:SimSun] \stoptypescript \definetypeface [myzhfont] [rm] [serif] [latin-modern] [default] \definetypeface [myzhfont] [ss] [sans] [latin-modern] [default] \definetypeface [myzhfont] [tt] [mono] [latin-modern] [default] \definetypeface [myzhfont] [cjk] [cjk] [zhfont] [default] \dorecurse{10000}{ hello {\cjk 你好 \it 你好}} In a similar way ConTeXt supports handwriting/calligraphy families (\hw, \cg I think; well, maybe you could even pretend that you are defining the "calligraphy font" to start with, as that one is already defined where it needs to be defined). If you manage to make the proposal above to run, this will save you the need for switching the whole typeface ten thousands of times and should become really fast (hopefully). Mojca

Mojca Miklavec wrote:

\starttext % test 1: takes 12 seconds \dorecurse{10000}{ hello {\cg 你好}} % test 2: takes 9 seconds \dorecurse{10000}{ hello {\ccg 你好}} \stoptext

Switching the whole typescript (\def\ccg{\zhfont} and running the second test) took 123 seconds eating 100% of processor time.

% test 1: takes 12 seconds \dorecurse{10000}{ hello {\cg 你好}} 8 sec on my machine % test 2: takes 9 seconds % \dorecurse{10000}{ hello {\ccg 你好}} 6 sec on my machine now, say that 10K switches represent 250 pages (some 40 per page which is already quite a lot for *full switches*) then we're talking of 0.01 sec per page for this, which is kind of neglectable compared to the rest of the typesetting also, mkiv takes less than 6 sec for the test and there we can use fallbacks and as such less switches (of the runtime, 3 sec is font loading which is due to the fact that for some reason my machine/vista does 300K page faults per big font, something that will not happen on more modern machines or on linux) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Am 13.05.2009 um 10:59 schrieb Mojca Miklavec:

\definetypeface [myzhfont] [rm] [serif] [latin-modern] [default] \definetypeface [myzhfont] [ss] [sans] [latin-modern] [default] \definetypeface [myzhfont] [tt] [mono] [latin-modern] [default] \definetypeface [myzhfont] [cjk] [cjk] [zhfont] [default]

\dorecurse{10000}{ hello {\cjk 你好 \it 你好}}

How should this work when you switch from serif to sans, e.g.: \rm hello {\cjk 你好 \it 你好} \ss hello {\cjk 你好 \it 你好} Wolfgang

Am 13.05.2009 um 12:17 schrieb Mojca Miklavec:

Or do you want to suggest that one would possibly need both "serif" and "sans" variants of some Chinese font, often switching between families inside a document?

That's what I mean, also in chinese you use different fonts for serif, sans and mono. Wolfgang

Am 13.05.2009 um 13:12 schrieb Mojca Miklavec:

On Wed, May 13, 2009 at 12:41, Wolfgang Schuster wrote:

...

Am 13.05.2009 um 12:17 schrieb Mojca Miklavec:

...

Or do you want to suggest that one would possibly need both "serif" and "sans" variants of some Chinese font, often switching between families inside a document?

That's what I mean, also in chinese you use different fonts for serif, sans and mono.

Wait a minute ... why do they need mono/typewriter? Aren't all the Chinese glyphs "fixed width" anyway?

Do you want code sample in the same font as the result? With certain features in OT fonts you can enable proportional chars for punctuation, kana etc.

Also, if one has a well-designed font that includes all the latin glyphs and all the bold/italic variants then one should in principle not need an extra "latin" family, but those are probably just nice dreams ... I always had the impression that there are not many high-quality Chinese fonts and that having bold and italic alone is a problem, not to speak about frequent mixing of several different families.

The latin glyphs in many free fonts are even worse than Comic Sans :( Adobe fills the latin ranges with glyphs from other fonts with different quality but they have lots of features, e.g. ital to enable italics :)

(Just thinking alound: aren't there plenty of books around that also mix lots of greek and latin, possibly using different fonts for them? How do they deal with the problem, or is the problem just neglectable?)

Dunno, AFAIR Thomas use switch to a different font in his greek module, in mkiv this is trivia with font fallbacks. Wolfgang

On Wed, May 13, 2009 at 7:12 PM, Mojca Miklavec wrote:

On Wed, May 13, 2009 at 12:41, Wolfgang Schuster wrote:

...

Am 13.05.2009 um 12:17 schrieb Mojca Miklavec:

...

Or do you want to suggest that one would possibly need both "serif" and "sans" variants of some Chinese font, often switching between families inside a document?

That's what I mean, also in chinese you use different fonts for serif, sans and mono.

Wait a minute ... why do they need mono/typewriter? Aren't all the Chinese glyphs "fixed width" anyway?

We do not need mono. We do not even need italic, bold italic, and small caps. ConTeXt/LaTeX's naming scheme just follow traditional latin way, and that's pretty different compared to Asian scripts. (of course, some of them are similar. for example, Song is like roman regular, CuSong is like roman bold. LiShu is like calligraphy, Dengxian is like Sans. CuDengXian (Hei) is like Sans bold). But why do we define them all as latin typefaces? that's just because we should follow traditional ConTeXt or LaTeX way, since the whole ConTeXt or LaTeX system is heavily based on their own font naming mechanism. You can grep the ConTeXt source code, and find font switching like \it, \rm, \ss appears on almost every file. If we insist on our own naming mechanism (like \song, \hei, \cusong), then ConTeXt will become unusable in the end. Moreover, the only way to write consist document in both latin and cjk language is to fall cjk fonts into latin system. Suppose you want to define a document chapter title font, neither \heid nor \bfd will suffice. The only way is to map all chinese font names (hei, song, for example) to latin names(bold italic, for example) [since latin names can be treat as a super set of asian names], or design a more comprehensive naming system, and later mapping both into that. Then return to your question. what happens if mono is not defined? well. when you want to write a web link or source code that contains Chinese like \type{http://zh.wikipedia.org/wiki/字体} or \starttyping if (filename == NIL){ printf("文件%s未找到\n", filename); return -1; } \stoptyping you will run into problems since mono for Chinese are not defined.

Also, if one has a well-designed font that includes all the latin glyphs and all the bold/italic variants then one should in principle not need an extra "latin" family, but those are probably just nice dreams ... I always had the impression that there are not many high-quality Chinese fonts and that having bold and italic alone is a problem, not to speak about frequent mixing of several different families.

Chinese fonts released in recent years have better latin glyphs design. But as I said before, the Chinese script system is quite different from the Latin one. Chinese companies are not obliged to include "italic" or "small caps" glyphs in their Chinese fonts, just like we cannot ask Adobe to include "kai" and "song" styles in their Minion Pro fonts.

(Just thinking alound: aren't there plenty of books around that also mix lots of greek and latin, possibly using different fonts for them? How do they deal with the problem, or is the problem just neglectable?)

Compound typeface is a very basic feature required for all typesetting software. Quark and InDesign have options to enable this feature. This is even possible in no-production-purpose software like Microsoft Word. But as far as I know, most typographic software programs are WYSIWYG systems, and rendering a page in computer is much faster than typing a word by human beings. So we will not notice the performance problem in these systems. But for TeX this is different, in order to see the result after positioning something to the right place, or changing a few words, several compile->view->adjust->compile->view->adjust procedures will be involved. So the long compilation time will make the software unusable (Imagine that you want to correct a minor error in a document and see the result, you press the typeset button in TeXWorks, leave the lualatex program running, and during the 5 minutes compilation time, you can have a cup of tea and enjoy the sunshine... Anyway, t-zhspacing is much faster than lualatex. but 1 minutes to compile a 100 pages document is still too long) But one thing is always puzzling me: When open MS Word, it consumes 16MB memory, then I typeset Chinese text in 10 different Chinese fonts (all are 10M+ each), word only consumes 30MB memory. If I do that in LuaTeX it will consume almost 1G memory... why MS Word parses fonts and renders text so fast while consumes such less memory? Can TeX do that too?

Mojca ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Am 13.05.2009 um 14:59 schrieb Hans Hagen:

Yue Wang wrote:

...

compilation time, you can have a cup of tea and enjoy the sunshine... Anyway, t-zhspacing is much faster than lualatex. but 1 minutes to compile a 100 pages document is still too long)

it all depends on what you use and want to achieve; also lualatex is not relevant when we discuss for instance context mkiv which follows different routes

(btw, mk.pdf takes some 55 seconds for a 300+ page document and that's 10 seconds for page faulting so on another machine it runs faster; and mk uses a lot of large fonts + some demanding typesetting like arabic)

1 min for 100 pages is very slow, I have a japanese/german book which takes about 1 min for 600 pages, the slowest part is a very huge natural table (over 700 cells) where ConTeXt needs about 1 second. Wolfgang