Re: [NTG-pdftex] About the font_max limitation for vf_nf
Hi, I took a closer look at the problem. vf_nf grows very fast because vf_nf = <number of vf fonts> * <number of tfm in each vf> We cannot reduce this, because the number of tfm inside a vf (let's call it the local tfm) is used to render the characters inside a vf. Hence pdftex must remember all local for all vf, which is the above number. However it seems reasonable to allow the limit to be much higher than font_max. I will submit a patch soon. Thanh On Wed, Nov 22, 2006 at 09:44:10PM +0800, Jie Luo wrote:
Hi,
There is a GB.tex in the test.tar.bz2 which is a modification of GB.tex in CJK package. This file can be compiled with TeXLive. When processing this file with pdflatex, the vf_nf value reaches 1978 which is close to the present font_max ( = 2000 ). I also attach the log files of another two tex documents, hope to be helpful.
Thanks,
Jie
Thanh Han The wrote:
Hi,
can you please send me a minimal test file so I can reproduce the problem? It is best to use fonts that are available in TeXLive so I can compile it.
Thanks, Thanh
On Wed, Nov 22, 2006 at 05:38:35PM +0800, Jie Luo wrote:
Hi Thanh,
Thanh Han The wrote:
Hi,
I don't know whether this is a bug or not, unless I can reproduce the behaviour. You can verify by checking how many tfm fonts are opened by pdftex. If the total number of tfm fonts is much less than vf_nf, then it's likely a bug. However there are a few things to consider:
- tex (and hence pdftex) loads the same tfm for different sizes, ef \font\f=cmr10 and \font\f=cmr10 at 12pt will load the tfm twice.
- the same applies for vf. In the above example, if cmr10 is a virtual font containing 2 tfm, the number of loaded fonts will be 4. So you get the idea how quick it grows.
You can use the script I attached to see which fonts have been opened during a tex run. Usage:
,-------- | list-tex-files pdftex foo.tex `--------
It will run pdftex and saves the list of opened files into a temporary file.
HTH, Thanh
I did some tests using this script.
GB.tex vf_nf = 3114 opened tfm files with pdflatex = 408 opened tfm files with latex = 59
book.tex vf_nf = 11834 opened tfm files with pdflatex = 1075 opened tfm files with latex = 364
So it seems that the vf_nf is about 10 times of the opened tfm files, this maybe a bug. By the way, I do not have the list-tex-files.vim file, is this a problem?
Thanks,
Jie
On Wed, Nov 22, 2006 at 01:12:56PM +0800, Jie Luo wrote:
Hi Thanh,
When using pdflatex to processing tex documents contain more than two Chinese GBK virtual fonts, pdflatex always failed with a divide by zero error. I read the mail archive of pdftex list and find it said to be related to the font_max limitation. I add some statement to debug this, and find that the vf_nf value is about 2300 when using two Chinese GBK virtual fonts. But when I use pdflatex to compile a book with about 250 pages, the value of vf_nf is about 12000 with less than 100 type1 fonts embedded in the pdf file. Is there any problem with the vf_nf? Or it is the expected value.
Jie
------------------------------------------------------------------------
#!/bin/sh
if test "x$1" = x; then echo Usage: "$0 <tex command>" echo Example: "$0 latex myfile" exit 1 fi export KPATHSEA_DEBUG=4 f=/tmp/kpof.$$.1
$* 2>$f vim -S ~/vim/list-tex-files.vim $f echo "List of opened files saved to $f"
Hi, I modified you patch to get it apply to tetex-3.0_p1, and change font_max back to 2000 in texmf.cnf. Everything is fine now. Thank you very much for patch. Jie Thanh Han The wrote:
Hi,
I took a closer look at the problem. vf_nf grows very fast because vf_nf = <number of vf fonts> * <number of tfm in each vf> We cannot reduce this, because the number of tfm inside a vf (let's call it the local tfm) is used to render the characters inside a vf. Hence pdftex must remember all local for all vf, which is the above number.
However it seems reasonable to allow the limit to be much higher than font_max. I will submit a patch soon.
Thanh
On Wed, Nov 22, 2006 at 09:44:10PM +0800, Jie Luo wrote:
Hi,
There is a GB.tex in the test.tar.bz2 which is a modification of GB.tex in CJK package. This file can be compiled with TeXLive. When processing this file with pdflatex, the vf_nf value reaches 1978 which is close to the present font_max ( = 2000 ). I also attach the log files of another two tex documents, hope to be helpful.
Thanks,
Jie
Hi,
can you please send me a minimal test file so I can reproduce the problem? It is best to use fonts that are available in TeXLive so I can compile it.
Thanks, Thanh
On Wed, Nov 22, 2006 at 05:38:35PM +0800, Jie Luo wrote:
Hi Thanh,
Thanh Han The wrote:
Hi,
I don't know whether this is a bug or not, unless I can reproduce the behaviour. You can verify by checking how many tfm fonts are opened by pdftex. If the total number of tfm fonts is much less than vf_nf, then it's likely a bug. However there are a few things to consider:
- tex (and hence pdftex) loads the same tfm for different sizes, ef \font\f=cmr10 and \font\f=cmr10 at 12pt will load the tfm twice.
- the same applies for vf. In the above example, if cmr10 is a virtual font containing 2 tfm, the number of loaded fonts will be 4. So you get the idea how quick it grows.
You can use the script I attached to see which fonts have been opened during a tex run. Usage:
,-------- | list-tex-files pdftex foo.tex `--------
It will run pdftex and saves the list of opened files into a temporary file.
HTH, Thanh
I did some tests using this script.
GB.tex vf_nf = 3114 opened tfm files with pdflatex = 408 opened tfm files with latex = 59
book.tex vf_nf = 11834 opened tfm files with pdflatex = 1075 opened tfm files with latex = 364
So it seems that the vf_nf is about 10 times of the opened tfm files, this maybe a bug. By the way, I do not have the list-tex-files.vim file, is this a problem?
Thanks,
Jie
On Wed, Nov 22, 2006 at 01:12:56PM +0800, Jie Luo wrote:
Hi Thanh,
When using pdflatex to processing tex documents contain more than two Chinese GBK virtual fonts, pdflatex always failed with a divide by
zero
error. I read the mail archive of pdftex list and find it said to be related to the font_max limitation. I add some statement to debug
Thanh Han The wrote: this,
and find that the vf_nf value is about 2300 when using two Chinese GBK virtual fonts. But when I use pdflatex to compile a book with about 250 pages, the value of vf_nf is about 12000 with less than 100 type1 fonts embedded in the pdf file. Is there any problem with the vf_nf? Or it is the expected value.
Jie
------------------------------------------------------------------------
#!/bin/sh
if test "x$1" = x; then echo Usage: "$0 <tex command>" echo Example: "$0 latex myfile" exit 1 fi export KPATHSEA_DEBUG=4 f=/tmp/kpof.$$.1
$* 2>$f vim -S ~/vim/list-tex-files.vim $f echo "List of opened files saved to $f"
Jie Luo
Hi,
I modified you patch to get it apply to tetex-3.0_p1, and change font_max back to 2000 in texmf.cnf. Everything is fine now. Thank you very much for patch.
Ah, great - can you publish this modified patch? Thanks in advance, Frank -- Dr. Frank Küster Single Molecule Spectroscopy, Protein Folding @ Inst. f. Biochemie, Univ. Zürich Debian Developer (teTeX/TeXLive)
Frank Küster wrote:
Jie Luo
wrote: Hi,
I modified you patch to get it apply to tetex-3.0_p1, and change font_max back to 2000 in texmf.cnf. Everything is fine now. Thank you very much for patch.
Ah, great - can you publish this modified patch?
Thanks in advance, Frank
Attached. Jie diff -urN tetex-src-3.0/texk/web2c/pdftexdir/hz.ch tetex-src-3.0.new/texk/web2c/pdftexdir/hz.ch --- tetex-src-3.0/texk/web2c/pdftexdir/hz.ch 2005-12-06 23:04:25.000000000 +0800 +++ tetex-src-3.0.new/texk/web2c/pdftexdir/hz.ch 2006-11-23 11:13:55.760977765 +0800 @@ -396,7 +396,7 @@ pdf_font_expand_ratio[k] := e; pdf_font_step[k] := pdf_font_step[f]; pdf_font_auto_expand[k] := pdf_font_auto_expand[f]; - pdf_font_blink[k] := f; {???} + pdf_font_blink[k] := f; pdf_font_lp_base[k] := pdf_font_lp_base[f]; pdf_font_rp_base[k] := pdf_font_rp_base[f]; pdf_font_ef_base[k] := pdf_font_ef_base[f]; @@ -467,6 +467,7 @@ e := pdf_font_expand_ratio[f]; for k := 0 to vf_local_font_num[bf] - 1 do begin lf := vf_default_font[bf] + k; + allocvffnts; vf_e_fnts[vf_nf] := vf_e_fnts[lf]; vf_i_fnts[vf_nf] := auto_expand_font(vf_i_fnts[lf], e); set_expand_param(vf_i_fnts[vf_nf], vf_i_fnts[lf], e); diff -urN tetex-src-3.0/texk/web2c/pdftexdir/pdftex.ch tetex-src-3.0.new/texk/web2c/pdftexdir/pdftex.ch --- tetex-src-3.0/texk/web2c/pdftexdir/pdftex.ch 2005-12-06 23:04:25.000000000 +0800 +++ tetex-src-3.0.new/texk/web2c/pdftexdir/pdftex.ch 2006-11-23 11:07:10.780656355 +0800 @@ -2491,6 +2491,7 @@ cmd := vf_byte; save_vf_nf := vf_nf; while (cmd >= fnt_def1) and (cmd <= fnt_def1 + 3) do begin + allocvffnts; vf_e_fnts[vf_nf] := vf_read(cmd - fnt_def1 + 1); vf_i_fnts[vf_nf] := vf_def_font(f); incr(vf_nf); diff -urN tetex-src-3.0/texk/web2c/pdftexdir/pdftex.defines tetex-src-3.0.new/texk/web2c/pdftexdir/pdftex.defines --- tetex-src-3.0/texk/web2c/pdftexdir/pdftex.defines 2005-12-06 23:04:25.000000000 +0800 +++ tetex-src-3.0.new/texk/web2c/pdftexdir/pdftex.defines 2006-11-23 11:03:46.722780867 +0800 @@ -63,7 +63,7 @@ @define procedure setjobid(); @define procedure unescapehex(); @define procedure writestreamlength(); - +@define procedure allocvffnts; { functions from vfpacket.c } @define function newvfpacket(); diff -urN tetex-src-3.0/texk/web2c/pdftexdir/utils.c tetex-src-3.0.new/texk/web2c/pdftexdir/utils.c --- tetex-src-3.0/texk/web2c/pdftexdir/utils.c 2005-12-06 23:04:26.000000000 +0800 +++ tetex-src-3.0.new/texk/web2c/pdftexdir/utils.c 2006-11-23 11:19:21.667685137 +0800 @@ -51,7 +51,15 @@ /* define char_ptr, char_array & char_limit */ typedef char char_entry; -define_array(char); +define_array(char); + +/* define vf_e_fnts_ptr, vf_e_fnts_array & vf_e_fnts_limit */ +typedef integer vf_e_fnts_entry; +define_array (vf_e_fnts); + +/* define vf_i_fnts_ptr, vf_i_fnts_array & vf_i_fnts_limit */ +typedef internalfontnumber vf_i_fnts_entry; +define_array (vf_i_fnts); integer fb_offset(void) { @@ -1143,3 +1151,23 @@ *q = '\0'; return name; } + +void allocvffnts(void) +{ + if (vf_e_fnts_array == NULL) { + vf_e_fnts_array = vfefnts; + vf_e_fnts_limit = fontmax; + vf_e_fnts_ptr = vf_e_fnts_array; + vf_i_fnts_array = vfifnts; + vf_i_fnts_limit = fontmax; + vf_i_fnts_ptr = vf_i_fnts_array; + } + alloc_array(vf_e_fnts, 1, fontmax); + vf_e_fnts_ptr++; + alloc_array(vf_i_fnts, 1, fontmax); + vf_i_fnts_ptr++; + if (vf_e_fnts_array != vfefnts) { + vfefnts = vf_e_fnts_array; + vfifnts = vf_i_fnts_array; + } +}
Jie Luo
Frank Küster wrote:
Jie Luo
wrote: Hi,
I modified you patch to get it apply to tetex-3.0_p1, and change font_max back to 2000 in texmf.cnf. Everything is fine now. Thank you very much for patch.
Ah, great - can you publish this modified patch?
Thanks in advance, Frank
Attached.
Thank you! Hm, I get a compilation error: ./tangle pdftex.web pdftex.ch This is TANGLE, Version 4.5 (Web2C 7.5.4) *1*17*25*38*54*72*99*110*115*133*162*173*199*203*207*211*220*256*268*289*297*300*321*332*366*402*464*487*511*539*583*592*644*652*664*666*673*696*779*819*838*858*907*952*1001*1030*1039*1058*1081*1106*1119*1168*1194*1275*1347*1438*1469*1477*1479*1577*1586*1588*1600*1609*1612 Writing the output file.....500.....1000.....1500.....2000.....2500.....3000.....3500.....4000.....4500.....5000.....5500.....6000.....6500.....7000.....7500.....8000.....8500.....9000.....9500... Done. 1468 strings written to string pool file. (No errors were found.) srcdir=. /bin/sh ./web2c/convert pdftex syntax error: Last token = 292 ($), error buffer = `allocvffnts', last id = `allocvffnts' (parameterless procedure). ./web2c/splitup: fatal: No #include pdftexcoerce.h line. ./web2c/convert: conversion of pdftex.p failed, moving dregs: ./web2c/convert: mv pdftex[0-9].c pdftexini.c pdftexd.h pdftexcoerce.h /tmp mv: cannot stat `pdftex[0-9].c': No such file or directory mv: cannot stat `pdftexini.c': No such file or directory This is the Debian package of tetex-src_3.0.tar.gz, with some patches applied. These patches are available at svn//svn.debian.org/svn/debian-tex/tetex-bin/trunk/debian/patches unfortunately the web interface at http://svn.debian.org/wsvn/debian-tex/tetex-bin/trunk/debian/patches/ seems to be down at the moment. The only patch that I think is interesting is patch-poppler, hm. Any ideas what's going on? Regards, Frank -- Dr. Frank Küster Single Molecule Spectroscopy, Protein Folding @ Inst. f. Biochemie, Univ. Zürich Debian Developer (teTeX/TeXLive)
Jie Luo
Frank Küster wrote:
Jie Luo
wrote: Hi,
I modified you patch to get it apply to tetex-3.0_p1, and change font_max back to 2000 in texmf.cnf. Everything is fine now. Thank you very much for patch.
Ah, great - can you publish this modified patch?
Thanks in advance, Frank
Attached.
And of course this patch does work. I had leftovers from previous tries still there. Sorry for the noise, and thanks again, Frank -- Dr. Frank Küster Single Molecule Spectroscopy, Protein Folding @ Inst. f. Biochemie, Univ. Zürich Debian Developer (teTeX/TeXLive)
participants (3)
-
Frank Küster
-
Jie Luo
-
Thanh Han The