Checksum of original font files stored in the cache?

Oliver Buerschaper

17 Jun 2010 17 Jun '10

11:54 p.m.

Dear wise guys, the other day I found myself caught in a trap that might be a frequent source of puzzlement for the casual ConTeXt user :-) I had replaced an OpenType font in the usual OS directory by a new version. Typesetting documents using that font produced surprising results... the PDF files were merrily messed up. I reckon I didn't pay attention to the interplay between font files and the LuaTeX cache... apparently ConTeXt was using the outdated font cache data together with the new font file since a few days later the problem was resolved once I had to regenerate the cache for other reasons. Only then did I notice I had also cured this font problem. Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache? Best, Oliver

Show replies by date

Taco Hoekwater

18 Jun 18 Jun

9:10 a.m.

New subject: Checksum of original font files stored in the cache?

Oliver Buerschaper wrote:

...

Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache?

This sounds useful. As it will need backend support in luatex, I will add a tracker item for this. Best wishes, Taco

Khaled Hosny

9:36 a.m.

On Fri, Jun 18, 2010 at 09:10:47AM +0200, Taco Hoekwater wrote:

...

Oliver Buerschaper wrote:

...
Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache?

This sounds useful. As it will need backend support in luatex, I will add a tracker item for this.

Why? Can't one just store the checksum in the cached lua file, and when loading check if it matches the actual font file, if no discard the cache and reload the file (I thought context is already doing that). -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

Taco Hoekwater

9:40 a.m.

New subject: Checksum of original font files stored in the cache?

Khaled Hosny wrote:

...

On Fri, Jun 18, 2010 at 09:10:47AM +0200, Taco Hoekwater wrote:

...
Oliver Buerschaper wrote:

...
Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache? This sounds useful. As it will need backend support in luatex, I will add a tracker item for this.

Why? Can't one just store the checksum in the cached lua file, and when loading check if it matches the actual font file, if no discard the cache and reload the file (I thought context is already doing that).

Sure, but using the backend is more secure. Once the backend opens the font file, it cannot be altered any more. In a large document, there may be as much as a minute between front end and back end. Using the backend is also faster. Best wishes, Taco

luigi scarso

9:53 a.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 9:40 AM, Taco Hoekwater wrote:

...

Khaled Hosny wrote:

...
On Fri, Jun 18, 2010 at 09:10:47AM +0200, Taco Hoekwater wrote:

...
Oliver Buerschaper wrote:

...
Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache?

This sounds useful. As it will need backend support in luatex, I will add a tracker item for this.

Isn't mtxrun --script fonts reload enough ? -- luigi

Taco Hoekwater

9:55 a.m.

New subject: Checksum of original font files stored in the cache?

luigi scarso wrote:

...

On Fri, Jun 18, 2010 at 9:40 AM, Taco Hoekwater wrote:

...
Khaled Hosny wrote:

...
On Fri, Jun 18, 2010 at 09:10:47AM +0200, Taco Hoekwater wrote:

...
Oliver Buerschaper wrote:

...
Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache? This sounds useful. As it will need backend support in luatex, I will add a tracker item for this.

Isn't mtxrun --script fonts reload enough ?

Only if you actually see the problem happening, which may not be the case. The output could be messed up without any kind of warning from the engine. Best wishes, Taco

luigi scarso

10:05 a.m.

New subject: Checksum of original font files stored in the cache?

...

luigi scarso wrote:

...
On Fri, Jun 18, 2010 at 9:40 AM, Taco Hoekwater wrote:

...
Khaled Hosny wrote:

...
On Fri, Jun 18, 2010 at 09:10:47AM +0200, Taco Hoekwater wrote:

...
Oliver Buerschaper wrote:

...
Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache?

This sounds useful. As it will need backend support in luatex, I will add a tracker item for this.

Isn't mtxrun --script fonts reload enough ?

Only if you actually see the problem happening, which may not be the case. The output could be messed up without any kind of warning from the engine. I mean: one should not simply copy an opentype font into (context) system and expected things to work. Checksum should be check at every runs, which is useless most of the time: in a production env. I don't change fonts for long time , and every change is tested. Maybe a lua script to install fonts is better -- we can eventually

On Fri, Jun 18, 2010 at 9:55 AM, Taco Hoekwater wrote: think to a wxLua gui interface . -- luigi

Oliver Buerschaper

10:16 a.m.

New subject: Checksum of original font files stored in the cache?

...

...
Only if you actually see the problem happening, which may not be the case. The output could be messed up without any kind of warning from the engine. I mean: one should not simply copy an opentype font into (context) system and expected things to work. Checksum should be check at every runs, which is useless most of the time: in a production env. I don't change fonts for long time , and every change is tested. Maybe a lua script to install fonts is better -- we can eventually think to a wxLua gui interface .

I'm not sure. In a desktop environment the user will just update the OpenType font via the OS (e.g. replace a file in the usual OS font directory). From that moment on the font will work in almost any application... don't you think it's reasonable to just have the updated font work in ConTeXt, too? Oliver

Oliver Buerschaper

10:22 a.m.

New subject: Checksum of original font files stored in the cache?

...

...
I mean: one should not simply copy an opentype font into (context) system and expected things to work. Checksum should be check at every runs, which is useless most of the time: in a production env. I don't change fonts for long time , and every change is tested. Maybe a lua script to install fonts is better -- we can eventually think to a wxLua gui interface .

I'm not sure. In a desktop environment the user will just update the OpenType font via the OS (e.g. replace a file in the usual OS font directory). From that moment on the font will work in almost any application... don't you think it's reasonable to just have the updated font work in ConTeXt, too?

Addendum: should an updated font not work out of the box (i.e. change the looks of an existing document) then I think the usual DTP suspects issue a warning. Oliver

luigi scarso

10:31 a.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 10:16 AM, Oliver Buerschaper wrote:

...

...
...
Only if you actually see the problem happening, which may not be the case. The output could be messed up without any kind of warning from the engine. I mean: one should not simply copy an opentype font into (context) system and expected things to work. Checksum should be check at every runs, which is useless most of the time: in a production env. I don't change fonts for long time , and every change is tested. Maybe a lua script to install fonts is better -- we can eventually think to a wxLua gui interface .

...

I'm not sure. In a desktop environment the user will just update the OpenType font via the OS (e.g. replace a file in the usual OS font directory). >From that moment on the font will work in almost any application... don't you think it's reasonable to just have the updated font work in ConTeXt, too?

...

Addendum: should an updated font not work out of the box (i.e. change the looks of an existing document) then I think the usual DTP suspects issue a warning. hm,no as general rule. I prefer to separate context issue from OS issue, just to be sure that I can move my projects in whatever OS I have Also a fonts can not be "good" for context (missing something), but can be ok for OS, and I don't want that an OS upgrade the font that is also used by potentially different projects -- each project has it own sets of fonts and I decide what to share, not the OS. In this way I can also work with linux winxp seven etc without problems.

-- luigi

Mojca Miklavec

11:09 a.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 10:05, luigi scarso wrote:

...

I mean: one should not simply copy an opentype font into (context) system and expected things to work.

Why not? (Do you really want to make things more difficult than they can be on purpose? Who is going to install polish fonts then ... ?)

...

I don't change fonts for long time, and every change is tested.

That's not even entirely true. Fonts in minimals DO change, without running a special installer. Most of time it is also true that ConTeXt version probably changes inbetween as well anyway, so font cache will be rebuilt, but you should not rely on that.

...

Maybe a lua script to install fonts is better -- we can eventually think to a wxLua gui interface .

Or start expanding fontinst to handle OpenType fonts? Anyone? Mojca

Hans Hagen

11:34 a.m.

New subject: Checksum of original font files stored in the cache?

On 18-6-2010 11:09, Mojca Miklavec wrote:

...

On Fri, Jun 18, 2010 at 10:05, luigi scarso wrote:

...
I mean: one should not simply copy an opentype font into (context) system and expected things to work.

Why not? (Do you really want to make things more difficult than they can be on purpose? Who is going to install polish fonts then ... ?)

...
I don't change fonts for long time, and every change is tested.

That's not even entirely true. Fonts in minimals DO change, without running a special installer. Most of time it is also true that ConTeXt version probably changes inbetween as well anyway, so font cache will be rebuilt, but you should not rely on that.

in most cases (and definitely with the polish ones) the file size changes (if only because they keep changing the names) an dthat gets detected the only problem is when a file has exactly the same size (can be the case if only values of numbers changed) i've added an extra check on modification time

...

...
Maybe a lua script to install fonts is better -- we can eventually think to a wxLua gui interface .

Or start expanding fontinst to handle OpenType fonts? Anyone?

no, and i see no reason why ... afaik fontinst does things with type 1 fonts for latex (i never used it but i think they read the files or so) ... even then, we want to drop fonts in directories and not run programs nowadays i always just drop 'm in texmf*/fonts/data/origin/collection so for example in texmf-fonts/fonts/data/khaled/xits (and i no longer bother about separating type1 file into different paths anyway as data is patr of all path vars) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Taco Hoekwater

11:38 a.m.

New subject: Checksum of original font files stored in the cache?

Hans Hagen wrote:

...

nowadays i always just drop 'm in

texmf*/fonts/data/origin/collection

so for example in

texmf-fonts/fonts/data/khaled/xits

(and i no longer bother about separating type1 file into different paths anyway as data is patr of all path vars)

That is also what I have been doing for the past year or so, and it works really well (but not in tl2010-pretest because it does not have the 'data' var extension yet). Best wishes, Taco

luigi scarso

11:40 a.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 11:38 AM, Taco Hoekwater wrote:

...

Hans Hagen wrote:

...
nowadays i always just drop 'm in

texmf*/fonts/data/origin/collection

so for example in

texmf-fonts/fonts/data/khaled/xits

(and i no longer bother about separating type1 file into different paths anyway as data is patr of all path vars)

That is also what I have been doing for the past year or so, and it works really well (but not in tl2010-pretest because it does not have the 'data' var extension yet). Also here, but I always run mtxrun --script fonts reload after

-- luigi

Hans Hagen

12:03 p.m.

On 18-6-2010 11:40, luigi scarso wrote:

...

mtxrun --script fonts reload

this is only needed when you access fonts by name and if a name cannot be found, the name cache will be regenerated automatically Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

luigi scarso

12:17 p.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 12:03 PM, Hans Hagen wrote:

...

On 18-6-2010 11:40, luigi scarso wrote:

...
mtxrun --script fonts reload

this is only needed when you access fonts by name and if a name cannot be found, the name cache will be regenerated automatically And it's ok, because the next runs mkiv will *not* rebuild the cache, and in mkiv we use fonts mostly by name. If I understand correctly, checksum or something similar (why not a binary cmp then ? just one byte is enough) must be done at *every* runs -- make no sense ,most the times check will be useless .

-- luigi

luigi scarso

12:20 p.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 12:17 PM, luigi scarso wrote:

...

On Fri, Jun 18, 2010 at 12:03 PM, Hans Hagen wrote:

...
On 18-6-2010 11:40, luigi scarso wrote:

...
mtxrun --script fonts reload

this is only needed when you access fonts by name and if a name cannot be found, the name cache will be regenerated automatically And it's ok, because the next runs mkiv will *not* rebuild the cache, and in mkiv we use fonts mostly by name. If I understand correctly, checksum or something similar (why not a binary cmp then ? just one byte is enough) must be done at *every* runs -- make no sense ,most the times check will be useless . Also a changing of a font in a middle of a run can be dangerous: the next run can be faked.

-- luigi

luigi scarso

11:38 a.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 11:34 AM, Hans Hagen wrote:

...

nowadays i always just drop 'm in

texmf*/fonts/data/origin/collection

so for example in

texmf-fonts/fonts/data/khaled/xits

(and i no longer bother about separating type1 file into different paths anyway as data is patr of all path vars) And then run mtxrun --script font reload, I suppose -- luigi

Mojca Miklavec

4:37 p.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 11:34, Hans Hagen wrote:

...

On 18-6-2010 11:09, Mojca Miklavec wrote:

...
...
Maybe a lua script to install fonts is better -- we can eventually think to a wxLua gui interface .

Or start expanding fontinst to handle OpenType fonts? Anyone?

no, and i see no reason why ... afaik fontinst does things with type 1 fonts

I thought it was ironic enough to understand it as a joke :) Mojca

luigi scarso

4:45 p.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 4:37 PM, Mojca Miklavec wrote:

...

On Fri, Jun 18, 2010 at 11:34, Hans Hagen wrote:

...
On 18-6-2010 11:09, Mojca Miklavec wrote:

...
...
Maybe a lua script to install fonts is better -- we can eventually think to a wxLua gui interface .

Or start expanding fontinst to handle OpenType fonts? Anyone?

no, and i see no reason why ... afaik fontinst does things with type 1 fonts

I thought it was ironic enough to understand it as a joke :)

Mojca

http://www.wxwidgets.org/ wxWidgets is a C++ library that lets developers create applications for Windows, OS X, Linux and UNIX on 32-bit and 64-bit architectures as well as several mobile platforms including Windows Mobile, iPhone SDK and embedded GTK+. It has popular language bindings for Python, Perl, Ruby and many other languages. Unlike other cross-platform toolkits, wxWidgets gives its applications a truly native look and feel because it uses the platform's native API rather than emulating the GUI. It's also extensive, free, open-source and mature. Why not give it a try, like many others have? http://wxlua.sourceforge.net/ wxLua is a set of bindings to the C++ wxWidgets cross-platform GUI library for the Lua programming language. You can write complex, graphical, interactive programs with wxLua to develop and deliver your software with the programming ease of an interpreted language like Lua. Nearly all of the functionality of wxWidgets is exposed to Lua, meaning that your programs can have windows, dialogs, menus, toolbars, controls, image loading and saving, drawing, sockets, streams, printing, clipboard access... and much more. For Lua x windows already comes with a set of examples of wxLua... erlang come with a wx binding by default... ah yes, wxPython is also good... -- luigi

luigi scarso

19 Jun 19 Jun

5:06 a.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 4:45 PM, luigi scarso wrote:

...

For Lua x windows already comes with a set of examples of wxLua... erlang come with a wx binding by default... ah yes, wxPython is also good... also PS_view uses wxlua http://www.tug.org/texlive/Contents/live/tlpkg/tlpsv/ --[[---------------------------------------------------------------------------- -- PS_View -- previewing utility for Ghostscript interpreter -- Authors: P. Strzelczyk, P. Pianowski. Created: 15.06.1993; Lua part 26.10.2007 -- Address: BOP s.c. Gen. T. Bora-Komorowskiego 24, 80-377 Gda\'nsk, Poland -- bop@bop.com.pl -- Copyright: (c) 2009 BOP s.c. -- Licence: GPL -- Current version: 5.12; 07.10.2009

-- luigi

Taco Hoekwater

9:06 a.m.

New subject: Checksum of original font files stored in the cache?

luigi scarso wrote:

...

On Fri, Jun 18, 2010 at 4:45 PM, luigi scarso wrote:

...
For Lua x windows already comes with a set of examples of wxLua... erlang come with a wx binding by default... ah yes, wxPython is also good... also PS_view uses wxlua

Anyone with recent experience with wxWindows? I tried it a few years ago when it was still 'new and fresh' and at that time it was worryingly unstable like that mis-clicking in a tree widget caused it to crash and the linux combo box selected the items it wanted all by itself. Best wishes, Taco

Mojca Miklavec

1:01 p.m.

New subject: Checksum of original font files stored in the cache?

On Sat, Jun 19, 2010 at 09:06, Taco Hoekwater wrote:

...

Anyone with recent experience with wxWindows?

No recent experience; I can only tell that somebody has written wx terminal in gnuplot (the idea was to make a nice multi-platform terminal), but ever since its introduction (must be a few years already) nobody was able to compile it on Mac (it should be possible, but none of current developers was able to do it properly). I do think that some people have managed to compile it on Windows, but I have no idea about its state. And I have no idea how to link it statically on linux or mac. Mojca

luigi scarso

3:58 p.m.

New subject: Checksum of original font files stored in the cache?

On Sat, Jun 19, 2010 at 1:01 PM, Mojca Miklavec wrote:

...

On Sat, Jun 19, 2010 at 09:06, Taco Hoekwater wrote:

...
Anyone with recent experience with wxWindows?

No recent experience; I can only tell that somebody has written wx terminal in gnuplot (the idea was to make a nice multi-platform terminal), but ever since its introduction (must be a few years already) nobody was able to compile it on Mac (it should be possible, but none of current developers was able to do it properly). I do think that some people have managed to compile it on Windows, but I have no idea about its state. And I have no idea how to link it statically on linux or mac.

Mojca

I'm impressed by http://code.google.com/p/luaforwindows/ I 'm also used to visit wxwidgets from time to time and I believe that 2.8 branch is good I have compiled gnuplot4.4 because I like its lua interface, but not with wxterminal -- I can check eventually BTW,IUP can be another guit toolkit to consider: it's in Lua, it has some TeX ideas, it comes from http://www.tecgraf.puc-rio.br/iup (Aditya also starts to use wxMaxima) -- luigi

Khaled Hosny

5:21 p.m.

On Sat, Jun 19, 2010 at 09:06:25AM +0200, Taco Hoekwater wrote:

...

luigi scarso wrote:

...
On Fri, Jun 18, 2010 at 4:45 PM, luigi scarso wrote:

...
For Lua x windows already comes with a set of examples of wxLua... erlang come with a wx binding by default... ah yes, wxPython is also good... also PS_view uses wxlua

Anyone with recent experience with wxWindows? I tried it a few years ago when it was still 'new and fresh' and at that time it was worryingly unstable like that mis-clicking in a tree widget caused it to crash and the linux combo box selected the items it wanted all by itself.

Not so much recent, but my experience with wx was always poor, it have very poor rtl support compared to native gtk, it might look native but it doesn't feel so; I can always tell this is a wx application by glance. Regards, Khaled -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

Hans Hagen

18 Jun 18 Jun

5:24 p.m.

On 18-6-2010 4:37, Mojca Miklavec wrote:

...

On Fri, Jun 18, 2010 at 11:34, Hans Hagen wrote:

...
On 18-6-2010 11:09, Mojca Miklavec wrote:

...
...
Maybe a lua script to install fonts is better -- we can eventually think to a wxLua gui interface .

Or start expanding fontinst to handle OpenType fonts? Anyone?

no, and i see no reason why ... afaik fontinst does things with type 1 fonts

I thought it was ironic enough to understand it as a joke :)

well, i've learned to take you serious ... actually i already imagined you trying to apply fontinst to antykwa as it would give us n*m extra fonts metric files which then results in another thousand lines of typescripts which taco then has to check into texlive -) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

luigi scarso

11:35 a.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 11:09 AM, Mojca Miklavec wrote:

...

On Fri, Jun 18, 2010 at 10:05, luigi scarso wrote:

...
I mean: one should not simply copy an opentype font into (context) system and expected things to work.

Why not? (Do you really want to make things more difficult than they can be on purpose? Who is going to install polish fonts then ... ?) Because you must check md5 font checksum of your document at every runs and on average the change of fonts <<< than number of context runs .

...

...
I don't change fonts for long time, and every change is tested.

That's not even entirely true. Fonts in minimals DO change, without running a special installer. Most of time it is also true that ConTeXt version probably changes inbetween as well anyway, so font cache will be rebuilt, but you should not rely on that.

...

...
Maybe a lua script to install fonts is better -- we can eventually think to a wxLua gui interface . Or start expanding fontinst to handle OpenType fonts? Anyone? it's trivial for an opentype font: 1)copy the file into an minimals dir (default tex/texmf/fonts/opentype//)

I'm not speaking of fonts that come with minimals: every change is followed by a mtxrun update, so it's not a problem. 2) run mtxrun --script fonts reload Where are the problems ? (texfonts was much more complicated and none worry about it ) -- luigi

Khaled Hosny

10:08 a.m.

On Fri, Jun 18, 2010 at 09:55:01AM +0200, Taco Hoekwater wrote:

...

luigi scarso wrote:

...
On Fri, Jun 18, 2010 at 9:40 AM, Taco Hoekwater wrote:

...
Khaled Hosny wrote:

...
On Fri, Jun 18, 2010 at 09:10:47AM +0200, Taco Hoekwater wrote:

...
Oliver Buerschaper wrote:

...
Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache? This sounds useful. As it will need backend support in luatex, I will add a tracker item for this.

Isn't mtxrun --script fonts reload enough ?

Only if you actually see the problem happening, which may not be the case. The output could be messed up without any kind of warning from the engine.

I'm not sure what the checksum field in tfm table is used for, but it can store the checksum of the actual font file (the lua end should fill it in lua loaded fonts) and the backend can warn the user in the case of checksum mismatch. -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

Hans Hagen

10:48 a.m.

New subject: Checksum of original font files stored in the cache?

On 18-6-2010 9:55, Taco Hoekwater wrote:

...

luigi scarso wrote:

...
On Fri, Jun 18, 2010 at 9:40 AM, Taco Hoekwater wrote:

...
Khaled Hosny wrote:

...
On Fri, Jun 18, 2010 at 09:10:47AM +0200, Taco Hoekwater wrote:

...
Oliver Buerschaper wrote:

...
Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache? This sounds useful. As it will need backend support in luatex, I will add a tracker item for this.

Isn't mtxrun --script fonts reload enough ?

Only if you actually see the problem happening, which may not be the case. The output could be messed up without any kind of warning from the engine.

is there unique info in the font file that we can use? version number that is guaranteed unique? if so, we can pass that to the backend then, if we can determine that something is wrong (a flag in statistics), we can at the end of a run issue a message, generate that specific font, flag the tuc file and we get a second run automatically taco and i can discuss a few such strategies off list Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Mojca Miklavec

11:05 a.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 10:48, Hans Hagen wrote:

...

is there unique info in the font file that we can use? version number that is guaranteed unique? if so, we can pass that to the backend

That almost reminds me on version numbers in ConTeXt files (the number written in header that almost never changes). If font vendors are equally careful to change version numbers, that doesn't guarantee uniqueness. Mojca

Hans Hagen

11:36 a.m.

On 18-6-2010 11:05, Mojca Miklavec wrote:

...

On Fri, Jun 18, 2010 at 10:48, Hans Hagen wrote:

...
is there unique info in the font file that we can use? version number that is guaranteed unique? if so, we can pass that to the backend

That almost reminds me on version numbers in ConTeXt files (the number written in header that almost never changes). If font vendors are equally careful to change version numbers, that doesn't guarantee uniqueness.

indeed. but we don't sell ... if you sell you can employ prople to change numbers (the dates in the context files are more an indication of whe something showed up) anyhow, i have been thinking of generating (with each beta) an automatically added key 'changedate' or so for changed files but i need to be really bored (or watching a one star movie on tv) to do that Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Taco Hoekwater

11:41 a.m.

New subject: Checksum of original font files stored in the cache?

Hans Hagen wrote:

...

On 18-6-2010 11:05, Mojca Miklavec wrote:

...
On Fri, Jun 18, 2010 at 10:48, Hans Hagen wrote:

...
is there unique info in the font file that we can use? version number that is guaranteed unique? if so, we can pass that to the backend

That almost reminds me on version numbers in ConTeXt files (the number written in header that almost never changes). If font vendors are equally careful to change version numbers, that doesn't guarantee uniqueness.

indeed. but we don't sell ... if you sell you can employ prople to change numbers (the dates in the context files are more an indication of whe something showed up)

anyhow, i have been thinking of generating (with each beta) an automatically added key 'changedate' or so for changed files but i need to be really bored (or watching a one star movie on tv) to do that

Tonight at 20:30 on RTL7: Timecop II: The Berlin decision I think that qualifies ;) Seriously: yes this field would be quite useful to have added to the source automatically. Even better for me personally would be if you could generate an md5sum of the distributed file, so I can easily check whether I made local changes. Best wishes, Taco

Oliver Buerschaper

12:11 p.m.

New subject: Checksum of original font files stored in the cache?

...

...
anyhow, i have been thinking of generating (with each beta) an automatically added key 'changedate' or so for changed files but i need to be really bored (or watching a one star movie on tv) to do that

Tonight at 20:30 on RTL7:

Timecop II: The Berlin decision

I think that qualifies ;)

Seriously: yes this field would be quite useful to have added to the source automatically. Even better for me personally would be if you could generate an md5sum of the distributed file, so I can easily check whether I made local changes.

Wouldn't this rather be an ideal task for, say, git? I mean tracking changes in ConTeXt source files... Oliver

Taco Hoekwater

12:19 p.m.

New subject: Checksum of original font files stored in the cache?

Oliver Buerschaper wrote:

...

...
I think that qualifies ;)

Seriously: yes this field would be quite useful to have added to the source automatically. Even better for me personally would be if you could generate an md5sum of the distributed file, so I can easily check whether I made local changes.

Wouldn't this rather be an ideal task for, say, git? I mean tracking changes in ConTeXt source files...

Probably, but I don't want a git repo in $TEXMF so that would mean even more bookkeeping.

Hans Hagen

12:12 p.m.

New subject: Checksum of original font files stored in the cache?

On 18-6-2010 11:41, Taco Hoekwater wrote:

...

Tonight at 20:30 on RTL7:

Timecop II: The Berlin decision

I think that qualifies ;)

must be indeed as it is not even discussed at http://outlawvern.com/ (one of my favourite sites) so it must be real bad

...

could generate an md5sum of the distributed file, so I can easily check

hm, will think about it Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Khaled Hosny

11:33 a.m.

New subject: Checksum of original font files stored in the cache?

On Fri, Jun 18, 2010 at 10:48:42AM +0200, Hans Hagen wrote:

...

On 18-6-2010 9:55, Taco Hoekwater wrote:

...
luigi scarso wrote:

...
On Fri, Jun 18, 2010 at 9:40 AM, Taco Hoekwater wrote:

...
Khaled Hosny wrote:

...
On Fri, Jun 18, 2010 at 09:10:47AM +0200, Taco Hoekwater wrote:

...
Oliver Buerschaper wrote: >Do you think it would be a good idea to store a checksum of the >actual font file used while generating the cache? This sounds useful. As it will need backend support in luatex, I will add a tracker item for this.

Isn't mtxrun --script fonts reload enough ?

Only if you actually see the problem happening, which may not be the case. The output could be messed up without any kind of warning from the engine.

is there unique info in the font file that we can use? version number that is guaranteed unique? if so, we can pass that to the backend

There is nothing guaranteed, but fonts can have a UniqueID (inside ttfnames) that should be, if present, unique. -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

Khaled Hosny

10:03 a.m.

On Fri, Jun 18, 2010 at 09:40:55AM +0200, Taco Hoekwater wrote:

...

Khaled Hosny wrote:

...
On Fri, Jun 18, 2010 at 09:10:47AM +0200, Taco Hoekwater wrote:

...
Oliver Buerschaper wrote:

...
Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache? This sounds useful. As it will need backend support in luatex, I will add a tracker item for this.

Why? Can't one just store the checksum in the cached lua file, and when loading check if it matches the actual font file, if no discard the cache and reload the file (I thought context is already doing that).

Sure, but using the backend is more secure. Once the backend opens the font file, it cannot be altered any more. In a large document, there may be as much as a minute between front end and back end. Using the backend is also faster.

Aha, I didn't think about this use case, I was thinking more about a context cached font changing before processing the document. -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

Hans Hagen

10:36 a.m.

New subject: Checksum of original font files stored in the cache?

On 17-6-2010 11:54, Oliver Buerschaper wrote:

...

Dear wise guys,

the other day I found myself caught in a trap that might be a frequent source of puzzlement for the casual ConTeXt user :-)

I had replaced an OpenType font in the usual OS directory by a new version. Typesetting documents using that font produced surprising results... the PDF files were merrily messed up. I reckon I didn't pay attention to the interplay between font files and the LuaTeX cache... apparently ConTeXt was using the outdated font cache data together with the new font file since a few days later the problem was resolved once I had to regenerate the cache for other reasons. Only then did I notice I had also cured this font problem.

Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache?

no, as calculating the checksum is quite slow normally context will detect a change in filesize so i wonder what went wrong at your end Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Khaled Hosny

11:35 a.m.

On Fri, Jun 18, 2010 at 10:36:36AM +0200, Hans Hagen wrote:

...

On 17-6-2010 11:54, Oliver Buerschaper wrote:

...
Dear wise guys,

the other day I found myself caught in a trap that might be a frequent source of puzzlement for the casual ConTeXt user :-)

I had replaced an OpenType font in the usual OS directory by a new version. Typesetting documents using that font produced surprising results... the PDF files were merrily messed up. I reckon I didn't pay attention to the interplay between font files and the LuaTeX cache... apparently ConTeXt was using the outdated font cache data together with the new font file since a few days later the problem was resolved once I had to regenerate the cache for other reasons. Only then did I notice I had also cured this font problem.

Do you think it would be a good idea to store a checksum of the actual font file used while generating the cache?

no, as calculating the checksum is quite slow

normally context will detect a change in filesize so i wonder what went wrong at your end

File size is not reliable, I do frequently change fonts and context doesn't reload the font, so I have to delete the cache by hand. -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

5299

Age (days ago)

5301

Last active (days ago)

List overview

Download

38 comments

6 participants

participants (6)

Hans Hagen
Khaled Hosny
luigi scarso
Mojca Miklavec
Oliver Buerschaper
Taco Hoekwater

Checksum of original font files stored in the cache?

tags

participants (6)