Using virtual fonts defined with lua-code
Hello, The luatex documentation contains in section "7.2.2 Example virtual font" an example how to define a virtual font "cmr10-red". The font is defined by adding the code to the define_font callback. Does some interface exists in contex/mkiv which allows to use such virtual fonts together with the normal font loading code? E.g. could I move the code for the font to some file ("cmr10-red.vflua") so that a request \font\mytest="cmr10-red" will simply find this file and use the code like vf, tfm or otf-files are found and used when needed? -- Ulrike Fischer
On 8-2-2011 10:40, Ulrike Fischer wrote:
Hello,
The luatex documentation contains in section "7.2.2 Example virtual font" an example how to define a virtual font "cmr10-red". The font is defined by adding the code to the define_font callback.
Does some interface exists in contex/mkiv which allows to use such virtual fonts together with the normal font loading code? E.g. could I move the code for the font to some file ("cmr10-red.vflua") so that a request \font\mytest="cmr10-red" will simply find this file and use the code like vf, tfm or otf-files are found and used when needed?
It's relatively easy to add support for that. However, when I've done that Khaled has to make sure that it also works in his wrapper code as I don't know to what extend he overloads code. I can only test in the plain variant and context. Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Am Tue, 08 Feb 2011 15:13:11 +0100 schrieb Hans Hagen:
On 8-2-2011 10:40, Ulrike Fischer wrote:
Hello,
The luatex documentation contains in section "7.2.2 Example virtual font" an example how to define a virtual font "cmr10-red". The font is defined by adding the code to the define_font callback.
Does some interface exists in contex/mkiv which allows to use such virtual fonts together with the normal font loading code? E.g. could I move the code for the font to some file ("cmr10-red.vflua") so that a request \font\mytest="cmr10-red" will simply find this file and use the code like vf, tfm or otf-files are found and used when needed?
It's relatively easy to add support for that.
That would be fine. There are quite often on various lists questions about how to "correct" a font, e.g. adjust kerning, or add a missing glyph etc and a interface for "virtual lua fonts" would make such corrections much easier. The example in the documentation uses as base font cmr10 and loads it with f = font.read_tfm('cmr10',size). How should one load the real font when it is an otf?
However, when I've done that Khaled has to make sure that it also works in his wrapper code as I don't know to what extend he overloads code. I can only test in the plain variant and context.
If you adapt the context files I can make some tests by exchanging the luaotfload files and tell Khaled the results. -- Ulrike Fischer
On 8-2-2011 3:41, Ulrike Fischer wrote:
Am Tue, 08 Feb 2011 15:13:11 +0100 schrieb Hans Hagen:
On 8-2-2011 10:40, Ulrike Fischer wrote:
Hello,
The luatex documentation contains in section "7.2.2 Example virtual font" an example how to define a virtual font "cmr10-red". The font is defined by adding the code to the define_font callback.
Does some interface exists in contex/mkiv which allows to use such virtual fonts together with the normal font loading code? E.g. could I move the code for the font to some file ("cmr10-red.vflua") so that a request \font\mytest="cmr10-red" will simply find this file and use the code like vf, tfm or otf-files are found and used when needed?
It's relatively easy to add support for that.
That would be fine. There are quite often on various lists questions about how to "correct" a font, e.g. adjust kerning, or add a missing glyph etc and a interface for "virtual lua fonts" would make such corrections much easier.
The example in the documentation uses as base font cmr10 and loads it with f = font.read_tfm('cmr10',size). How should one load the real font when it is an otf?
the lua file has to return a function that then gets a specification and has to return a tfmtable, something like return myhackery(specification) local t = fonts.tfm.read(specification) -- mess around with t return t end but I didn't test that. Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On Tue, Feb 08, 2011 at 03:13:11PM +0100, Hans Hagen wrote:
On 8-2-2011 10:40, Ulrike Fischer wrote:
Hello,
The luatex documentation contains in section "7.2.2 Example virtual font" an example how to define a virtual font "cmr10-red". The font is defined by adding the code to the define_font callback.
Does some interface exists in contex/mkiv which allows to use such virtual fonts together with the normal font loading code? E.g. could I move the code for the font to some file ("cmr10-red.vflua") so that a request \font\mytest="cmr10-red" will simply find this file and use the code like vf, tfm or otf-files are found and used when needed?
It's relatively easy to add support for that. However, when I've done that Khaled has to make sure that it also works in his wrapper code as I don't know to what extend he overloads code. I can only test in the plain variant and context.
I keep it as minimal as possible (I've even some "feature requests" to keep our local changes more smaller, but no time to check it now). Regards, Khaled -- Khaled Hosny Egyptian
On 8-2-2011 3:46, Khaled Hosny wrote:
On Tue, Feb 08, 2011 at 03:13:11PM +0100, Hans Hagen wrote:
On 8-2-2011 10:40, Ulrike Fischer wrote:
Hello,
The luatex documentation contains in section "7.2.2 Example virtual font" an example how to define a virtual font "cmr10-red". The font is defined by adding the code to the define_font callback.
Does some interface exists in contex/mkiv which allows to use such virtual fonts together with the normal font loading code? E.g. could I move the code for the font to some file ("cmr10-red.vflua") so that a request \font\mytest="cmr10-red" will simply find this file and use the code like vf, tfm or otf-files are found and used when needed?
It's relatively easy to add support for that. However, when I've done that Khaled has to make sure that it also works in his wrapper code as I don't know to what extend he overloads code. I can only test in the plain variant and context.
I keep it as minimal as possible (I've even some "feature requests" to keep our local changes more smaller, but no time to check it now).
You only need to load one extra file: loadmodule('font-lua.lua') Then, if \font\mine=file:luatex-fonts-demo-vf-1.lua at 12pt \mine \input tufte \par works ok, you're fine, otherwise you can delay integration. As lua lookup is not part of any font scheme best use an explicit suffix (the file: is not mandate, at least not in context). It's anyway the user's responsibility to return a proper tfm table so no one can bother us about it not working as expected -) (For context users there is actually some more virtual building code but as it's somewhat experimental I will clean that up before documenting it.) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Am Tue, 08 Feb 2011 17:07:39 +0100 schrieb Hans Hagen:
It's relatively easy to add support for that. However, when I've done that Khaled has to make sure that it also works in his wrapper code as I don't know to what extend he overloads code. I can only test in the plain variant and context.
I keep it as minimal as possible (I've even some "feature requests" to keep our local changes more smaller, but no time to check it now).
You only need to load one extra file:
loadmodule('font-lua.lua')
Then, if
\font\mine=file:luatex-fonts-demo-vf-1.lua at 12pt
\mine \input tufte \par
works ok, you're fine, otherwise you can delay integration.
I downloaded the (hopefully) current lua-files from http://minimals.contextgarden.net/current/context/alpha/tex/context/base/, renamed them and tested with luaotfload [2011/02/04 v1.23 OpenType layout system] (as a first step without the new font-lua.lua). It doesn't work currently. I identified already some problematic changes between the versions used currently by luaotfload and the newest context files: * register_base_substitution has been renamed to registerbasesubstitution (used in luaotfload.lua line 137) * fonts.define seems to be named fonts.definers now (luaotfload.lua line 87) * Instead of logs.report the context files use now report_prepare (with a different syntax) which is perhaps the source of the error I'm getting now: J:/MiKTeX2.9/tex/luatex/luaotfload/otfl-luat-ovr.lua:19: bad arg ument #2 to 'format' (string expected, got nil). -- Ulrike Fischer
On 9-2-2011 10:55, Ulrike Fischer wrote:
I downloaded the (hopefully) current lua-files from http://minimals.contextgarden.net/current/context/alpha/tex/context/base/, renamed them and tested with luaotfload [2011/02/04 v1.23 OpenType layout system] (as a first step without the new font-lua.lua).
You can best test with the plain format first: luatex --ini --fmt=luatex-plain etc as that is the benchmark.
* register_base_substitution has been renamed to registerbasesubstitution (used in luaotfload.lua line 137)
* fonts.define seems to be named fonts.definers now (luaotfload.lua line 87)
indeed, and some probably will get cleaned up as some code is still experimental
* Instead of logs.report the context files use now report_prepare (with a different syntax) which is perhaps the source of the error I'm getting now:
You can probably define function logs.reporter(c) return function(...) texio.write(c .. " : " .. string.format(...)) end end somewhere if you want some logging. Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Am Tue, 08 Feb 2011 17:07:39 +0100 schrieb Hans Hagen:
Does some interface exists in contex/mkiv which allows to use such virtual fonts together with the normal font loading code? E.g. could I move the code for the font to some file ("cmr10-red.vflua") so that a request \font\mytest="cmr10-red" will simply find this file and use the code like vf, tfm or otf-files are found and used when needed?
It's relatively easy to add support for that. However, when I've done that Khaled has to make sure that it also works in his wrapper code as I don't know to what extend he overloads code. I can only test in the plain variant and context.
You only need to load one extra file:
loadmodule('font-lua.lua')
Then, if
\font\mine=file:luatex-fonts-demo-vf-1.lua at 12pt
What are the search pathes for such virtual lua fonts? I suggested the file extension ".vflua" because I thought it would be senseful if not the search path for general .lua files is used. I wanted to put this virtual fonts in a dedicated location e.g. fonts/vflua/.... -- Ulrike Fischer
On 9-2-2011 12:23, Ulrike Fischer wrote:
Am Tue, 08 Feb 2011 17:07:39 +0100 schrieb Hans Hagen:
Does some interface exists in contex/mkiv which allows to use such virtual fonts together with the normal font loading code? E.g. could I move the code for the font to some file ("cmr10-red.vflua") so that a request \font\mytest="cmr10-red" will simply find this file and use the code like vf, tfm or otf-files are found and used when needed?
It's relatively easy to add support for that. However, when I've done that Khaled has to make sure that it also works in his wrapper code as I don't know to what extend he overloads code. I can only test in the plain variant and context.
You only need to load one extra file:
loadmodule('font-lua.lua')
Then, if
\font\mine=file:luatex-fonts-demo-vf-1.lua at 12pt
What are the search pathes for such virtual lua fonts? I suggested the file extension ".vflua" because I thought it would be senseful if not the search path for general .lua files is used. I wanted to put this virtual fonts in a dedicated location e.g. fonts/vflua/....
I'm not going to hard code something like that as it's a macro specific issue. You can try (untested): fonts.formats.vflua = "lua" Paths and lookups are also a macro package issue and as khaled probably has specific file lookup code for latex, I suppose that the lookup can be handled there. If not, we can make the remapper extensible. Anyhow, "tex" with "other text files" sounds good enough for me. Also, I suppose that names should be unique and the files that you make in some latex specific (sub)path in order not to clash. As this is not that generic, maybe something: tex/latex/fonts/vflua is best. Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Am Wed, 09 Feb 2011 14:30:54 +0100 schrieb Hans Hagen:
\font\mine=file:luatex-fonts-demo-vf-1.lua at 12pt
What are the search pathes for such virtual lua fonts? I suggested the file extension ".vflua" because I thought it would be senseful if not the search path for general .lua files is used. I wanted to put this virtual fonts in a dedicated location e.g. fonts/vflua/....
I'm not going to hard code something like that as it's a macro specific issue. You can try (untested):
fonts.formats.vflua = "lua"
Paths and lookups are also a macro package issue and as khaled probably has specific file lookup code for latex, I suppose that the lookup can be handled there. If not, we can make the remapper extensible.
Anyhow, "tex" with "other text files" sounds good enough for me. Also, I suppose that names should be unique and the files that you make in some latex specific (sub)path in order not to clash. As this is not that generic, maybe something: tex/latex/fonts/vflua is best.
While I certainly in the end want to use the (virtual) fonts also with LaTeX I don't want to define them in a LaTeX-specific way. A standard virtual font (tfm + vf) can be used with latex, plain and context with an identical syntax so why should this be different for this new type of virtual font only because they are based on lua-code? Also if a font is defective (e.g. wrong kerning, missing glyphs, faulty glyph names, missing open type features ...) a virtual font which improves this font should be useful for all formats which can use it, so I don't think that a format specific location is sensible. (I must say I'm wondering a bit why problems with fonts are so seldom discussed in the context list. In the last weeks I have seen in the xetex list a discussion how to add missing glyphs to a font, a discussion about kerning flaws, discussions about bad accent placement, in another group someone missed the "m with dot" in a font, in the luatex.user list you can find a message about a problem with fea-files and kerning, and so one ... Why do context user seems to have no problems with defective fonts? Do they use a smaller set of fonts?) -- Ulrike Fischer
On 9-2-2011 5:09, Ulrike Fischer wrote:
While I certainly in the end want to use the (virtual) fonts also with LaTeX I don't want to define them in a LaTeX-specific way. A standard virtual font (tfm + vf) can be used with latex, plain and context with an identical syntax so why should this be different for this new type of virtual font only because they are based on lua-code?
The you can put them under tex/fonts/generic i suppose; but even then, there is no reason to assume that context or latex provides additional lua code in the engine that is used in such a font.
Also if a font is defective (e.g. wrong kerning, missing glyphs, faulty glyph names, missing open type features ...) a virtual font which improves this font should be useful for all formats which can use it, so I don't think that a format specific location is sensible.
We have a different machinery for things like that and support is integrated into context (and targets at context). Also, there is an experimental vf creation interface that will be extended.
(I must say I'm wondering a bit why problems with fonts are so seldom discussed in the context list. In the last weeks I have seen in the xetex list a discussion how to add missing glyphs to a font, a discussion about kerning flaws, discussions about bad accent placement, in another group someone missed the "m with dot" in a font, in the luatex.user list you can find a message about a problem with fea-files and kerning, and so one ... Why do context user seems to have no problems with defective fonts? Do they use a smaller set of fonts?)
I don't know. Personally I just reject a font if it's flawed esp because there are enough fonts out there that are ok. And I definitely (as context user) don't want to go along the route of using all kind of patches to fonts, because we then end up with the same mess as we have with special encodings in traditional tex. Also, in the past I've found out that one cannot rely to much on provided metrics (also because latex assume special values, as is the case in some tfm metrics) and this is why in mkii (pdftex variant) we always generated metrics a directly from afm files. The lm and gyre font projects are in fact triggered by people within the context community as a solution to all these incomplete fonts that have been used for ages. Fortunately these projects soon became generic and were adopted and supported by latex as well. (In context we switched immediately to them.) I'm glad that we got rid of the font mess and would like to keep it that way. If a font is bugged ... preferably fix the font. (We currently impose a few runtime fixes to e.g. cambria but I'd expect that to be a temporary thing.) Don't get me wrong, context users can do a lot of tweaking if they want, which can be seen from questions on the list with respect to layout, but maybe they don't want to create a dependency on tweaks in fonts. Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Am Wed, 09 Feb 2011 17:36:41 +0100 schrieb Hans Hagen:
Also if a font is defective (e.g. wrong kerning, missing glyphs, faulty glyph names, missing open type features ...) a virtual font which improves this font should be useful for all formats which can use it, so I don't think that a format specific location is sensible.
We have a different machinery for things like that and support is integrated into context (and targets at context). Also, there is an experimental vf creation interface that will be extended.
Where is this interface? Does some documentation exists about how it works and how it can be used?
(I must say I'm wondering a bit why problems with fonts are so seldom discussed in the context list. In the last weeks I have seen in the xetex list a discussion how to add missing glyphs to a font, a discussion about kerning flaws, discussions about bad accent placement, in another group someone missed the "m with dot" in a font, in the luatex.user list you can find a message about a problem with fea-files and kerning, and so one ... Why do context user seems to have no problems with defective fonts? Do they use a smaller set of fonts?)
I don't know. Personally I just reject a font if it's flawed esp because there are enough fonts out there that are ok. And I definitely (as context user) don't want to go along the route of using all kind of patches to fonts, because we then end up with the same mess as we have with special encodings in traditional tex.
Well it is certainly easier if there are less encodings. But the small encodings had one advantage: if you used a e.g. T1-encoded font you not only knew which characters are encoded and their position but also that the characters are actually present. You could safely switch from one font to another. With unicode fonts this is no longer the case. If you switch fonts there is always the danger that a char or an accent suddenly disappears.
The lm and gyre font projects are in fact triggered by people within the context community as a solution to all these incomplete fonts that have been used for ages. Fortunately these projects soon became generic and were adopted and supported by latex as well. (In context we switched immediately to them.)
The lm and gyre fonts are fine. But they cover only a small part of the glyphs used in the world. Many of the discussions I see are started by people trying to use non-western/non-latin scripts.
I'm glad that we got rid of the font mess and would like to keep it that way. If a font is bugged ... preferably fix the font. (We currently impose a few runtime fixes to e.g. cambria but I'd expect that to be a temporary thing.) Don't get me wrong, context users can do a lot of tweaking if they want, which can be seen from questions on the list with respect to layout, but maybe they don't want to create a dependency on tweaks in fonts.
Fixing a font needs either the rights to do it oneself or the will of the author(s) to do it. Both is often not the case. And fixing a font may remove a dependency to a virtual font but it will add a dependency to the fixed font version - which can get quite difficult if more than one "fixed" version exists. -- Ulrike Fischer
On 10-2-2011 11:03, Ulrike Fischer wrote:
Where is this interface? Does some documentation exists about how it works and how it can be used?
Some of it has been reported in articles (and mk.pdf) and I will document it in more detail when I've cleaned it up and feel satisfied about it. Also, it's context speficic (I guess) and it is not on my agenda to make every context feature portable (the concepts between context and plain/latex/.. simply differ too much).
Well it is certainly easier if there are less encodings. But the small encodings had one advantage: if you used a e.g. T1-encoded font you not only knew which characters are encoded and their position but also that the characters are actually present. You
that's not 100% true ... in the past I've ran into situations where fonts missed glyphs; also, having a bad replacement glyph is also no option (funny ogoneks are an example)
could safely switch from one font to another. With unicode fonts this is no longer the case. If you switch fonts there is always the danger that a char or an accent suddenly disappears.
One always need to check each font and the problem with e.g. otf is not so much the coverage but more the fact that things like features are somewhat unpredictable (defaults, correct implemenation, etc) and successive versions can differ. So, patching them then also boils down to keeping track of all kind of changes in releases. Nu fun. Anyway, in context mkiv there are several extension mechanisms (aka font goodies) and some of them also depend on support in the core (context) machinery. I expect more of them and virtual trickery fits into that picture. What ends up there is also user driven.
The lm and gyre fonts are fine. But they cover only a small part of the glyphs used in the world. Many of the discussions I see are started by people trying to use non-western/non-latin scripts.
Sure. Anyhow, I'm not going to spend time following discussions on the pdftex and xetex list as I don't use these engines and context support for them is frozen. If context users have demands in this area I'm quite willing to fulfill them in mkiv using appropriate mechanisms and interfaces. Support for advanced arabic (using additional features and dedicated optimizers) is an example.
Fixing a font needs either the rights to do it oneself or the will of the author(s) to do it. Both is often not the case. And fixing a font may remove a dependency to a virtual font but it will add a dependency to the fixed font version - which can get quite difficult if more than one "fixed" version exists.
True, but as I mentioned, there are many fonts out there and one can try to avoid the crappy ones. (btw, context mkiv has some features for adding missing glyphs, which might be why users don't complain too much here) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
participants (3)
-
Hans Hagen -
Khaled Hosny -
Ulrike Fischer