Corsair wrote:

On Thu, May 21, 2009 at 11:55:45AM +0200, Hans Hagen wrote:

...

open type fonts have no italic correction info (except in math)

But I notice that using the same fonts in XeTeX produces italic correction. Is it fake?

I guess it is using the glyph boundingbox.

Corsair wrote:

On Thu, May 21, 2009 at 01:59:14PM +0200, Taco Hoekwater wrote:

...

Corsair wrote:

...

On Thu, May 21, 2009 at 11:55:45AM +0200, Hans Hagen wrote:

...

open type fonts have no italic correction info (except in math) But I notice that using the same fonts in XeTeX produces italic correction. Is it fake? I guess it is using the glyph boundingbox.

Thank you. This sounds reasonable. Is there any way I can achieve this in MkIV? I'm currently using \def\/{\kern0.1em}, which is kinda dirty...

Hans could implement something like this easily, but whether it does much good is doubtful (that square box does not actually tell you where something sticks out, just that it does). Best wishes, Taco

Khaled Hosny wrote:

Not very helpful in this situation, but FontForge has a non-standard italic correction (ITLC) table[1], may be TeX related OpenTyp font projects like Latin Modern and Gyre fonts can use it?

That would perhaps not be a bad idea. If that table is there then luatex will automatically use it (it is a subtable of 'TeX ', which also contains height and depth information, and font dimensions). Best wishes, Taco

Hans Hagen wrote:

Taco Hoekwater wrote:

...

Khaled Hosny wrote:

...

Not very helpful in this situation, but FontForge has a non-standard italic correction (ITLC) table[1], may be TeX related OpenTyp font projects like Latin Modern and Gyre fonts can use it?

That would perhaps not be a bad idea. If that table is there then luatex will automatically use it (it is a subtable of 'TeX ', which also contains height and depth information, and font dimensions).

so, that data would end up in a regular feature/lookup? of is it an entry in the glyph?

They are automatically merged into the glyph, as glyph.italic_correction glyph.tex_height glyph.tex_depth Best wishes, Taco

but instead of arbitrary adding a 0.25em before and 1em after the punctuation mark you should use the real nnbsp (U+202F) before and real normal space (U+0020) after.

I don't think so. Space characters don't mix very well with TeX glue and should best be avoided, generally speaking. In particular, all inter-word spaces that are input in the TeX source as one or more of U+0020 are simply ignored, and replaced by normal inter-word glue, with its appropriate stretchability and shrinkability. This has always been the case in TeX and is not going to change. All other types of Unicode spaces should really, in my opinion, be processed in the same way, while respecting their additional properties in the case of non-breakable spaces, for instance. In addition, characters like U+202F are very badly supported across fonts, and if you take in account the fact that the most appropriate width will probably change depending on the language, you're likely to observe much more arbitrary results if you use the glyph for that character in font. I seriously doubt you want to rely on the font for that.

Why? Let me take your example again:

{\setcharacterspacing[frenchpunctuation]a? aa? aaa? abba?}

a\,? aa\,? aaa\,? abba\,?

Surprise: the first line is longer than the second. It's because sizes of the U+0020 and U+202F depend on the font design, their size are not exactly 1em and 0.25em.

That's not the reason. The reason is simply that \, is defined as a \kern by one sixth of an em (see core-spa.mkiv: it's equivalent to \thinspace, which is \kern .16667em). In the first line, the value of .25em is defined in core-spa.mkiv; you can redefine it if you want. In any case, every space is completely controlled by ConTeXt, we don't let the font mess around. For that matter, Latin Modern doesn't have a glyph for U+202F, so if we'd use it, we'd just see nothing: there would be no space at all, see attached file.

Moreover, this is true that in french typography we should use a thin space before some punctuation mark, and a thin space in France *was* one fourth of an em. BUT due to modern digital typography, a thin space now correspond to something like one fifth of an em. In digital typography, the normal space, i.e. inter-word separation, i.e. "espace justifiante" in French, has generally a size around one fourth of an em, calculated by the engine (you know that better than me). The thin space is a bit more than a half of the inter-word separation, i.e. between 1/8 and 1/6 of an em.

All this really calls for more coordination in order to produce decent specifications, in my opinion. If you think ConTeXt's default should be different, it's fine and I encourage you to contact Sébastien to discuss about it. Report then to Hans and Peter for the implementation.

So I *really* believe that you should not define space before and after punctuation with "arbitrary" em spacing. There must be a way to do a "search and replace" with the correct glyphs (U+202F and U+0020) before and after punctuation marks.

As my example file demonstrates, we clearly shouldn't use the glyphs from the font. Arthur

Not knowing the internals, that's what I tried to say with adding a space after instead of 1em, i.e. "calculated by the engine". If w an em is added after, it is not stretchable and shrinkable, right?

Right. But no particular space is added after '?' or '!' in French punctuation mode, TeX sets the standard inter-word glue, as it should. The only intervention by ConTeXt is the addition of the unbreakable space before those marks (and, obviously, after '«').

Thank you so much for the detailed technical explanation! So, AFAIK, I believe that the space before should be equivalent to thinspace.

You'd need to take this up with Olivier Guéry, who suggested the 1/4 em value in the first place (refer to his interventions from last Fall, e.g., http://www.ntg.nl/pipermail/ntg-context/2008/034845.html). He wrote up a small proposal on the wiki page for French Punctuation (http://wiki.contextgarden.net/French_Punctuation).

Thanks, I'll see that. Maybe I could write some detailled specs in the wiki.

Yes, please do so by extending the aforementioned page if needed. Arthur

luigi scarso wrote:

On Fri, May 22, 2009 at 5:51 PM, Arthur Reutenauer < arthur.reutenauer@normalesup.org> wrote:

...

...

but instead of arbitrary adding a 0.25em before and 1em after the punctuation mark you should use the real nnbsp (U+202F) before and real normal space (U+0020) after.

I don't think so. Space characters don't mix very well with TeX glue and should best be avoided, generally speaking. In particular, all inter-word spaces that are input in the TeX source as one or more of U+0020 are simply ignored, and replaced by normal inter-word glue, with its appropriate stretchability and shrinkability. This has always been the case in TeX and is not going to change. All other types of Unicode spaces should really, in my opinion, be processed in the same way, while respecting their additional properties in the case of non-breakable spaces, for instance.

Maybe one can use something like this \defineremapper[filterItem] \remapcharacter[filterItem][`•]{\item}

for "spaces" too

as such a remapper is applied on all input eventually you end up in a mess and we don't want a mess ... the current mechanism acts upon the to be typeset text which is way more safe ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Dohyun Kim wrote:

Considering current state that we don't know any fonts that has ITLC table, it would be better than nothing to implement italic correction as follows. In the following code, "fontdata" is a table returned by the function "fonts.define.read".

local param = fontdata.parameters local italicangle = fontdata.shared.otfdata.metadata.italicangle if italicangle and italicangle < 0 then local uwidth = fontdata.shared.otfdata.metadata.uwidth or 40 local factor = fontdata.factor or 655.36 param.slant = - math.tan(italicangle*math.pi/180) * param.quad for i,v in pairs(fontdata.characters) do local gl = fontdata.descriptions[i] local it = (gl.boundingbox[3] - gl.width + uwidth*0.5) * factor if it > 0 then v.italic = it end end end

there are seleveral solutions: - extend the font with this info (faster but then it's always there which might not be ok as it's an approximation) - calculate it after loading (which is what you propose) in the mkiv code we do have a hook for that kind of things so this is then what i propose. watch how we don't scale here, we just add an entry to the shared data as that's where we hook in; the real implementation would look slightly different as an optimization is possible \starttext \startluacode table.insert(fonts.triggers,"itlc") local function itlc(tfmdata,value) if value then -- the magic 40 and it formula come from Dohyun Kim local fontdata = tfmdata.shared.otfdata or tfmdata.shared.afmdata local metadata = fontdata and fontdata.metadata if metadata then local italicangle = metadata.italicangle if italicangle and italicangle ~= 0 then local uwidth = (metadata.uwidth or 40)/2 for unicode, d in next, tfmdata.descriptions do local it = d.boundingbox[3] - d.width + uwidth if it ~= 0 then d.italic = it end end end end end end fonts.initializers.base.otf.itlc = itlc fonts.initializers.node.otf.itlc = itlc fonts.initializers.base.afm.itlc = itlc fonts.initializers.node.afm.itlc = itlc \stopluacode \definedfont[SerifItalic*default at 24pt] test\/test \definefontfeature[xdefault][default][itlc=yes] \definedfont[SerifItalic*xdefault at 24pt] test\/test \stoptext i could add it to the generic code (although i'm not going to add all the other context goodies to the generic code definitely not as long as they're experimental) ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Khaled Hosny wrote:

On Wed, May 27, 2009 at 01:34:57PM +0900, Dohyun Kim wrote:

...

2009/5/22 Taco Hoekwater :

...

Hans Hagen wrote:

...

Taco Hoekwater wrote:

...

Khaled Hosny wrote:

...

Not very helpful in this situation, but FontForge has a non-standard italic correction (ITLC) table[1], may be TeX related OpenTyp font projects like Latin Modern and Gyre fonts can use it? That would perhaps not be a bad idea. If that table is there then luatex will automatically use it (it is a subtable of 'TeX ', which also contains height and depth information, and font dimensions). so, that data would end up in a regular feature/lookup? of is it an entry in the glyph? They are automatically merged into the glyph, as

glyph.italic_correction glyph.tex_height glyph.tex_depth

Hi,

Considering current state that we don't know any fonts that has ITLC table, it would be better than nothing to implement italic correction as follows.

Per FontForge's documentation, it can generate italic correction values, may be LuaTeX could make use of such feature and provide a way to generate italic correction values for fonts missing it, may be the tex height and depth too, if it isn't doing so already? Since FontForge has access to actual glyph shapes, it might be generating better guesses.

I can generate sample fonts with FontForge for testing, if needed.

as such an italic correction is not part of the font design but a guess it makes sense to keep it 'under lua control' so i implemented a itls=yes feature (base mode and node mode; is in beta) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------