Rendering of ZWNJ and ZWJ

Khaled Hosny

9 Jun 2008 9 Jun '08

3:42 a.m.

Using Luatex engine, ZWNJ and ZWJ chars are rendered incorrectly. If the font has a glyph for it, it does the expected behaviour (force non-joining or joining respectively) but the glyph get rendered while it shouldn't. But if the font has no glyphs for it, it has no effect at all. See the attached example. -- Khaled Hosny Arabic localizer and member of Arabeyes.org team

Attachments:

zwnj_ScheherazadeRegOT.pdf (application/pdf — 9.1 KB)
zwnj_ae_AlMohanad.pdf (application/pdf — 5.8 KB)
zwnj.tex (text/x-tex — 449 bytes)
signature.asc (application/pgp-signature — 189 bytes)

Show replies by date

Hans Hagen

9 Jun 9 Jun

3:04 p.m.

Khaled Hosny wrote:

...

Using Luatex engine, ZWNJ and ZWJ chars are rendered incorrectly. If the font has a glyph for it, it does the expected behaviour (force non-joining or joining respectively) but the glyph get rendered while it shouldn't. But if the font has no glyphs for it, it has no effect at all. See the attached example.

ok, the case when it's not in the font is easy to fix by a slightly different test (aroun dline 4500 in font-otf.lua) local chardata = characters[char] if chardata and chardata.description.class == "mark" then set_attribute(current,state,5) -- mark if trace then fcs(current,"font:mark") end elseif isol[char] then -- can be zwj or zwnj too ..... you need to remake the format the problem of the glyph showing up ... i have to think of an option (because i guess that sometimes it's handy for tracing) ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Hans Hagen

5:45 p.m.

Khaled Hosny wrote:

...

Using Luatex engine, ZWNJ and ZWJ chars are rendered incorrectly. If the font has a glyph for it, it does the expected behaviour (force non-joining or joining respectively) but the glyph get rendered while it shouldn't. But if the font has no glyphs for it, it has no effect at all. See the attached example.

i uploaded a beta after analyzing i now remove the joiners ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Khaled Hosny

7:05 p.m.

On Mon, Jun 09, 2008 at 05:45:10PM +0200, Hans Hagen wrote:

...

Khaled Hosny wrote:

...
Using Luatex engine, ZWNJ and ZWJ chars are rendered incorrectly. If the font has a glyph for it, it does the expected behaviour (force non-joining or joining respectively) but the glyph get rendered while it shouldn't. But if the font has no glyphs for it, it has no effect at all. See the attached example.

i uploaded a beta

after analyzing i now remove the joiners

Thanks for the quick response, I'll try it as soon as I can. Regards, Khaled

...

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

-- Khaled Hosny Arabic localizer and member of Arabeyes.org team

Idris Samawi Hamid

10 Jun 10 Jun

2:32 a.m.

Hi Hans, On Mon, 09 Jun 2008 09:45:10 -0600, Hans Hagen wrote:

...

Khaled Hosny wrote:

...
Using Luatex engine, ZWNJ and ZWJ chars are rendered incorrectly. If the font has a glyph for it, it does the expected behaviour (force non-joining or joining respectively) but the glyph get rendered while it shouldn't. But if the font has no glyphs for it, it has no effect at all. See the attached example.

i uploaded a beta

after analyzing i now remove the joiners

Not sure if this is the best approach: Despite what MS Typography says, I think it's important to maintain the joiner info after analysis. If I select text from a pdf, the joiner info should be there. This is important for, eg, switching from Arabic-language to Farsi-language typography. So If I select some pdf text formatted according to Farsi rules, I should be able to save that text with the rules included. Can this be made an option, so fonts that contain zwj zwnj are not affected? Best wishes Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523

Hans Hagen

6:45 p.m.

Idris Samawi Hamid wrote:

...

Can this be made an option, so fonts that contain zwj zwnj are not affected?

ok, the next version will have otf.remove_joiners = true -- default later we can make it an option in the font definition ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Idris Samawi Hamid

8:12 p.m.

On Tue, 10 Jun 2008 10:45:31 -0600, Hans Hagen wrote:

...

Idris Samawi Hamid wrote:

...
Can this be made an option, so fonts that contain zwj zwnj are not affected?

ok, the next version will have

otf.remove_joiners = true -- default

Our main test fonts + professional fonts generally have zwj zwnj so actually the default should be otf.remove_joiners = false at least until we can

...

make it an option in the font definition

Otherwise I have to keep modifying the file everytime I download a beta... Best wishes Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523

Hans Hagen

8:52 p.m.

Idris Samawi Hamid wrote:

...

On Tue, 10 Jun 2008 10:45:31 -0600, Hans Hagen wrote:

...
Idris Samawi Hamid wrote:

...
Can this be made an option, so fonts that contain zwj zwnj are not affected? ok, the next version will have

otf.remove_joiners = true -- default

Our main test fonts + professional fonts generally have zwj zwnj so actually the default should be

sure, but i need to check in what way they interfere in the lookups later on

...

otf.remove_joiners = false

at least until we can

...
make it an option in the font definition

Otherwise I have to keep modifying the file everytime I download a beta...

you can put \ctxlua{otf.remove_joiners = false} on cont-loc.mkiv or so ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Khaled Hosny

12 Jun 12 Jun

3:55 p.m.

On Mon, Jun 09, 2008 at 06:32:13PM -0600, Idris Samawi Hamid wrote:

...

Hi Hans,

On Mon, 09 Jun 2008 09:45:10 -0600, Hans Hagen wrote:

...
Khaled Hosny wrote:

...
Using Luatex engine, ZWNJ and ZWJ chars are rendered incorrectly. If the font has a glyph for it, it does the expected behaviour (force non-joining or joining respectively) but the glyph get rendered while it shouldn't. But if the font has no glyphs for it, it has no effect at all. See the attached example.

i uploaded a beta

after analyzing i now remove the joiners

Not sure if this is the best approach:

Despite what MS Typography says, I think it's important to maintain the joiner info after analysis. If I select text from a pdf, the joiner info should be there. This is important for, eg, switching from Arabic-language to Farsi-language typography. So If I select some pdf text formatted according to Farsi rules, I should be able to save that text with the rules included.

Can this be made an option, so fonts that contain zwj zwnj are not affected?

Well, I've very little ConTeXt knowledge to actually understand this. My main point was that non-printable formatting characters like ZWNJ, ZWJ should be font independent i.e. when I insert a ZWNJ it inhibits the joining of Arabic letters without being printed, the font having or not having a glyph for it shouldn't affect this. In the other hand, it should be their so that when I copy the text I get it (since it is part of the input and has semantical value). Is this what we have here? Regards, Khaled -- Khaled Hosny Arabic localizer and member of Arabeyes.org team

Hans Hagen

4:06 p.m.

Khaled Hosny wrote:

...

On Mon, Jun 09, 2008 at 06:32:13PM -0600, Idris Samawi Hamid wrote:

...
Hi Hans,

On Mon, 09 Jun 2008 09:45:10 -0600, Hans Hagen wrote:

...
Khaled Hosny wrote:

...
Using Luatex engine, ZWNJ and ZWJ chars are rendered incorrectly. If the font has a glyph for it, it does the expected behaviour (force non-joining or joining respectively) but the glyph get rendered while it shouldn't. But if the font has no glyphs for it, it has no effect at all. See the attached example. i uploaded a beta

after analyzing i now remove the joiners Not sure if this is the best approach:

Despite what MS Typography says, I think it's important to maintain the joiner info after analysis. If I select text from a pdf, the joiner info should be there. This is important for, eg, switching from Arabic-language to Farsi-language typography. So If I select some pdf text formatted according to Farsi rules, I should be able to save that text with the rules included.

Can this be made an option, so fonts that contain zwj zwnj are not affected?

Well, I've very little ConTeXt knowledge to actually understand this. My main point was that non-printable formatting characters like ZWNJ, ZWJ should be font independent i.e. when I insert a ZWNJ it inhibits the joining of Arabic letters without being printed, the font having or not having a glyph for it shouldn't affect this. In the other hand, it should be their so that when I copy the text I get it (since it is part of the input and has semantical value).

Is this what we have here?

i made them font independent in the sense that they need not be present in the font; however, the rendering is optional ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

6037

Age (days ago)

6040

Last active (days ago)

List overview

Download

9 comments

3 participants

participants (3)

Hans Hagen
Idris Samawi Hamid
Khaled Hosny

Rendering of ZWNJ and ZWJ

tags

participants (3)