On Fri, Feb 20, 2009 at 12:31:49PM -0700, Idris Samawi Hamid ادريس سماوي حامد wrote:

On Fri, 20 Feb 2009 11:57:22 -0700, Hans Hagen wrote:

...

Khaled Hosny wrote:

...

Currently, when defining a font feature one has to enable all features by hand which is IMHO not very user friendly as it implies prior knowledge about OpenType font features and the meaning of each one, not every Arabic user, for example, knows what does 'init', 'medi, etc. ligatures mean yet to know that he must enable them to get proper font rendering. I think some font features should be on by default, so that \definefontfeature[script=arabic] should be enough to get an Arabic font rendered correctly with the default features as its designer intended (designers assume that certain will be on while other are off by default, like liga vs. dlig), and if some one wants to disable a certain default feature he can turn it off, not the reverse.

Hmm, not sure if this is a good idea, see below.

...

...

Microsoft's OpenType features list page (http://www.microsoft.com/typography/otspec/features_ae.htm) gives a "UI suggestion" for each feature noting if it should be on by default, I think those are what most OpenType enable by default (at least the ones I tested).

...

i've been thinking of a features=default option (as there is already features=yes|no)

Maybe we can have a features=ms_arabic

instead of defaulting to MS' recommendation. So in the final high-end user interface we can have keys like

featureset=ms_arabic

I think the 'ms' part is't really needed, we can just call it"arabic", or are we going to have more Arabic feature sets?

But I agree with Hans that this is a matter that needs more thought. For example, Traditional Arabic mixes OpenType and older M$ specs in Uniscribe, so just plugging in the default features that M$ suggests is not sufficient for, eg, vowel function in Tr Ar.

That is beyond OpenType support, since Traditional Arabic is essentially broken at many levels, even when Uniscribe is used (vowel marks break lam-alef ligatures for example), Ms Arabic fonts are special case since they were developed long ago before OpenType and aren't the best examples.

Also, what about Arabic fonts on the mac, so they follow the same specs as M$?

AFAIK, Microsoft's Uniscribe is the reference OpenType implementation, and I assume that Apple's OpenType implementation follows its recommendations (I can't test that).

We've still got lots to do before settling on a very high-end interface so there is time to think about this some more...

Of course, we aren't in hurry. Regards, Khaled -- Khaled Hosny Arabic localizer and member of Arabeyes.org team

Idris Samawi Hamid ادريس سماوي حامد wrote:

Khaled: Have you considered making an inventory of available Arabic fonts? Or does one already exist? We could put together a table of what's available, checking for available OT features, etc.

also, it would be nice to have some quality otf arabic fonts in tex live Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Why not just include the SIL arabic fonts? The OFL is fine for a license. TL already has several other OFL'd fonts. If someone puts the fonts on CTAN, I don't mind incorporating them in TL. (As a practical matter, it is a lot of extra work for me to work with any package that is not on CTAN. And besides, we want CTAN to be "comprehensive".) It would also be nice to have supporting TeXish files, but if the bare otf's/ttf's/whatever SIL gives are useful with LuaTeX/ConTeXt, I don't insist.

Khaled Hosny wrote:

I think we can make default features per script (HarfBuzz seems to do that).

...

if it was trivial i'd already done it -)

I know :) just my two cents.

how about making a wiki page where we collect fontnames + assumed default features (+ test if possible); the problem is that fonts themselves don't have that info it's no problem to have a database in mkiv that uses this info so that at some point we can default to a feature set based on font name Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

In fact there are some Persian/Arabic fonts in TL08 and there is no need to include them. They are a part of xepersian package. I think they should be at fonts/truetype/xepersian These fonts include SIL's font in addition to farsiweb fonts. Looking at CTAN, it seems that the author of xepersian is not going to include these fonts in his next release.

Hi Hans, Khaled, and Idris, If a humble opinion from an ordinary user may be issued, I agree with Khaled that it would be extremely useful to have some basic default settings for Arabic fonts, and even more generally for any particular fonts used for other languages. This would not prevent those specialist typesetters who want particular features to be turned on, to do so through appropriate mechanisms. As a basic user I am frustrated when using mkiv, that most declaration of features are completely cryptic, and not being a specialist of OTF or other font specifications, I don't know which features are essential for writing and typesetting an article in Persian or any language using Arabic alphabet. While the following is quite easy to understand and to use in XeConTeXt %%%%%%%%% %!TEX TS-program = xecontext \TeXXeTstate=1 % defining a font for Arabic, Persian \font\Faarsi="Scheherazade:script=arab" at 14pt \Faarsi % defining a font for Roman languages \font\romfont="Times Roman" at 12pt \def\rom#1{{\beginL\romfont #1\endL}} \everypar={\setbox0=\lastbox \beginR \box0 } \starttext \rom{goedemorgen Hans} سلام خالد، درود بر ادریس \stoptext %%%%%%%%%%% it is not so easy to figure out how to typeset the same thing in mkiv: for instance the non specialist has to try several fonts which are installed on his system before getting a result… %%%%%%%%%%% %!TEX TS-program = mkiv \setupdirections[bidi=global] % defining a font for Arabic, Persian \definefontfeature [CrypticFeatures] [analyze=yes, mode=node, language=dflt, script=arab, aalt=yes, init=yes, medi=yes, fina=yes, isol=yes, liga=yes, mset=yes] %\font\Faarsi=Scheherazade*CrypticFeatures at 14pt % this does not work % after trying a few other fonts one finds that the following works: \font\Faarsi=arabtype*CrypticFeatures at 14pt % defining a font for Roman languages \setupbodyfont[12pt] \def\rom#1{{\start \textdir TLT #1 \stop}} \pardir TRT \textdir TRT \starttext \rom{goedemorgen Hans} \start \Faarsi سلام خالد، درود بر ادریس \stop \stoptext %%%%%%%%%%%% The ideal situation would be: • in a document, when one sets a language then a certain font, with certain standard features is set by default; • when an adapted font for that language is defined by the user, then certain features are set by default; • possibly a command like \setupArabic[state=start] [...=...,features=...] (and the analogous settings for other languages such as \setupHanzi[state=start][...=...,features=...], and alike), could be imagined; • for instance, as we have now in ConTeXt, when one has a document written in English, then without even defining a font one can write \starttext goedmorgen Hans! This is some maths formula: $a^2+b^2 = c^2$. \stoptext Therefore, in the same spirit one should have some default allowing the user to write \setupArabic[state=start] \setupHanzi[state=start] \starttext goedmorgen Hans! This is some maths formula: $a^2+b^2 = c^2$. And this is some Arabic text written and typeset from right to left, in the midst of some English text: \startArabic سلام خالد، درود بر ادریس \stopArabic and analogously this is in Hanzi \startHanzi 大家你好 stopHanzi \stoptext I understand that this can be done by each user upon defining his own environment and installing fonts, etc. But for the non specialist it is not that easy to understand the intricacies. With my best regards: OK On 20 févr. 09, at 19:57, Hans Hagen wrote:

Khaled Hosny wrote:

...

Currently, when defining a font feature one has to enable all features by hand which is IMHO not very user friendly as it implies prior knowledge about OpenType font features and the meaning of each one, not every Arabic user, for example, knows what does 'init', 'medi, etc. ligatures mean yet to know that he must enable them to get proper font rendering. I think some font features should be on by default, so that \definefontfeature[script=arabic] should be enough to get an Arabic font rendered correctly with the default features as its designer intended (designers assume that certain will be on while other are off by default, like liga vs. dlig), and if some one wants to disable a certain default feature he can turn it off, not the reverse. Microsoft's OpenType features list page (http://www.microsoft.com/typography/otspec/features_ae.htm) gives a "UI suggestion" for each feature noting if it should be on by default, I think those are what most OpenType enable by default (at least the ones I tested).

i've been thinking of a features=default option (as there is already features=yes|no)

even then it can never be fully automatic as some usage of fonts (think of verbatim) demands devation from defaults

now, if we implement a default list then we first need to make a detailed list of what the supposed defaults are (and i'm not sure if ms is the only resource for that; after all, not all machineries support all features)

a related issue is that fonts can be used for different languages and scripts and therefore a more dynamic feature switching might be needed i.e. arabic might need init, but when the same font is used for latin it not handy to have it enabled, so there might be a matrix of features / scripts needed

if it was trivial i'd already done it -)

(implementing is trivial but i don't want to make the wrong decision here as it will influence compatibility)

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

On Sun, 22 Feb 2009 14:03:56 -0700, Khaled Hosny wrote:

I totally agree with you, more ever I hop that at some point in the future, just selecting certain language will be enough to get proper display of it, some thing like:

\mainlanguage[arabic] \startext أهلا بالعالم! \stoptext

Fonts will always be complicated... for what you are describing we need to start with a default Arabic-script font for TeX, like LM for Latin. My suggestion is to make Scheherazade that font, since it is free, distributable, and supports nearly all Arabic script languages. We could make two settings: one for Scheherazade as the LM fallback for the needed Ar-script ranges for unicode, another for Scheherazade as the main font, with Termes as Latin fallback. That will give us a benchmark from which to proceed with other fonts. If I get time I'll try to build typescripts etc for this scenario, or Khaled or someone else can take on this task, and I'll review it. Even Scheherazade has lots of options, and it will take some research to get the "ideal" default even here. Best wishes Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523

Khaled Hosny wrote:

I'm not very fond if Scheherazade as it has many wrong glyphs (most of the Quranic glyphs are completely wrong) and it doesn't even have an الله ligature, which most Arabic users won't accept. I was experimenting with a modified version that fixes those issues, but I gave up since the font lacks any contextual forms but the basic four ones and extending the font the way it is designed proved to be cumbersome and very error prone.

so, in addition to collecting 'default features per font' we also need to collect info about quality and missing or broken things Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On Mon, Feb 23, 2009 at 08:34:54AM -0700, Idris Samawi Hamid ادريس سماوي حامد wrote:

On Mon, 23 Feb 2009 05:02:39 -0700, Khaled Hosny wrote:

...

I'm not very fond if Scheherazade as it has many wrong glyphs (most of the Quranic glyphs are completely wrong) and it doesn't even have an الله ligature, which most Arabic users won't accept.

There is a hadith

لا يسقط الميسور بالمعسور

Scheherazade is certainly not perfect -- and I doubt I'll ever use it personally -- but it's the best freely available option we have. If I understand the license correctly, we can rename it and turn it into a project. You, Otared, and others are expressing the need for a standard, default interface, so let's start with this and build on it. If there is another pure opentype font that is better, let me know and I'll test it.

The key point is that SIL has taken care to support nearly every Arabic-script language, not just Arabic, so this gives us a platform that all Arabic-script users can use by default until other free fonts -- or a fork of Scheherazade -- become available.

I already have a fork, I just don't want to encourage people to use such aesthetically poor fonts, but as it seems to be the only solution at hand right now (especially the Unicode covering), I think I'll try to clean my font and publish it this week ISA.

BTW: There is an الله lig in Scheherazade -- rather unattractive, but it's there. Maybe you have an old version?

Sorry, yes it has one but is no better than not having one at all.

...

I was experimenting with a modified version that fixes those issues, but I gave up since the font lacks any contextual forms but the basic four ones and extending the font the way it is designed proved to be cumbersome and very error prone.

I'm not sure if the VOLT sources are available, although I used a font-to-volt script that seems to have captured nearly everything. I guess you use FF but no matter: Just make a list of problems and the minimal -- as opposed to ideal -- set of recommended changes and one or both of us can work on this over the next few months.

FF does a great job here, it does read all lookups that I don't need VOLT sources so much.

...

If it is needed, I can clean the font and provide it to be included in the minimal distribution temporarily, as I hope that we'll have a viable alternative by the end of this year.

See the above... make a list of errors in the font and we'll both work on it one way or other. Back to Lotus: I suggest to you and Otared the following project: develop Scheherazade into a free alternative to Lotus. Lotus is that is as close to a pure OpenType font as we are going to easily get. Although it supports less of Arabic-script unicode than Scheherazade, it is a standard in the Arabic-script publishing world. So if our default interface supports Lotus and Scheherazade out of the box, we will be well on our way to what you guys are looking for.

My initial idea was to extend Scheherazade in a similar way, but I gave up on this, there are so many similar glyphs in the Arabic Unicode block that a simple ligature like بي means 28 (Baa' forms) * 10 (Yaa' forms): 280 glyphs for one ligatures, I don't think this is a wise idea. Regards, Khaled -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

On Mon, Feb 23, 2009 at 10:56:07AM -0700, Idris Samawi Hamid wrote:

So just start with Arabic, then Farsi, and forget the rest for now. No need for perfection at the outset ;-)

Actually, I suggest forgetting about ligatures etc. for now, and just get the basics done for Scheherazade. We can work on ligatures etc later.

OK, here is an initial version. The glyphs are scaled up since Scheherazade is tiny, fixed الله ligature, added لله and فلله ligatures as well, fixed the swash kaf and the Sindi meem and a bunch of Quranic marks as well: http://www.khaledhosny.org/files/simplenaskhi.zip Regards, Khaled -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

On Mon, Feb 23, 2009 at 04:19:13PM -0700, Idris Samawi Hamid ادريس سماوي حامد wrote:

...

OK, here is an initial version. The glyphs are scaled up since Scheherazade is tiny, fixed الله ligature, added لله and فلله ligatures as well, fixed the swash kaf and the Sindi meem and a bunch of Quranic marks as well: http://www.khaledhosny.org/files/simplenaskhi.zip

Ok, thnx, Khaled. I looked at it: The الله is infinitely better, but does not quite match the rest -- the shaddah is way too big for it (or maybe the الله lig is too short). On closer inspection, I see that this is not the full الله lig, but a mix of the no-vowel version with the default shaddah. The actual FDF2 looks better. So there is still some OT work that needs to be done.

I also have some poetry macros that might help, need to port them to the new system...

That'd be great!

Not too bad a font. Like Lotus, it works best at smaller text sizes, not greater than 12 point.

The فلله lig should be off by default, there are other Arabic word combinations that use that combo that have nothing to do with الله.

I can't think in any other word on top of my head, but I'm fine with making it a 'dlig' instead, it isn't very common any way.

Next steps:

1. Can I add some fixes to this thing as well or would you like to keep 100% maintainance?

2. I can try to put in some time in the coming weeks to develop this into a default setup for ConTeXt-mkiv -- per recent discussions -- or I can leave that to you/Otared and just review the results. Just let me know how you would like to proceed.

Of course you are welcomed, actually I won't have much free time till the end of march, it'd be nice if you can work on it. Regards, Khaled -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

Khaled Hosny wrote:

I totally agree with you, more ever I hop that at some point in the future, just selecting certain language will be enough to get proper display of it, some thing like:

\mainlanguage[arabic] \startext أهلا بالعالم! \stoptext

to some extend that can already be done with dynamic features but it needs more research etc

\setupbodyfont[arabic=foo,latin=bar]

for that we have the fallbacks mechanism; of course one can mix any font but even then it always demands tuning, i.e. foo and bar can look completely weird when combined without relative scaling (which is what the fallbacks do: thereone chooses one main font and overlays other fonts properly scaled) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On Feb 23, 2009, at 1:09 PM, Khaled Hosny wrote:

For scaling, we can specify a size, some thing like:

\setupbodyfont[arabic=foo,20pt,latin=bar,18pt]

or so.

I don't know Arabic, but I've done similar things for Greek, and that is not a good interface. You want a scaling factor, so users won't have to worry about scaling in footnotes, titles, etc., an absolute size is not a viable alternative. Thomas

On Mon, Feb 23, 2009 at 01:28:23PM +0100, Hans Hagen wrote:

Thomas A. Schmitz wrote:

...

On Feb 23, 2009, at 1:09 PM, Khaled Hosny wrote:

...

For scaling, we can specify a size, some thing like:

\setupbodyfont[arabic=foo,20pt,latin=bar,18pt]

or so.

I don't know Arabic, but I've done similar things for Greek, and that is not a good interface. You want a scaling factor, so users won't have to worry about scaling in footnotes, titles, etc., an absolute size is not a viable alternative.

indeed.

btw, i have no problem with a bunch of predefined combinations (using the fallbacks mechanism) so that users can quickly initialize a typeface

\usetypescript[fancyarabicwithgreekandtraditionallatin] \setupbodyfont[fancyarabicwithgreekandtraditionallatin,13pt]

or so, collected in some type-* file, but that's as far as we can go; we need to guard at least some minimal quality

a quick start but with bad output is not the way to go

I was about to forget why I didn't like font fallbacks in the first place, the current font fallback mechanism assigns fonts per Unicode characters, this is fine until we come to common characters like numbers or brackets: you can only assign it to one font, which isn't usually desirable. Think of this sentence: عربي 1234 عربي (English 1234 English (English) English) عربي. Here, the outer most parentheses and first numbers should use the same font of the Arabic text, while the inner ones should use the font of the English text, font fallback can't do this. Instead we need to segment the text per script and apply fonts on whole segments, the Unicode Script Property annex describes a way to handle this, see http://www.unicode.org/reports/tr24/#Script_Names_in_Rendering. Regards, Khaled -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

Taco Hoekwater wrote:

I believe you should write

...

عربي 1234 عربي ({\language[en] English 1234 English (English) English}) عربي.

then the font switch can be made to do something sensible with the font

I meant the language switch of course, sorry.

Khaled Hosny wrote:

On Mon, Feb 23, 2009 at 01:28:23PM +0100, Hans Hagen wrote:

...

Thomas A. Schmitz wrote:

...

On Feb 23, 2009, at 1:09 PM, Khaled Hosny wrote:

...

For scaling, we can specify a size, some thing like:

\setupbodyfont[arabic=foo,20pt,latin=bar,18pt]

or so. I don't know Arabic, but I've done similar things for Greek, and that is not a good interface. You want a scaling factor, so users won't have to worry about scaling in footnotes, titles, etc., an absolute size is not a viable alternative. indeed.

btw, i have no problem with a bunch of predefined combinations (using the fallbacks mechanism) so that users can quickly initialize a typeface

\usetypescript[fancyarabicwithgreekandtraditionallatin] \setupbodyfont[fancyarabicwithgreekandtraditionallatin,13pt]

or so, collected in some type-* file, but that's as far as we can go; we need to guard at least some minimal quality

a quick start but with bad output is not the way to go

I was about to forget why I didn't like font fallbacks in the first place, the current font fallback mechanism assigns fonts per Unicode characters, this is fine until we come to common characters like numbers or brackets: you can only assign it to one font, which isn't usually desirable. Think of this sentence:

عربي 1234 عربي (English 1234 English (English) English) عربي.

Here, the outer most parentheses and first numbers should use the same font of the Arabic text, while the inner ones should use the font of the English text, font fallback can't do this. Instead we need to segment the text per script and apply fonts on whole segments, the Unicode Script Property annex describes a way to handle this, see http://www.unicode.org/reports/tr24/#Script_Names_in_Rendering.

Sure but that can be done by using explicit switches to another 'environment' i.e. explicitly marking sections I'm not that sure if i want to add some fuzzy automatism which then needs to handle all kind of exceptions too proper markup is a good solution Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Idris Samawi Hamid wrote:

On Mon, 23 Feb 2009 16:23:59 -0700, Hans Hagen wrote:

...

...

...

a quick start but with bad output is not the way to go I was about to forget why I didn't like font fallbacks in the first place, the current font fallback mechanism assigns fonts per Unicode characters, this is fine until we come to common characters like numbers or brackets: you can only assign it to one font, which isn't usually desirable. Think of this sentence: عربي 1234 عربي (English 1234 English (English) English) عربي. Here, the outer most parentheses and first numbers should use the same font of the Arabic text, while the inner ones should use the font of the English text, font fallback can't do this. Instead we need to segment the text per script and apply fonts on whole segments, the Unicode Script Property annex describes a way to handle this, see http://www.unicode.org/reports/tr24/#Script_Names_in_Rendering.

Sure but that can be done by using explicit switches to another 'environment' i.e. explicitly marking sections

I'm not that sure if i want to add some fuzzy automatism which then needs to handle all kind of exceptions too

proper markup is a good solution

Not sure I understand you, but could we do something like the following: Along the lines of \quote, \quotation:

Latin text\paren{here is some arabic text\paren[somefallback]{here is some latin} more arabic}.

so the fallback will be used when explicitly called. Put another way, could we add a conditional parameter to the fallback macro so that those chars are used only in situations when they are specifically called?

\paren[fallback=...] \bracket[fallback=...] \brace[fallback=...] \anglebracket[fallback=...]

sure, it makes sense to add that kind of structure ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Hi Otared, On Sat, 21 Feb 2009 06:05:01 -0700, Otared Kavian wrote:

If a humble opinion from an ordinary user may be issued, I agree with Khaled that it would be extremely useful to have some basic default settings for Arabic fonts, and even more generally for any particular fonts used for other languages. This would not prevent those specialist typesetters who want particular features to be turned on, to do so through appropriate mechanisms.

sure, but it needs some thought. As I mentioned in my latest reply to Khaled, we can start with Scheherazade as an initial benchmark and proceed from there.

As a basic user I am frustrated when using mkiv, that most declaration of features are completely cryptic, and not being a specialist of OTF or other font specifications, I don't know which features are essential for writing and typesetting an article in Persian or any language using Arabic alphabet.

The difference -- I think -- is that xetex uses the system libraries. Uniscribe is not configurable, so you're stuck with however uniscribe interprets the ot features. <snip>

%\font\Faarsi=Scheherazade*CrypticFeatures at 14pt % this does not work

Hmm, I wonder why...

The ideal situation would be: • in a document, when one sets a language then a certain font, with certain standard features is set by default;

sure, I suggest Scheherazade

• when an adapted font for that language is defined by the user, then certain features are set by default;

that needs some research, for reasons Hans has explained.

• possibly a command like \setupArabic[state=start] [...=...,features=...] (and the analogous settings for other languages such as \setupHanzi[state=start][...=...,features=...], and alike), could be imagined;

sure

Therefore, in the same spirit one should have some default allowing the user to write \setupArabic[state=start] \setupHanzi[state=start] \starttext goedmorgen Hans! This is some maths formula: $a^2+b^2 = c^2$. And this is some Arabic text written and typeset from right to left, in the midst of some English text: \startArabic سلام خالد، درود بر ادریس \stopArabic

I understand that this can be done by each user upon defining his own environment and installing fonts, etc. But for the non specialist it is not that easy to understand the intricacies.

For now the focus is getting OpenType and the mid-level interface solid, then we can focus on the high-level interface as you have described. OTOH, Getting Scheherazade (or some other mutually agreed-upon font) set up as default with a high-level interface should not be that hard. سلام Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523

Hi Hans, Khaled, and Idris, If a humble opinion from an ordinary user may be issued, I agree with Khaled that it would be extremely useful to have some basic default settings for Arabic fonts, and even more generally for any particular fonts used for other languages. This would not prevent those specialist typesetters who want particular features to be turned on, to do so through appropriate mechanisms. As a basic user I am frustrated when using mkiv, that most declaration of features are completely cryptic, and not being a specialist of OTF or other font specifications, I don't know which features are essential for writing and typesetting an article in Persian or any language using Arabic alphabet. While the following is quite easy to understand and to use in XeConTeXt %%%%%%%%% %!TEX TS-program = xecontext \TeXXeTstate=1 % defining a font for Arabic, Persian \font\Faarsi="Scheherazade:script=arab" at 14pt \Faarsi % defining a font for Roman languages \font\romfont="Times Roman" at 12pt \def\rom#1{{\beginL\romfont #1\endL}} \everypar={\setbox0=\lastbox \beginR \box0 } \starttext \rom{goedemorgen Hans} سلام خالد، درود بر ادریس \stoptext %%%%%%%%%%% it is not so easy to figure out how to typeset the same thing in mkiv: for instance the non specialist has to try several fonts which are installed on his system before getting a result… %%%%%%%%%%% %!TEX TS-program = mkiv \setupdirections[bidi=global] % defining a font for Arabic, Persian \definefontfeature [CrypticFeatures] [analyze=yes, mode=node, language=dflt, script=arab, aalt=yes, init=yes, medi=yes, fina=yes, isol=yes, liga=yes, mset=yes] %\font\Faarsi=Scheherazade*CrypticFeatures at 14pt % this does not work % after trying a few other fonts one finds that the following works: \font\Faarsi=arabtype*CrypticFeatures at 14pt % defining a font for Roman languages \setupbodyfont[12pt] \def\rom#1{{\start \textdir TLT #1 \stop}} \pardir TRT \textdir TRT \starttext \rom{goedemorgen Hans} \start \Faarsi سلام خالد، درود بر ادریس \stop \stoptext %%%%%%%%%%%% The ideal situation would be: • in a document, when one sets a language then a certain font, with certain standard features is set by default; • when an adapted font for that language is defined by the user, then certain features are set by default; • possibly a command like \setupArabic[state=start] [...=...,features=...] (and the analogous settings for other languages such as \setupHanzi[state=start][...=...,features=...], and alike), could be imagined; • for instance, as we have now in ConTeXt, when one has a document written in English, then without even defining a font one can write \starttext goedmorgen Hans! This is some maths formula: $a^2+b^2 = c^2$. \stoptext Therefore, in the same spirit one should have some default allowing the user to write \setupArabic[state=start] \setupHanzi[state=start] \starttext goedmorgen Hans! This is some maths formula: $a^2+b^2 = c^2$. And this is some Arabic text written and typeset from right to left, in the midst of some English text: \startArabic سلام خالد، درود بر ادریس \stopArabic and analogously this is in Hanzi \startHanzi 大家你好 stopHanzi \stoptext I understand that this can be done by each user upon defining his own environment and installing fonts, etc. But for the non specialist it is not that easy to understand the intricacies. With my best regards: OK On 20 févr. 09, at 19:57, Hans Hagen wrote:

Khaled Hosny wrote:

...

Currently, when defining a font feature one has to enable all features by hand which is IMHO not very user friendly as it implies prior knowledge about OpenType font features and the meaning of each one, not every Arabic user, for example, knows what does 'init', 'medi, etc. ligatures mean yet to know that he must enable them to get proper font rendering. I think some font features should be on by default, so that \definefontfeature[script=arabic] should be enough to get an Arabic font rendered correctly with the default features as its designer intended (designers assume that certain will be on while other are off by default, like liga vs. dlig), and if some one wants to disable a certain default feature he can turn it off, not the reverse. Microsoft's OpenType features list page (http://www.microsoft.com/typography/otspec/features_ae.htm) gives a "UI suggestion" for each feature noting if it should be on by default, I think those are what most OpenType enable by default (at least the ones I tested).

i've been thinking of a features=default option (as there is already features=yes|no)

even then it can never be fully automatic as some usage of fonts (think of verbatim) demands devation from defaults

now, if we implement a default list then we first need to make a detailed list of what the supposed defaults are (and i'm not sure if ms is the only resource for that; after all, not all machineries support all features)

a related issue is that fonts can be used for different languages and scripts and therefore a more dynamic feature switching might be needed i.e. arabic might need init, but when the same font is used for latin it not handy to have it enabled, so there might be a matrix of features / scripts needed

if it was trivial i'd already done it -)

(implementing is trivial but i don't want to make the wrong decision here as it will influence compatibility)

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

On Mon, Feb 23, 2009 at 11:42:41AM +0100, Hans Hagen wrote:

Otared Kavian wrote:

...

Hi Hans, Khaled, and Idris,

If a humble opinion from an ordinary user may be issued, I agree with Khaled that it would be extremely useful to have some basic default settings for Arabic fonts, and even more generally for any particular fonts used for other languages. This would not prevent those specialist typesetters who want particular features to be turned on, to do so through appropriate mechanisms.

As a basic user I am frustrated when using mkiv, that most declaration of features are completely cryptic, and not being a specialist of OTF or other font specifications, I don't know which features are essential for writing and typesetting an article in Persian or any language using Arabic alphabet. While the following is quite easy to understand and to use in XeConTeXt %%%%%%%%%

well, it might be easy to understand given that default features are enabled but on the other hand,. when you want to disable a feature you need to know what is enabled so in practice you have the same problem

Yes, but the majority if people want need that, if I want to play with OpenType features then I'm supposed to know what I'm doing, while most people will be happy with the default features (given it is a reasonable default of course.) [...]

so, it seems to depend on calt but turning on calt by default is *not* what the ms spec recommends for arabtype

you see the problem?

I think is is arabtype's problem if they suggest turning calt off by default, if I've contextual alternatives in my font this means I think those alternatives are necessary to render text correctly, otherwise I would have used a stylistic set. Regards, Khaled -- Khaled Hosny Arabic localizer and member of Arabeyes.org team

Khaled Hosny wrote:

I think is is arabtype's problem if they suggest turning calt off by default, if I've contextual alternatives in my font this means I think those alternatives are necessary to render text correctly, otherwise I would have used a stylistic set.

it's not arabtype that suggests it, thats the problem; open type fonts *don't* have any information about what features should be on by default; it's the renderers that kind of decide it and there is no standard, so any renderer will follow its own rules also, some features (like frac) are often implemented only partial and as such can only be applied very selectively so, the only way out is something: defaults{'arabtype'] = { arab = { dflt = { calt=true, mkmk=true, ...}, }, } a kind of database Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On Fri, Feb 20, 2009 at 07:58:11PM +0100, Hans Hagen wrote:

Khaled Hosny wrote:

...

I think some font features should be on by default, so that \definefontfeature[script=arabic] should be enough to get an Arabic font

btw, this is why we already have arabic as featureset defined

I wasn't aware of that, I think having a set of predefined features would be perfectly OK, just if one can reuse it: some thing like \definefontfeature[include=arabic], may be some thing like that is already implemented? Regards, Khaled -- Khaled Hosny Arabic localizer and member of Arabeyes.org team

On Sat, Feb 21, 2009 at 12:41:48PM +0100, Hans Hagen wrote:

Idris Samawi Hamid ادريس سماوي حامد wrote:

...

On Fri, 20 Feb 2009 11:39:23 -0700, Khaled Hosny wrote:

...

Microsoft's OpenType features list page (http://www.microsoft.com/typography/otspec/features_ae.htm) gives a "UI suggestion" for each feature noting if it should be on by default,

Hmm, I think this page is more relevant for what you have in mine:

http://www.microsoft.com/typography/otfntdev/arabicot/features.htm

:-)

watch the "The standard order for applying Arabic features encoded in OpenType fonts" ... an earlier mkiv otf handler did this but then we found out that it violates the otf 'any order possible' rule so now we do just the order as we encounter it (and even then it took us a while to figure out where/how to avoid interferences); so ... it's up to the font designer (or technician)

IIRC, the 'ccmp' should be applied before any other lookup, the rest are applied as they are ordered in the font, at least this makes sense more.

now, having init, medi, fina, isol, rlig enabled might be the default, but for instance arabtype output looks much better when instead of rlig we use calt/clig etc so, in that case the defaults would bot be the best choice; again an argument for a database approach

Arabic Typesetting is a quit a special case font, though I believe that 'calt' and 'clig' should be on by default for any font.

there are even fonts out there that have the right gsub/gpos info but lack the feature being listed in the script/lang dictionary which is complicating live even more; maybe we should assume that the renderers of ms and adobe have some additional built in heuristics of using specific fonts

Any links to such fonts? Looks interesting. Regards, Khaled -- Khaled Hosny Arabic localizer and member of Arabeyes.org team

On Fri, Feb 20, 2009 at 03:46:50PM -0700, Idris Samawi Hamid ادريس سماوي حامد wrote:

On Fri, 20 Feb 2009 11:39:23 -0700, Khaled Hosny wrote:

...

Microsoft's OpenType features list page (http://www.microsoft.com/typography/otspec/features_ae.htm) gives a "UI suggestion" for each feature noting if it should be on by default,

Hmm, I think this page is more relevant for what you have in mine:

http://www.microsoft.com/typography/otfntdev/arabicot/features.htm

Though it doesn't suggest what should be on/off by default, I think it is a good starting point. From the features listed there, I think only 'dlig' and 'cswh' don't make much sense to be on by default, so I think the predefined 'arabic' font features should be changed to disable 'dlig' and enable 'calt' and 'mset' instead, as I think both are needed for Arabic fonts (especially 'calt' which is heavily used in Nasta'liq fonts). So, I think it should be: \definefontfeature [arabic] [mode=node,language=dflt,script=arab, init=yes,medi=yes,fina=yes,isol=yes, liga=yes,rlig=yes,clig=yes,calt=yes, mark=yes,mkmk=yes,kern=yes,curs=yes, mset=yes] Regards, Khaled -- Khaled Hosny Arabic localizer and member of Arabeyes.org team