Dear, Taco and Hans Thanks for your answer. TeX-e-Parsi is better than omega because it was written a long time ago (15 years ago) and since then, the developer (Yazdipur) has modified and corrected the code. So firstly I think it does not have any bugs or if it has, it has a very small number of bugs comparing to Omega. Secondly TeX-e-Parsi have been used by the Iranians for the whole period and still people continue to use it and they are very happy with it since it is not buggy at all. I believe TeX-e-Parsi is much more advanced than Omega and it defines a set of new primitives to support LTR and RTL typesetting fully in addition to changing some of TeX codes. It also provides the \eqprimitive primitive so that we can define TeX primitive commands to the appropriate Persian/Arabic names and that can be done for any other language. So a user who does not speak English, does not need to learn TeX's English primitive to write macros, he just can use his language version of the primitives and at the same time the original TeX commands will work. Let me tell you that unfortunately the manual is written in Persian and that does not say anything about the actual engine, it is just Parsi LaTeX user manual. If you intend to look at this seriously, I will be more than happy to translate the comments in the tex.ch to English (Since the comments are written in Persian and TeX-e-Parsi is 8 bit, you will see some strange characters when Persian comments are present) Just let me, if you need me to write the comments in English. Thanks I trust my family jewels only to Linux _________________________________________________________________ Time for change? Find your ideal job with SEEK. http://a.ninemsn.com.au/b.aspx?URL=http%3A%2F%2Fninemsn%2Eseek%2Ecom%2Eau%2F%3Ftracking%3Dsk%3Atl%3Ask%3Anine%3A0%3Ahottag%3Achange&_t=757263783&_r=SEEK_tagline&_m=EXT

Dear Vafa, Salaam On Thu, 25 Dec 2008 04:03:03 -0700, وفا خلیقی، Vafa Khalighi wrote:

TeX-e-Parsi is better than omega because it was written a long time ago (15 years ago) and since then, the developer (Yazdipur) has modified and corrected the code.

Hmm, not sure if this is completely correct: See below. I'm just curious: Did Yazdipur try to communicate with the Omega developers and offer patches etc? Another question: TeX-e-Parsi was originally closed source, I think: What is the status of the source code?

So firstly I think it does not have any bugs or if it has, it has a very small number of bugs comparing to Omega.

I think the same is true of luatex in this regard.

Secondly TeX-e-Parsi have been used by the Iranians for the whole period and still people continue to use it and they are very happy with it since it is not buggy at all.

Have you tested the bidi in luatex and compared it with that of TeX-e-Parsi? It is already much better than etex's. According to Behdad Esfahbod and Roozbeh Pournader, http://www.tug.org/TUGboat/Articles/tb23-1/farsitex.pdf TeX-e-Parsi has not been developed since 1996. I know that there has been development of Omega since then, including a new bidi model -- with new primitives -- that replaced the one in pre-1996 Omega. Looking at the primitive list you provided, I don't think TeX-e-Parsi builds on the later, improved Omega model. A better approach might be to 1. Make an exhaustive test suite that compares bidi behavior in luatex with that of TeX-e-Parsi. You don't need to use any arabic-script, just latin modern samples; 2. Identify the diffferences between the two and determine which -- if either -- has the saner behavior for each case; 3. Make specific feature/debugging requests and -- if possible -- point to areas where TeX-e-Parsi code may be useful in implementing the features you want. I did such a suite when we were testing bidi a couple of years back -- comparing aleph with luatex and looking for bugs. IIRC the remaining area where luatex needs work is in its handling of verbatim, but I need to go back to my test files and check.

I believe TeX-e-Parsi is much more advanced than Omega and it defines a set of new primitives to support LTR and RTL typesetting fully in addition to changing some of TeX codes.

As I mentioned above, I think the last Omega primitives are newer than TeX-e-Parsi's. But a related issue: Are those primitives based on the unicode bidi algorithm? There is also an ongoing implementation of the algorithm in the MKIV inteface to luatex. Again, the best thing may be to identify missing features and bugs, then the developers can identify the best approach to implementing/fixing things. That may or may not involve using TeX-e-Parsi code. سلام Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523

On Thu, 25 Dec 2008 19:53:00 -0700, وفا خلیقی، Vafa Khalighi wrote:

...

I'm just curious: Did Yazdipur try to communicate with the Omega developers and offer patches etc?

I do not think so, It seems that he has got no interest in the Unicode world and he is a big fan of Knuth's TeX engine only and does not want to write codes for PDFTeX, XeTeX and LuaTeX.

Hmm, if that is true then it is not very promising...

...

Have you tested the bidi in luatex and compared it with that of TeX-e-Parsi? It is already much better than etex's.

Yes, I did and I still think TeX-e-Parsi's bidirectional algorithm is more powerful than LuaTeX's.

Ok, but how do you qualitatively and/or quantitatively justify that?

As I said Yazdipour has faith only in Donald Knuth and his work and no interest in other TeX engines, that is what I have heard. Donald Knuth himself congratulated him for writing the TeX-e-Parsi's engine.

Again, I think that Yazdipour's approach here is anachronistic -- with all due respect. But if this is indeed the case, then this tells me that there is no Omega code in TeX-e-Parsi at all. So in effect you are asking that luatex ditch the entire bidi model it has and replace it with TeX-e-Parsi's. Then we have to hook that in to pdf output and many other things. Given the stability and polish of the bidi engine as it stands I am not sure this is wise at the moment

...

1. Make an exhaustive test suite that compares bidi behavior in luatex with that of TeX-e-Parsi. You don't need to use any arabic-script, just latin modern samples;

I absolutely agree. But Let's have a test file. I write that test file with TeX-e-Parsi and you write the test file with LuaTeX and then we compare the results. If you can send a sample/test file so then I can typeset in with TeX-e-Parsi.

Hmm: You should create and run your test file -- which should contain a variety of individual torture tests -- with both luatex and TeX-e-Parsi. Compare the pdf's and identify the differences. Write a report and submit it to Hans, Taco, and me. I have attached two test files from almost exactly one year ago. You can modify these to your liking and see what happens. But I suspect you'll want to create your own tests from scratch. The apparent verbatim bug is there, on the first page of global_rl2.pdf, but other than that bidi is quite solid. Taco: Interestingly, running the file today results in a bug, but a year ago it showed up in the last two lines whereas now it only shows up in the last line on page 1.

...

2. Identify the diffferences between the two and determine which -- if either -- has the saner behavior for each case;

Agree. It might turns out that TeX-e-Parsi is powerful in some areas and LuaTeX in other areas. So we just can look at the powerful areas of bidirectional typesetting.

exactly. Best wishes Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523

On Thu, 25 Dec 2008, Idris Samawi Hamid ????? ????? ???? wrote:

I have attached two test files from almost exactly one year ago.

just a tiny remark: maybe it would be practical if we could stick to plain TeX for such test files, as this can exercise the primitives directly, may produce more compact/generic/primitive PDF code, and there is no need then to think whether what we see are effects of ConTeXt or "the engine". Regards, Hartmut

On Fri, 26 Dec 2008 05:25:46 -0700, Hartmut Henkel wrote:

On Thu, 25 Dec 2008, Idris Samawi Hamid ????? ????? ???? wrote:

...

I have attached two test files from almost exactly one year ago.

just a tiny remark: maybe it would be practical if we could stick to plain TeX for such test files, as this can exercise the primitives directly, may produce more compact/generic/primitive PDF code, and there is no need then to think whether what we see are effects of ConTeXt or "the engine".

Sure, just that this was all I had at the moment, and no time at all to develop new files (I did suggest to Vafa to start from scratch) :-) Best wishes Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523

Le 26 déc. 08 à 02:03, Idris Samawi Hamid ادريس سماوي حامد a écrit :

As I mentioned above, I think the last Omega primitives are newer than TeX-e-Parsi's. But a related issue: Are those primitives based on the unicode bidi algorithm?

Implementing Unicode bidi will mean keeping track of the (directionality of the) first character of every horizontal list so that calculations of embedding levels can be done. Omega has never done that, and it couldn't do it because OTPs use buffers which are most of the time smaller than the whole paragraph. Is TeX-e-Parsi doing it? Furthermore you all talk about LTR and RTL, but what about vertical typesetting, in the Chinese or in the Mongolian way? Obviously TeX-e- Parsi doesn't care about that. -- + -----------------------------------------------------------------------+ | Yannis Haralambous, Ph.D. yannis.haralambous@telecom- bretagne.eu | | Directeur d'Études http://omega.enstb.org/ yannis | | Tel. +33 (0)2.29.00.14.27 | | Fax +33 (0)2.29.00.12.82 | | Département Informatique | | TELECOM Bretagne | | Technopôle de Brest Iroise, CS 83818, 29238 Brest CEDEX 3, France | | Coordonnées Google-Earth : 48°21'31.57"N 4°34'16.76"W | + -----------------------------------------------------------------------+ ...pour distinguer l'extérieur d'un aquarium, mieux vaut n'être pas poisson ...the ball I threw while playing in the park has not yet reached the ground Es gab eine Zeit, wo ich nur ungern über Schubert sprechen, nur Nächtens den Bäumen und Sternen von ihm vorerzählen mögen.

I have produced some sample files with TeX-e-Parsi. You can have a look at them at http://parsilatex.org/parsitex/ Please let me know if you were looking for a different kind of output. In my next email, I will be explaining TeX-e-Parsi changes. _________________________________________________________________ Holiday cheer from Messenger. Download free emoticons today! http://livelife.ninemsn.com.au/article.aspx?id=669758

I have attached a document which explains the main changes which needs to be done. _________________________________________________________________ Time for change? Find your ideal job with SEEK. http://a.ninemsn.com.au/b.aspx?URL=http%3A%2F%2Fninemsn%2Eseek%2Ecom%2Eau%2F%3Ftracking%3Dsk%3Atl%3Ask%3Anine%3A0%3Ahottag%3Achange&_t=757263783&_r=SEEK_tagline&_m=EXT

I read "If we have \autoLRdirset, then characters 0 to 127 as LTR characters and characters 128 to 255 as RTL characters, will be assumed and based on this the direction of the typesetting will be determined." First of all, both LTR and RTL characters are far more than 128. And secondly, there are many characters which are neither LTR nor RTL (like the period, exclamation mark, delimiters, etc.). Are you using two different periods, two different exclamation marks, etc.? That was the Apple approach during the Worldscript era. Unicode bidi algorithm has be designed to take all these issues into account. Le 27 déc. 08 à 11:08, وفا خلیقی، Vafa Khalighi a écrit :

-- + -----------------------------------------------------------------------+ | Yannis Haralambous, Ph.D. yannis.haralambous@telecom- bretagne.eu | | Directeur d'Études http://omega.enstb.org/ yannis | | Tel. +33 (0)2.29.00.14.27 | | Fax +33 (0)2.29.00.12.82 | | Département Informatique | | TELECOM Bretagne | | Technopôle de Brest Iroise, CS 83818, 29238 Brest CEDEX 3, France | | Coordonnées Google-Earth : 48°21'31.57"N 4°34'16.76"W | + -----------------------------------------------------------------------+ ...pour distinguer l'extérieur d'un aquarium, mieux vaut n'être pas poisson ...the ball I threw while playing in the park has not yet reached the ground Es gab eine Zeit, wo ich nur ungern über Schubert sprechen, nur Nächtens den Bäumen und Sternen von ihm vorerzählen mögen.

As you can see in TeX-e-Parsi we can have automatic direction change.

The fact automatic direction change is hardcoded is a serious drawback, and not and advantage, imo. There are many subtle points in bidi typesetting which can be addressed only in a programmable environment, like that provided by luatex. Even with otps, which are very limited as compared with luatex, I managed to implement a preliminary automatic direction change (in fact, an automatic script change, including direction). Javier ----------------------------------- http://www.tex-tipografia.com

"automatic direction change" is actually what we call "bidi algorithm". Unicode provides special characters for explicitly changing direction when the automatic direction change is not sufficient. OTPs cannot do it because they give up on every token which is not a character with specific catcodes. Le 27 déc. 08 à 17:35, وفا خلیقی، Vafa Khalighi a écrit :

The fact automatic direction change is hardcoded is a serious drawback, and not and advantage, imo. There are many subtle points in bidi typesetting which can be addressed only in a programmable environment, like that provided by luatex. Even with otps, which are very limited as compared with luatex, I managed to implement a preliminary automatic direction change (in fact, an automatic script change, including direction).

I believe the automatic direction change is the main advantage otherwise typesetting with TeX is unnatural and useless.

Download free emoticons today! Holiday cheer from Messenger. _______________________________________________ dev-luatex mailing list dev-luatex@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-luatex

-- + -----------------------------------------------------------------------+ | Yannis Haralambous, Ph.D. yannis.haralambous@telecom- bretagne.eu | | Directeur d'Études http://omega.enstb.org/ yannis | | Tel. +33 (0)2.29.00.14.27 | | Fax +33 (0)2.29.00.12.82 | | Département Informatique | | TELECOM Bretagne | | Technopôle de Brest Iroise, CS 83818, 29238 Brest CEDEX 3, France | | Coordonnées Google-Earth : 48°21'31.57"N 4°34'16.76"W | + -----------------------------------------------------------------------+ ...pour distinguer l'extérieur d'un aquarium, mieux vaut n'être pas poisson ...the ball I threw while playing in the park has not yet reached the ground Es gab eine Zeit, wo ich nur ungern über Schubert sprechen, nur Nächtens den Bäumen und Sternen von ihm vorerzählen mögen.

You say "it takes care of everything" as if all problems of Arabic typography have been solved. They are not! It would be interesting to find out what exactly your system does more than (a) Omega and (b) luatex. And I don't really see what the issue is: are we talking for or against the bidirectional algorithm? or is it for or against your system's typesetting features? In the first case, then I think the issue is clear: every current Arabic system must be bidi-compliant. In the second case, I would like to know more about your system, so please do not disappear. Le 27 déc. 08 à 20:09, وفا خلیقی، Vafa Khalighi a écrit :

I had tried my best to show another approach which I think is more advanced, it is bug free and takes care of everything and at the same time can be modeled for vertical texts as well.

If you think that the current bidirectional algorithm of LuaTeX is just fine, then this will be my last email.

Find your ideal job with SEEK Time for change? _______________________________________________ dev-luatex mailing list dev-luatex@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-luatex

-- + -----------------------------------------------------------------------+ | Yannis Haralambous, Ph.D. yannis.haralambous@telecom- bretagne.eu | | Directeur d'Études http://omega.enstb.org/ yannis | | Tel. +33 (0)2.29.00.14.27 | | Fax +33 (0)2.29.00.12.82 | | Département Informatique | | TELECOM Bretagne | | Technopôle de Brest Iroise, CS 83818, 29238 Brest CEDEX 3, France | | Coordonnées Google-Earth : 48°21'31.57"N 4°34'16.76"W | + -----------------------------------------------------------------------+ ...pour distinguer l'extérieur d'un aquarium, mieux vaut n'être pas poisson ...the ball I threw while playing in the park has not yet reached the ground Es gab eine Zeit, wo ich nur ungern über Schubert sprechen, nur Nächtens den Bäumen und Sternen von ihm vorerzählen mögen.

On So, 28 Dez 2008, وفا خلیقی، Vafa Khalighi wrote:

I just felt that you guys think the current bidirectional algorithm of LuaTeX is fine and have no interest to study other systems.

Because the bidi algo of luatex is more advanced, it can do arbitrary rotation text/char/page directions, not simply left-right-left. Finished. That is needed for Chinese, Japanese, Mongolian to name a few. Best wishes Norbert ------------------------------------------------------------------------------- Dr. Norbert Preining Vienna University of Technology Debian Developer Debian TeX Group gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094 ------------------------------------------------------------------------------- GRIMSBY (n.) A lump of something gristly and foultasting concealed in a mouthful of stew or pie. Grimsbies are sometimes merely the result of careless cookery, but more often they are placed there deliberately by Freemasons. Grimbies can be purchased in bulk from any respectable Masonic butcher on giving him the secret Masonic handbag. One is then placed correct masonic method of dealing with it. If the guest is not a Mason, the host may find it entertaining to watch how he handles the obnoxious object. It may be (a) manfully swallowed, invariably bringing tears to the eyes. (b) chewed with resolution for up to twenty minutes before eventually resorting to method (a) (c) choked on fatally. The Masonic handshake is easily recognised by another Mason incidentally, for by it a used grimsby is passed from hand to hand. The secret Masonic method for dealing with a grimsby is as follows : remove it carefully with the silver tongs provided, using the left hand. Cross the room to your host, hopping on one leg, and ram the grimsby firmly up his nose, shouting, 'Take that, you smug Masonic bastard.' --- Douglas Adams, The Meaning of Liff

Hi Just a reminder, which is probably not necessary. In The TeX Boook, Donald Knuth wrote (page 226): === The author hopes that such extensions will not be made very often, because he doesn't want incompatible pseudo-TeX systems to proliferate; yet he realizes that certain special books deserve special treatment. === I'm encouraged, that a dialog is taking place. -- Jonathan

David Kastrup wrote:

As far as I understand, LuaTeX is a secular institution with regard to the worship of Knuth and his word.

What? I'm confident that all of us have a great respect and admiration for Donald Knuth and his work, and that none of us worship him. In particular, it is accepted and public knowledge that he has made mistakes in writing TeX. I'd like to stop this particular discussion here, because it is off-topic. -- Jonathan

Hans Hagen wrote:

well, you made a remark and david's responded, so the discussion is now two mails long

anyhow, if it's so off topic (and david's mail was quite on-the-topic that you started), then why did you made your remark in the first place;

To answer your question, briefly. In my view, my remark (on proliferation of pseudo-TeXs) was relevant and David's remark (on worship of Donald Knuth) was not.

but we can indeed stop here

Having answered your question, I'm happy for this discussion to end. -- Jonathan

Hans Hagen skribis:

actually i've heard Don say (at an ntg QA session long ago) that he was surprised that there were no more fundamental extensions like pdftex (at that time upcoming)

TeX was written in Pascal instead of C, and so there didn't end up being IconTeX in the 1980s instead of LuaTeX in the 2000s. :) C ended up being what Pascal had been expected to become, circa 1980.

Jonathan Fine wrote:

...

actually i've heard Don say (at an ntg QA session long ago) that he was surprised that there were no more fundamental extensions like pdftex (at that time upcoming); also, there is a difference between (say) 2^4 fundamental different tex variants and 2^32 confusingly slightly unsimilar ones

Hans: I'm looking at the transcripts (in Digital Typography) and I think you're referring to the CS TUG session (pages 617-8) and not the NTG session.

Huh? I was at that ntg session in Amsterdam which is an 10 hour drive from Czech. Ok, I might be making things up and creating my own truth but somehow my I stored that remark in my memory (as usual at such a QA session someone asks "what do you think of developments like etex, omega, next generation tex etc" and i think it was related to such a question. But anyhow, I don't want to put answers in someone's mouth so if you're so sure that I'm wrong, forget about it.

Hans: Your comment has two parts, separated by an 'also'. Just to clarify, the first part is your paraphrase of Don Knuth and the second part (so far as I can tell) comes just from you, and not at all from Don Knuth.

Sure, I could have used a . instead of a. I was refering to your quote of "pseudo-TeX systems to proliferate" and I get the feeling that etex, omega, pdftex, luatex don't fall into this category ... all have been presented at meetings, discussed, even with Don being present and never have I heard him say/write 'please quit the development of your pseudo tex' or 'hey, you may not use tex as part of its name'. And this is what David meant (I think): let Don talk for himself and don't start a religion (by interpreting and putting things in someones mouth, like: we should use dvi because that is what Don wants (who says?) or stick to tex (because Don wrote it and sticks to it for clear reasons that everyone understands), while actually Don is way more open minded than some of his followers can grasp).

No need for anyone to reply to this (except David K, off-list if he wishes).

Well, no need for *you* to reply for this (after all you're now continuing a discussion that you yourself quit), but david might have another go at it -) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

What? I'm confident that all of us have a great respect and admiration for Donald Knuth and his work, and that none of us worship him.

Just out of interest, how do you know this? I, myself, indulge in daily practice of libations and creed in honor of His achievements. I consider using TeX a commitment whose extent can only be truly expressed by observing the appropriate forms and rites. I encourage you, and all other users of TeX and similar programs to do the same. This is, in His words, what Knuth asked of us. For myself, when I use TeX, I mean to honor the program that Donald Knuth wrote for us. To do otherwise is against the spirit if not the letter of Knuth's wishes. Obviously, I don't expect that everyone's faith be as strong as mine, but I am confident that TeX believers will, in time, find a suitable manner for expressing their reverence. And of course, where it comes to celebrations, I trust that all of us enjoy the occasional naked dance around the bonfire at BachoTeX, during Walpurgis Night (und raubt man uns den alten Brauch, / dein Licht, wer kann es rauben?). Arthur -- Tutto nel mondo è burla!

Can you show me an example of vertical typesetting in LuaTeX?

Try the attached file. It demonstrates vertical typesetting with LuaTeX in three steps; unfortunately it only works in DVI mode for the moment because the PDF output is completely buggy, I discussed with Taco about that. 1. At first, you simply need to switch to vertical directions, using the primitives inherited from Omega. Here I used LTL as the general direction: the first letter indicates the first edge of the page is at the left (lines of text move globally from left to right); 'T' means the second edge of the page is at the top (inside a single line, text moves from the top downwards). The last letter defines the notion of "top" for individual letters, I chose left, more or less randomly. It has some influence, as we will see later, but not as much as the first two letters. Then you can set English text vertically, with the columns running from left to right. It's akward to use OpenType fonts in DVI mode, so I simply used Computer Modern Typewriter, stacking the letters on top of each other. Of course it's weird to set English text vertically without rotating the letters, but it is possible. There is a problem, though, because the text looks completely clumsy: the reason is the font's metrics are not designed for vertical typesetting, so you need to change them. The way metrics work in Omega's directional model is that width runs along the direction of the line, here top-to-bottom; hence a glyph's width, in that example, is actually the sum of its height and its depth in the original font designed for horizontal typesetting. Its height and depth, in turn, need to be chosen so that their sum is equal to the horizontal width, and I think it's a good choice to make them equal to each other, so that each glyph is centered on the vertical base line. 2. On the second page I demonstrate a first way to do that, with Lua code: you can use the define_font callback to redefine width, height and depth for each glyph in the font, and LuaTeX will use these new metrics. Note that the vertical width should, actually, be a little bit more than horizontal height + horizontal depth, because these dimensions are not meant to blend well with stacks of letters, unlike the horizontal width: the former are really more like the dimensions of a bounding box, whereas the latter is the distance by which you should advance the line of text in order for the text not to look to crowded. Thus, I cheated a bit, by stretching the vertical width by 30% (chosen, again, at random by trial and error). But that's still not enough: as you can see, letters with descender, like 'g' in the first line, sticks into the next letter. That's because it has a horizontal depth, and its origin is thus too low with respect to its enclosing box: we need to raise those letters by their horizontal depth. 3. We could do this by editing the font and modify every glyph so that it sits on the baseline, but we can do better by using virtual fonts. In LuaTeX, there are two ways to use virtual fonts: you can use a .vf file, or you can define it with Lua commands in the font data. Unfortunately, the latter doesn't work in DVI mode because your Lua code is lost once the DVI file is produced, and your DVI reader / driver doesn't know how to create the font, so we have to stick to the traditional method and produce a .vf file. I wrote a small script to that effect which you will find in the attached zip, but the .vf and .tfm files are also attached anyway. Now the text looks quite nice, in my opinion. It's still hard to read, but that's of course because English is not supposed to be typeset that way. The (figurative) bottom line of all this is that you need to typeset vertical text with fonts that have vertical metrics. Once the problems with PDF mode are fixed, I can try and do that with Chinese or Japanese fonts; I expect it would be quite straightforward. Arthur

Since you are interested in vertical typesetting by Omega, here are ten pages from a book published in Greece: the translation of 90 haiku poems. The name of the author is written in Japanese with furigana. The book is: <http://www.bibliopolio.gr/ιαπωνικά-ποιήματα-p-95636.html?osCsid=6971e3a0dbd790ac 8014e83491cf4c11> (ISBN 960-7360-60-5) Le 30 déc. 08 à 18:57, Yue Wang a écrit :

Hi:

WOW! This is very impressive! Thanks for your work.

So I think we can add vertical CJK typesetting feature to LuaTeX/MKIV next year?

[Wolfgang is busy writing the wishlist for cjk typesetting, so i think he will be very interested in experimenting your results]

Yue Wang

On Tue, Dec 30, 2008 at 4:08 PM, Arthur Reutenauer wrote:

...

...

Can you show me an example of vertical typesetting in LuaTeX?

Try the attached file. It demonstrates vertical typesetting with LuaTeX in three steps; unfortunately it only works in DVI mode for the moment because the PDF output is completely buggy, I discussed with Taco about that.

1. At first, you simply need to switch to vertical directions, using the primitives inherited from Omega. Here I used LTL as the general direction: the first letter indicates the first edge of the page is at the left (lines of text move globally from left to right); 'T' means the second edge of the page is at the top (inside a single line, text moves from the top downwards). The last letter defines the notion of "top" for individual letters, I chose left, more or less randomly. It has some influence, as we will see later, but not as much as the first two letters.

Then you can set English text vertically, with the columns running from left to right. It's akward to use OpenType fonts in DVI mode, so I simply used Computer Modern Typewriter, stacking the letters on top of each other. Of course it's weird to set English text vertically without rotating the letters, but it is possible. There is a problem, though, because the text looks completely clumsy: the reason is the font's metrics are not designed for vertical typesetting, so you need to change them. The way metrics work in Omega's directional model is that width runs along the direction of the line, here top-to-bottom; hence a glyph's width, in that example, is actually the sum of its height and its depth in the original font designed for horizontal typesetting. Its height and depth, in turn, need to be chosen so that their sum is equal to the horizontal width, and I think it's a good choice to make them equal to each other, so that each glyph is centered on the vertical base line.

2. On the second page I demonstrate a first way to do that, with Lua code: you can use the define_font callback to redefine width, height and depth for each glyph in the font, and LuaTeX will use these new metrics.

Note that the vertical width should, actually, be a little bit more than horizontal height + horizontal depth, because these dimensions are not meant to blend well with stacks of letters, unlike the horizontal width: the former are really more like the dimensions of a bounding box, whereas the latter is the distance by which you should advance the line of text in order for the text not to look to crowded. Thus, I cheated a bit, by stretching the vertical width by 30% (chosen, again, at random by trial and error).

But that's still not enough: as you can see, letters with descender, like 'g' in the first line, sticks into the next letter. That's because it has a horizontal depth, and its origin is thus too low with respect to its enclosing box: we need to raise those letters by their horizontal depth.

3. We could do this by editing the font and modify every glyph so that it sits on the baseline, but we can do better by using virtual fonts. In LuaTeX, there are two ways to use virtual fonts: you can use a .vf file, or you can define it with Lua commands in the font data. Unfortunately, the latter doesn't work in DVI mode because your Lua code is lost once the DVI file is produced, and your DVI reader / driver doesn't know how to create the font, so we have to stick to the traditional method and produce a .vf file. I wrote a small script to that effect which you will find in the attached zip, but the .vf and .tfm files are also attached anyway. Now the text looks quite nice, in my opinion. It's still hard to read, but that's of course because English is not supposed to be typeset that way.

The (figurative) bottom line of all this is that you need to typeset vertical text with fonts that have vertical metrics. Once the problems with PDF mode are fixed, I can try and do that with Chinese or Japanese fonts; I expect it would be quite straightforward.

Arthur

_______________________________________________ dev-luatex mailing list dev-luatex@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-luatex

-- + -----------------------------------------------------------------------+ | Yannis Haralambous, Ph.D. yannis.haralambous@telecom- bretagne.eu | | Directeur d'Études http://omega.enstb.org/ yannis | | Tel. +33 (0)2.29.00.14.27 | | Fax +33 (0)2.29.00.12.82 | | Département Informatique | | TELECOM Bretagne | | Technopôle de Brest Iroise, CS 83818, 29238 Brest CEDEX 3, France | | Coordonnées Google-Earth : 48°21'31.57"N 4°34'16.76"W | + -----------------------------------------------------------------------+ ...pour distinguer l'extérieur d'un aquarium, mieux vaut n'être pas poisson ...the ball I threw while playing in the park has not yet reached the ground Es gab eine Zeit, wo ich nur ungern über Schubert sprechen, nur Nächtens den Bäumen und Sternen von ihm vorerzählen mögen.

On So, 28 Dez 2008, وفا خلیقی، Vafa Khalighi wrote:

If you think that the current bidirectional algorithm of LuaTeX is just fine, then this will be my last email.

It seems that you are not open for discussion. Taco, Hans, et al have asked you for explanations why you believe your approach is better, but you missed to provide facts. That is life, those who are doing the programming work have the rights to make decisions, and without good arguments from other sides they will do what they think best. And since there are quite a lot working with Arabic texts here on the list and it seems nobody chimed in with you, it seems that you are having an even smaller ground. Anyway, if you come back with good arguments I am sure Taco and Hans and Idris and whoever will take it up. Enjoy Norbert ------------------------------------------------------------------------------- Dr. Norbert Preining Vienna University of Technology Debian Developer Debian TeX Group gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094 ------------------------------------------------------------------------------- SCRONKEY (n.) Something that hits the window as a result of a violent sneeze. --- Douglas Adams, The Meaning of Liff

وفا خلیقی، Vafa Khalighi wrote:

...

It seems that you are not open for discussion. Taco, Hans, et al have asked you for explanations why you believe your approach is better, but you missed to provide facts.

I am open to discussions, and I think Hans said " a short description of the primitives is a good start" and I tried my first attempt by sending that doc explaining some of the primitives.

you have to give us some time to study things so there will be no quick answer (apart from the fact that it's a bit holiday season here) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Hi, وفا خلیقی، Vafa Khalighi wrote:

I have written a small doc explaining some of the main primitives. You can find it at http://parsilatex.org/parsitex/primitives.pdf

I've read that document and browsed through tex.ch a bit, and here are my thoughts so far: On direction changes -------------------- If I understand the document correctly, it appears that the automatic switch works in one of two ways: A you use a special 8-bit font with the upper half persian and the lower half latin. This case confines the input to an 8-bit encoding table, something like Microsoft page 1256. B or you can use two fonts; a full 8-bit arabic and an 8-bit latin or symbolic font, and then your font switch triggers the direction change. And you can always go back to explicit \begin[LR] \end[LR] commands, whenever you so desire. I hope I got that right. Method A can easily be implemented using lua input filtering in such a way that it is transparent to the luatex user (it should use Unicode characters and a Unicode-enabled font, of course). Doing that would not be hard at all and does not require engine changes. Method B is absolutely unsuitable for luatex because it assumes that there is only duality in scripts, instead of multiplicity. For the same reason, a number of the new primitives is unsuited for luatex (\semichar, \semispaceskip, \everysemi<>, etc.). On localization --------------- It is interesting that TeX-e-Parsi can read primitives and keywords in Persian, and report errors in Persian as well. In Luatex, it is possible to \let any utf-8 sequence to a primitive, but the localization of keywords and error messages is something that we definately can/should look into. Other stuff ----------- I can deduce that there are some accent extensions in TeX-e-Parsi, can you explain a bit about how that works? Reading the rest of the document, I am a bit curious what \lcode is. It seems related to wordstring detection? As far as I can see, the other typesetting-related primitives are already provided by luatex via Aleph via Omega, and usually in a more generic fashion. For example, TeX-e-Parsi has \curboxdir % read \rightvbox \leftvbox % write Luatex has \the\boxdir % read \boxdir <DIR> % write If you feel something is currently missing from luatex in this regard than you have to be explicit about that, I may well have overlooked something. Best wishes, Taco

well, knuth tex+cjk macro and xetex + macro support vertical typesetting. but will be very messy:( see http://gezhu.googlecode.com/files/demo-xelatex-071106.pdf for examples. Yue Wang 2008/12/26 Yannis Haralambous :

Le 26 déc. 08 à 02:03, Idris Samawi Hamid ادريس سماوي حامد a écrit :

As I mentioned above, I think the last Omega primitives are newer than TeX-e-Parsi's. But a related issue: Are those primitives based on the unicode bidi algorithm?

Implementing Unicode bidi will mean keeping track of the (directionality of the) first character of every horizontal list so that calculations of embedding levels can be done. Omega has never done that, and it couldn't do it because OTPs use buffers which are most of the time smaller than the whole paragraph. Is TeX-e-Parsi doing it? Furthermore you all talk about LTR and RTL, but what about vertical typesetting, in the Chinese or in the Mongolian way? Obviously TeX-e-Parsi doesn't care about that.

-- +-----------------------------------------------------------------------+ | Yannis Haralambous, Ph.D. yannis.haralambous@telecom-bretagne.eu | | Directeur d'Études http://omega.enstb.org/yannis | | Tel. +33 (0)2.29.00.14.27 | | Fax +33 (0)2.29.00.12.82 | | Département Informatique | | TELECOM Bretagne | | Technopôle de Brest Iroise, CS 83818, 29238 Brest CEDEX 3, France | | Coordonnées Google-Earth : 48°21'31.57"N 4°34'16.76"W | +-----------------------------------------------------------------------+ ...pour distinguer l'extérieur d'un aquarium, mieux vaut n'être pas poisson ...the ball I threw while playing in the park has not yet reached the ground Es gab eine Zeit, wo ich nur ungern über Schubert sprechen, nur Nächtens den Bäumen und Sternen von ihm vorerzählen mögen.

_______________________________________________ dev-luatex mailing list dev-luatex@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-luatex