Hans,

...

...

\parshape doesn't work as expected for TRT direction in PDF mode, it works fine in DVI mode but in PDF indentation moves the text to the right instead of to the left. This breaks LaTeX lists in TRT direction. ... do you have a simple example (btw, changes to the dir model and handling are very unlikely to happen)

Sure. Compare the behaviour of \leftskip and \parshape. IMO, they should be the same. --------------------------------- \parindent=0pt \hsize 11cm \outputmode=1 % No difference with 0 Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquid ex ea commodi consequat. {\leftskip1cm Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquid ex ea commodi consequat.\par} {\parshape 1 0cm 10cm Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquid ex ea commodi consequat.\par} \textdir TRT \pardir TRT \bodydir TRT \pagedir TRT {\leftskip1cm Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquid ex ea commodi consequat.\par} {\parshape 1 0cm 10cm Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquid ex ea commodi consequat.\par} \bye ---------------------- Javier

...

Sure. Compare the behaviour of \leftskip and \parshape. IMO, they should be the same.

Well, after thinking a little more about this I'm not so sure.

But same problem with \hangindent. This is easier to catch (just change the sign), but it would be nice if its behaviour is mirrored like \parindent, \leftskip, etc. I think LuaTeX must overcome the limitations of the XeT-TeX bidi model. Javier

Hans,

...

...

...

Sure. Compare the behaviour of \leftskip and \parshape. IMO, they should be the same.

Well, after thinking a little more about this I'm not so sure.

But same problem with \hangindent. This is easier to catch (just change the sign), but it would be nice if its behaviour is mirrored like \parindent, \leftskip, etc. I think LuaTeX must overcome the limitations of the XeT-TeX bidi model.

changing such things now will break existing code

I know, but not changing it breaks also existing code for bidi texts, except if we modify lots of things, which is an overkill (and very likely the code will end up with many hacks). And after all, 0.95 also broke existing code. I think not adding now a proper bidi model is a missed opportunity and in the years to come very likely we'll speak of a "long standing bug" in the bidi behavior of \hangindent. It's MHO. Javier

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

Hans,

I'll look at it but if something gets added, then it will be an option (keyword or variable) and not a default.

That would be nice.

After all, for proper bidi support one also needs to implement a proper layout model so it's never a simple switch thing. I'm pretty sure that this is an area where macro packages make fundamental different choices.

Agreed, but at the primitive level mirroring \hangindent would simplify the code a lot. More complicated is \parshape for the reasons I myself explained and I wonder how to handle it (about 130 files, iirc, are using it in TeXLive, many of them emulating what LaTeX does in \list, the only place where it's used in the core). To be honest, I don't know what to do. Suggestions? Javier

On 7/8/2016 7:18 PM, Javier Bezos wrote:

Hans,

...

I'll look at it but if something gets added, then it will be an option (keyword or variable) and not a default.

That would be nice.

...

After all, for proper bidi support one also needs to implement a proper layout model so it's never a simple switch thing. I'm pretty sure that this is an area where macro packages make fundamental different choices.

Agreed, but at the primitive level mirroring \hangindent would simplify the code a lot. More complicated is \parshape for the reasons I myself explained and I wonder how to handle it (about 130 files, iirc, are using it in TeXLive, many of them emulating what LaTeX does in \list, the only place where it's used in the core). To be honest, I don't know what to do. Suggestions?

I can't speak for latex, sorry. Anyway, I'll add this: \shapemode=0 % current behaviour \shapemode=1 % hangindent mirrored \shapemode=2 % parshape mirrored \shapemode=3 % both mirrored So we can be compatible and you can use value 3 and cross your fingers that there is no interference with other packages Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

On Fri, 08 Jul 2016 10:55:36 -0600, Hans Hagen wrote:

On 7/8/2016 6:48 PM, Javier Bezos wrote:

...

Hans,

...

...

...

...

Sure. Compare the behaviour of \leftskip and \parshape. IMO, they should be the same.

Well, after thinking a little more about this I'm not so sure.

But same problem with \hangindent. This is easier to catch (just change the sign), but it would be nice if its behaviour is mirrored like \parindent, \leftskip, etc. I think LuaTeX must overcome the limitations of the XeT-TeX bidi model.

changing such things now will break existing code

I know, but not changing it breaks also existing code for bidi texts, except if we modify lots of things, which is an overkill (and very likely the code will end up with many hacks). And after all, 0.95 also broke existing code. I think not adding now a proper bidi model is a missed opportunity and in the years to come very likely we'll speak of a "long standing bug" in the bidi behavior of \hangindent. It's MHO.

Well, if one looks at it from the point of viuew of the par builder where left/right skips are kind of global and applied later to the broken lines, then it's not a bug but a choice.

Since Hans is adding support for more mirroring options, I suppose the following is moot. In any case, I played around with this today. It seems that \parshape and \hangindent are interpreted so as to assume a global coordinate system for the page, i.e., beyond the paragraph level. After thinking about this global-page vs local-paragraph distinction, went back and found the old Omega documentation: There were two other primitives \pardirHR \pardirLR (which were later expunged from luatex). I suspect that \parshape and \hangindent were mirrored with \pardirHR invoked. If so, then the current issue raised by Javier is a residue of the old Omega/Aleph model: Dropping \pardirHR \pardirLR left an ambiguity. If the above analysis is correct, we should ask: What other primitives if any (besides \parshape and \hangindent) did the Omega/Aleph bidi model treat at the \pardirHR \pardirLR level? I think Hans' point (philosophy?) is that the macropackage should handle things at the page/coordinate-system level. Perhaps that had something to do with dropping \pardirHR \pardirLR in the first place? A related point is that luatex is explicitly designed for extensibility of, e.g, primitives via lua, so there should also be a straightforward way for experts to implement the desired low-level behavior. For example, something along the lines of \pardirHR \pardirLR could be reimplemented in lua. That *might* be a better approach than the mirroring Javier requested and which Hans has agreed to do; especially if there is additional residue out there of more primitives that need this kind of treatment. Again, the question may be moot now, but that's my 2 cents (perhaps 1 cent). Best wishes Idris -- Idris Samawi Hamid Professor of Philosophy Colorado State University Fort Collins, CO 80523

On 7/10/2016 2:19 AM, Idris Samawi Hamid ادريس سماوي حامد wrote:

On Fri, 08 Jul 2016 10:55:36 -0600, Hans Hagen wrote:

...

On 7/8/2016 6:48 PM, Javier Bezos wrote:

...

Hans,

...

...

...

> Sure. Compare the behaviour of \leftskip and \parshape. > IMO, they should be the same.

Well, after thinking a little more about this I'm not so sure.

But same problem with \hangindent. This is easier to catch (just change the sign), but it would be nice if its behaviour is mirrored like \parindent, \leftskip, etc. I think LuaTeX must overcome the limitations of the XeT-TeX bidi model.

changing such things now will break existing code

I know, but not changing it breaks also existing code for bidi texts, except if we modify lots of things, which is an overkill (and very likely the code will end up with many hacks). And after all, 0.95 also broke existing code. I think not adding now a proper bidi model is a missed opportunity and in the years to come very likely we'll speak of a "long standing bug" in the bidi behavior of \hangindent. It's MHO.

Well, if one looks at it from the point of viuew of the par builder where left/right skips are kind of global and applied later to the broken lines, then it's not a bug but a choice.

Since Hans is adding support for more mirroring options, I suppose the following is moot. In any case, I played around with this today. It seems that \parshape and \hangindent are interpreted so as to assume a global coordinate system for the page, i.e., beyond the paragraph level.

After thinking about this global-page vs local-paragraph distinction, went back and found the old Omega documentation: There were two other primitives

\pardirHR \pardirLR

(which were later expunged from luatex).

I suspect that \parshape and \hangindent were mirrored with \pardirHR invoked. If so, then the current issue raised by Javier is a residue of the old Omega/Aleph model: Dropping \pardirHR \pardirLR left an ambiguity.

i can imagine a real mess with using them .. it definitely assumes macro package level support for consistency (and what if different mechanisms want different pdfdirs); storing some state in the pardir node will not help here

If the above analysis is correct, we should ask: What other primitives if any (besides \parshape and \hangindent) did the Omega/Aleph bidi model treat at the \pardirHR \pardirLR level?

with parindent we can imagine a key but not with hangindent

I think Hans' point (philosophy?) is that the macropackage should handle things at the page/coordinate-system level. Perhaps that had something to do with dropping \pardirHR \pardirLR in the first place?

Indeed, as the document layout is kind of independent of the writing direction. In fact one can even dispute the left|right skip but those skips are added later (in fact, in context i have places where i do quite some juggling to get around it but that's beyond this discussion)

A related point is that luatex is explicitly designed for extensibility of, e.g, primitives via lua, so there should also be a straightforward way for experts to implement the desired low-level behavior. For example, something along the lines of \pardirHR \pardirLR could be reimplemented in lua. That *might* be a better approach than the mirroring Javier requested and which Hans has agreed to do; especially if there is additional residue out there of more primitives that need this kind of treatment.

hm, that make me think that we can reset \shapemode after each paragraph, like hangindent and parshape are .. let me think about that as it makes much sense

Again, the question may be moot now, but that's my 2 cents (perhaps 1 cent).

Best wishes Idris

-- ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

On 7/11/2016 4:05 PM, Javier Bezos wrote:

...

Indeed, as the document layout is kind of independent of the writing direction.

The writing direction could influence the perception of pictures and graphic elements. A good example is The Great Wave off Kanagawa. In a document, think of an arrow introducing an item. But, of course, these things belong to the package level, while indents and the like are pure text and therefore should be handled by TeX itself.

fighting hard codes built in assumptions is a pain which is why we don't extend the inner working (apart from a bit configurability and callbacks) ... there are many views on how mixed direction can be combined with a specific directional layout and rendering the paragraph is just one thing that happens and even 'pure text' is a fluent concept ... getting some agreement on how-to is basically a no-go and therefore lua is chosen as way to reach your goals Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

Hi Javier, On Mon, 11 Jul 2016 07:47:01 -0600, Javier Bezos wrote:

...

A related point is that luatex is explicitly designed for extensibility of, e.g, primitives via lua, so there should also be a straightforward way for experts to implement the desired low-level behavior. For example, something along the lines of \pardirHR \pardirLR could be reimplemented in lua. That *might* be a better approach than the mirroring Javier requested and which Hans has agreed to do; especially if there is additional residue out there of more primitives that need this kind of treatment.

Actually, \parshape can be mirrored even in pure TeX. But I still believe this must be done at the primitive level, and not with some hack. I think luatex is about making even better paragraphs from good paragraphs, and not as a hack to fix some directional properties.

lua extension =! hack In MkIV Hans built a paragraph optimizer in lua; he did not do it at the primitive level where one might have expected. A proper extension of functionality is just that, an extension in functionality. If \pagedir TRT etc. is no longer there, a properly designed lua extension to provide the needed functionality is not a "hack". luatex is about paragraphs, but it is equally about taking advantage of lua to extend functionality as needed by various applications, *without having to recode the primitives*. Indeed, that kind of extensibility was arguably its most pressing raison d'etre. Put another way: In luatex we are not given the binary opposition of primitive support vs TeX-"hack". We have a third choice, and that is genuine extensibility. Best wishes Idris -- Idris Samawi Hamid Professor of Philosophy Colorado State University Fort Collins, CO 80523

...

I think luatex is about making even better paragraphs from good paragraphs, and not as a hack to fix some directional properties.

lua extension =! hack

In MkIV Hans built a paragraph optimizer in lua; he did not do it at the primitive level where one might have expected.

This is what I said :-). Javier