On Tue, 07 Oct 2008 12:16:42 -0600, Hans Hagen wrote:

...

Hans and Taco can probably tell you more. it all depends on what we want to achieve ...

- we can have font fixers (dynamically extend fonts with features, but this demands knowledge of the font) - we can have replacements in the node list (attribute driven, relatively easy to implement) so before i start looking into this, we need to investigate what is needed (and what for)

For things like my journal, I get lots of inconsistent transliterations, and we are not yet at the stage where we can enforce behavior on authors. I would like to be able to do things like Mohamed -> Muḥammad which would save me a lot of time in proofreading, editing, etc. Getting the transliteration for the entire journal consistent is a tedious pain, and eventually I can have a nice database that will cover most cases. This is also much more convenient than using control-sequence definitions and watching out for spaces etc.

(only mkiv)

Indeed ;-) Best wishes Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523

On Wed, 08 Oct 2008 08:46:21 -0600, Mojca Miklavec wrote:

On Wed, Oct 8, 2008 at 2:41 PM, Idris Samawi Hamid ادريس سماوي حامد wrote:

...

I would like to be able to do things like

Mohamed -> Muḥammad

which would save me a lot of time in proofreading, editing, etc.

It's easy to write a simple ruby/perl(or even lua) script that will do such changes for you *before* typesetting articles. That's probably much better, since you also want to have clean sources at some point.

Yes, but the above is just one kind of example. Indeed, what I am thinking of is just an extension of the kind of configurability texies like to do. For your average user -- or a secretary -- we don't want to have to point them to lua or ruby etc...

Of course, writing the list of substitutions still needs to be done by you, but that's in either case.

If you need that, write kind-of-specification how you would like to use such a command.

Maybe I misunderstand... I could see something like I mentioned in the original mail \definesubstitution{<string1>}{<string2>} or even a database approach: \definesubstitutions[ <string1>,| <string2>,| <string3>, ] Here is another example. I generally prefer an en-dash padded with spaces -- and with a no-break space after -- to an em-dash for parenthetical constructions. So I would like to do, eg \definesubstitution{<emdash>}{ <endash>~} Again, this saves time in extensive editing, and I can stay in TeX without fiddling so much with scripting, CTRL-F/H/R etc. It maintains control in case I change my mind and want to revert etc. For long documents and projects this gets unwieldy pretty fast and I end up with lots of typos etc. I guess we could get really funky and add a regular expressions engine using lua: \definesubstitution[regex]{<string1>}{<string2>} And I'm sure Hans, Taco, and other programmers can think of other nifty macros... But the basic functionality would suffice for now :-) Best wishes Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523

(I know this an old thread, but I just happen to exactly the same feature Idris was asking for) On Thu, Oct 09, 2008 at 12:53:47AM +0200, Mojca Miklavec wrote:

usage: ruby scriptname.rb < input.tex > output.tex

#!/usr/bin/env ruby

subs_hash = { "Mohamed" => "Muḥammad", "Idris" => "ادريس", }

while line = STDIN.gets subs_hash.each do |key,value| line.gsub!(Regexp.new(key), value) end puts line end

Such approach has a major disadvantage, one have to take care of not altering macros and tex commands or we may end with a broken document (I have more general use case where the to be substituted strings can occur in macros as well), and we know that "only TeX can read TeX" :) I'm thinking in lua callback that get fed with text buffers in a similar way to OTPs, so one can use regular expressions to pre-process the text without messing with node list, think in code that replaces 2nd with 2\high{nd} etc. which would be very complex to be done at node list level. Regards, Khaled -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

On Sat, Mar 28, 2009 at 07:27:07PM -0600, Idris Samawi Hamid ادريس سماوي حامد wrote:

On Fri, 27 Mar 2009 06:05:03 -0600, Khaled Hosny wrote: If it's not so complex, have you considered overloading font features with fea files? Now that Adobe has (finally!) completed a major overhaul of the spec, including the features neeeded for arabic, that's an option.

Yes, it is an option, but not for complex operations as you know, and also font dependant (not all fonts use Adobe glyph names etc.)

But for your more complex needs, see the m-translate module. Maybe that can be developed further? Note that Hans has pointed out that this is NOT a core module and that it is preferable to fix the source beforehand.

That module shows exactly the problem I mentioned in my earlier message, it processes the whole input file not only the text buffers that will be actually typesetted, so If I've an \Idris macro it'll get replaced as well, or I've to parse the file myself which isn't really an option. OTP fits nicely in this, but it isn't as powerful as lua, and there seem not to be much interest in maintaining it in the future. I think, being able to process the to-be-typesetted text in lua is desirable and have many use cases. Now we have callbacks for input buffer, which is not very handy when you need to handle only the textual material without messing with TeX commands and macros, and callbacks for token and node list, which make very complex what is otherwise simple task, like 1st -> 1\high{st} conversion, etc. Regards, Khaled -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer

On Tue, 07 Oct 2008 04:53:40 -0600, Mojca Miklavec wrote:

On Mon, Oct 6, 2008 at 1:57 AM, Idris Samawi Hamid ادريس سماوي حامد

...

Sometimes I use the following for simple substitutions:

\defineactivecharacter ' {\otfchar{quoteright}}

But is there a more general mechanism to do things

\definesubstitution{<string1>}{<string2>}

eg

\definesubstitution{--}{–}

Of course ConTeXt already provides this particular substitution

In mkii it doesn't. Fonts do that substitution. In mkiv a hack is applied to fonts (not to TeX macros), so that this particular substitution works.

Hmm, so this is much more complicated than I thought...

...

but I'm interested in such a mechanism for more general purposes.

In mkii this is not possible, except with some ugly hacks. You can easily substitute a single character by making it active. To substitute whole words, you either need to modify fonts, or write some dirty macros.

I see. In any case, mkii is "obsolete" for my purposes, if/when I get to writing a book on ConTeXt it will cover mkiv only.

In mkiv you can either apply some "patches" to fonts (search for tlig in ConTeXt source, for example font-otf.lua, also, there are some fea files in fonts/fea/context capable of doing that), or change input text while reading/digesting some TeX file.

tlig was buggy, but it works in the latest beta. Thank you, Hans!

Hans and Taco can probably tell you more.

And Thank you, Mojca! Idris -- Professor Idris Samawi Hamid, Editor-in-Chief International Journal of Shi`i Studies Department of Philosophy Colorado State University Fort Collins, CO 80523