On Oct 11, 2013, at 1:55 PM, luigi scarso wrote:

On Fri, Oct 11, 2013 at 10:59 AM, luigi scarso wrote:

On Fri, Oct 11, 2013 at 5:35 AM, Thangalin wrote: Hi,

A font causes mtxrun to hang upon reloading. Replicate:

mkdir $HOME/.fonts cd $HOME/.fonts wget http://whitemagicsoftware.com/Copperplate-ThirtyThreeBC.ttf export OSFONTDIR=$HOME/.fonts mtxrun --script fonts --reload

Expected Results

All the fonts in $HOME/.fonts and subdirectories are globbed and imported, within a few seconds.

Actual Results

The program hangs at Copperplate 33BC font with the CPU locked at 100%. The process must be killed.

Can you check the fotn with fontforge ?

also ttx (under linux) should shows some informations

Font Book (Apple) says this font has five serious errors and should not be used.

On 10/11/2013 2:02 PM, Taco Hoekwater wrote:

On Oct 11, 2013, at 1:55 PM, luigi scarso wrote:

...

On Fri, Oct 11, 2013 at 10:59 AM, luigi scarso wrote:

On Fri, Oct 11, 2013 at 5:35 AM, Thangalin wrote: Hi,

A font causes mtxrun to hang upon reloading. Replicate:

mkdir $HOME/.fonts cd $HOME/.fonts wget http://whitemagicsoftware.com/Copperplate-ThirtyThreeBC.ttf export OSFONTDIR=$HOME/.fonts mtxrun --script fonts --reload

Expected Results

All the fonts in $HOME/.fonts and subdirectories are globbed and imported, within a few seconds.

Actual Results

The program hangs at Copperplate 33BC font with the CPU locked at 100%. The process must be killed.

Can you check the fotn with fontforge ?

also ttx (under linux) should shows some informations

Font Book (Apple) says this font has five serious errors and should not be used.

it has no hyphen so it's rather useless anyway ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

·········

FYI,

I didn't know that the font was corrupt. I have over 400 fonts on my machine, and had to reload to pick up the new ones. You'll find that having a large number of fonts will start to become a common situation as more fonts become public (e.g., Google's Free Web Fonts project). This increases the chances that some of those fonts will be corrupt.

It is a low priority, but a corrupt font should not cause a program to hang or crash. The software should exit gracefully, and perhaps strongly suggest to standard error that the user investigate that particular font file.

+1

For my particular application, eventually I will allow users to upload custom fonts. If they upload a font that is corrupt (accidentally or intentionally), a run-away process would make for a rainy day.

That’s a serious problem with Luatex. The fontforge libraries used to import font data simply hang with various corrupt fonts. So far the only viable workaround is to manually create a blacklist of bad fonts. E.g. this is the current list for Luaotfload: https://github.com/lualatex/luaotfload/blob/master/luaotfload-blacklist.cnf (Copperplate is going to be added soon.) Unfortunately, Context does not yet have blacklisting functionality (it’s marked as todo in the source) so you’re going to have to filter out bad files from your font directories by hand. Regards, Philipp

On 10/12/2013 12:48 AM, Thangalin wrote:

Hi

...

(Copperplate is going to be added soon.) Unfortunately, Context

Keep in mind it was only Copperplate 33 BC. Also note that I could not find any version of Copperplate 33 BC online that had the same file size as my corrupt version. (I was trying to find the source of the corrupt copy.)

Most other copies, I'd imagine, are fine, so don't be too quick to blacklist it.

indeed, and when you don't notice that it's blacklisted, it can happen that this one instance gets unnoticed

...

does not yet have blacklisting functionality (it’s marked as todo in the source) so you’re going to have to filter out bad files from your font directories by hand.

Sounds like the real solution is to fix fontforge so that it doesn't hang.

sure, although a crash has the nice advantage of knowing that a font (collection) is crap (which i then can blacklist permanently in my mind) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On 10/12/2013 2:15 AM, Philipp Gesang wrote:

·········

...

On 10/12/2013 12:48 AM, Thangalin wrote:

...

Hi

...

(Copperplate is going to be added soon.) Unfortunately, Context

Keep in mind it was only Copperplate 33 BC. Also note that I could not find any version of Copperplate 33 BC online that had the same file size as my corrupt version. (I was trying to find the source of the corrupt copy.)

Most other copies, I'd imagine, are fine, so don't be too quick to blacklist it.

indeed, and when you don't notice that it's blacklisted, it can happen that this one instance gets unnoticed

...

...

does not yet have blacklisting functionality (it’s marked as todo in the source) so you’re going to have to filter out bad files from your font directories by hand.

Sounds like the real solution is to fix fontforge so that it doesn't hang.

sure, although a crash has the nice advantage of knowing that a font (collection) is crap (which i then can blacklist permanently in my mind)

Sure, but there’s a difference between a crash and a freeze. The latter can be quite annoying for those who work with strange editors that run TeX somewhere in the background making it impossible to kill the process using Ctrl-C.

it depends what causes the freeze, for instance if there is a circular reference someplace, then that is hard to catch unless one uses timeouts which in themselves are tricky (not much different from browsers locking up on some javascript); keep in mind that we load a whole font, while other applications might do a partial load and never see the problematic data (maybe even ignore portions of the font) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On 10/12/2013 9:27 AM, Khaled Hosny wrote:

On Sat, Oct 12, 2013 at 02:19:16AM +0200, Hans Hagen wrote:

...

On 10/12/2013 2:15 AM, Philipp Gesang wrote:

...

·········

...

On 10/12/2013 12:48 AM, Thangalin wrote:

...

Hi

...

(Copperplate is going to be added soon.) Unfortunately, Context

Keep in mind it was only Copperplate 33 BC. Also note that I could not find any version of Copperplate 33 BC online that had the same file size as my corrupt version. (I was trying to find the source of the corrupt copy.)

Most other copies, I'd imagine, are fine, so don't be too quick to blacklist it.

indeed, and when you don't notice that it's blacklisted, it can happen that this one instance gets unnoticed

...

...

does not yet have blacklisting functionality (it’s marked as todo in the source) so you’re going to have to filter out bad files from your font directories by hand.

Sounds like the real solution is to fix fontforge so that it doesn't hang.

sure, although a crash has the nice advantage of knowing that a font (collection) is crap (which i then can blacklist permanently in my mind)

Sure, but there’s a difference between a crash and a freeze. The latter can be quite annoying for those who work with strange editors that run TeX somewhere in the background making it impossible to kill the process using Ctrl-C.

it depends what causes the freeze, for instance if there is a circular reference someplace, then that is hard to catch unless one uses timeouts which in themselves are tricky (not much different from browsers locking up on some javascript); keep in mind that we load a whole font, while other applications might do a partial load and never see the problematic data (maybe even ignore portions of the font)

Which is something we ought to do, serializing the whole font to a lua table is problematic in many ways (too slow, takes much memory, etc) while SFNT fonts are designed in such a way that you can go directly to the part you just want. And FontForge is not that robust (and it is not a font loading library after all).

The whole font is not loaded, unless one does the to_table of the font table, but normally it goes stepwise. Also, once the font data is at the lua end it's fast and efficient to use it while any interfacing to a library will be much slower (crossing the lua-c boundary) and (at least for me a no-go given what things i hav ein mind).

I have been dreaming for a while of making an optional font loader for LuaTeX using mature font libraries, e.g. FreeType for loading fonts, HarfBuzz for shaping, may be FriBiDi (not a priority, BiDi in Lua is not hard) and even FontConfig (when available) for searching system fonts. But no much time unfortunately, and the fear that I wouldn't be able to use it with ConTeXt is not that motivating. Someone is, however, experimenting with such a thing: http://www.readytext.co.uk/?p=3143

Personally I think that this will defeat the purpose we had in mind with luatex: small basic tex kernel and everything else in extensible lua, not too far off what classical tex had as purpose: long term stability. We already have xetex that uses libraries as alternative. (Although the fact that xetex changed libs is proof that libraries introduce a dependency and possible stability factor on the long run.) I also think that people who want to use all those libraries are better off with other tools than tex. We already have a dependency in xpdf which is a moving target too and has seen changes. The fontforge code was deliberately included to be independent. Okay, we have a few dependencies in luatex, like lfs, png, jpg, xpdf but that's somewhat limited. In the end we hope to have the backend at least properly isolated. One problem of moving to for instance freetype is that we then need to get at least the same kind of data structures (ok, one could interface compatible using lua, but then I'd end up in a rewrite again, and it makes no sense to spend my live rewriting every few years). In the end I'd still need to construct and cache lua tables in order to be able to do the same (and more). A nice thing about the current fontforge libs (which taco wants to isolate as independent lib too) is that we also have a open source editor that has a similar structure / view on the font. And that was a delibarate choice! Concerning a shaper, it would demand list deconstruction and construction, working within the constaints of the shaper that normally operates on different data structures than node lists, where (i guess) it becomes pretty hard then to do intermediate steps, mess around a bit, bypass etc etc. while in luatex we have a more or less consistent view on matters (read: lists). So, in the end one ends up with a patched shaper to deal with some matters which then defeats the purpose. If we'd have chose a shaper a few years ago, would we switch now? And in 5 years? And in 10? Of course, when we have the swiglib interface ready (sometime next year), one can always load a library and mess with it, but on purpose we keep libraries outside the core then. So, in that case one could serialize a node list to wherever and push back the result. Of course that could not play well with existing (mixed) font models but that is then up to the user. One important aspect is that when using tex, one also wants flexibility and control and so a font system will have hooks and such. Using a mixed library / lua / tex approach would become pretty complex. Sure, the current lua code is not trivial either but can probably be simplified over time once settled down. One reason for me not to use xetex (plus its libs) is inflexibility. If we hadn't started luatex I might as well have opt out of tex altogether, especially because only with luatex i can do some of our projects within acceotable bounds (dev time, speed of machinery, flexibility). Talking of dreams: mine is to have a real clean and independent luatex, not dependent on large third party libraries, only lua + a few small libs maintained as part of luatex, in a simple source tree that can be compiled with a simple setup. I've seen too many open source projects divert, change, abort, etc to believe in long term support (which is why i get more and more careful in what i choose to use). Just look how often browsers change policies, components, etc. Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On 10/12/2013 12:19 PM, Khaled Hosny wrote:

The problem is that OpenType is hard, you already know that. ConTeXt will never be able to dedicate enough resources to catch up with development, so it makes much sense to reuse the efforts of other free software projects. HarfBuzz is used by much more software projects than what XeTeX was using before (Android, Mozilla, Chrome, LibreOffice, Pango, EFL, to name few), so it is here to stay. That being said, the

I bet that previous libs and whatever also have that impression .. well, in the end I think that open type will fade away too into something else as its design is sort of a compromise. The dev cycles get smaller each year, the claims for 'being the final solution' get more, who can predict ...

switch in XeTeX did not affect user documents that much (apart from fixing bugs and supporting more OpenType feature, but so does ConTeXt all the time). However, I’m not proposing that LuaTeX be dependant on HarfBuzz or FreeType, but have an optional font loader and shaper that need not even be managed by LuaTeX team.

once we have a more or less standardized lib loader (the swiglib project) one can use such libraries, i.e. there is no need to have something more in the luatex core; after all, it all boils down to passing info around; anything hard plugged into to core (even options) will be hard to fight if one wants something else at that point i can look into for instance freetype and see if a better loader can be made, who knows ... for me loading and shaping are different things

Fonts change, font formats evolve, Knuth-style stability is not really achievable, unless one freezes the source code and the fonts forever, and you can do this with external libraries, too; TeX Live is self-contained, just take a snapshot and freeze it forever, and it should be buildable as long as there are C(++) compilers.

well, practice ... one also needs to freeze the operating system then but I'm not claiming that long term stability; there was a time that tex was a big system, but nowadays the tds tree is relatively small; unless something magic happens, i think that at some point the complexity of all these big things will explode (one can according to the evangelists get a ruby on rails app up and running in minutes ... but try to update one a few years later) ... with respect to tex: the source tree/build is not trivial (how many folks know all ins-and-outs?) and it makes me already feel quite dependent .. it's the instability of the whole eco system that bothers me well, the formats don't evolve that much, at least not with respect to what we need in tex ... most features are rather generic, but tex user demands evolve and those will always influence matters; also, in the (context) machinery at some point i want to play with other approaches and then the only thing that matters is having data available (we already have some par optimizing code in place for instance) (i'm more worried about inconsistencies and a mess in fonts than in the opentype standard ... many characters / scripts / languages have well properties, so in fact designers could do with predefined sets of features and rules ... sort of the reverse of making shapes and then the features: instead of for each font reinventing the wheel, choose a set of logic, make shapes etc ... positioning is probably most of the issue then; but that's another disucssion)

There are already few parts of OpenType that I’m not able to use in my fonts for years because they break the fonts with LuaTeX horribly. I understand the priorities of the team, that is why I think that offloading font support is beneficial to everyone.

break in what sense? what features (just curious) ... anyway, there's always xetex as alternative -) (unless you're referring to wanting to use the microsoft word math renderer trickery -)

...

One problem of moving to for instance freetype is that we then need to get at least the same kind of data structures (ok, one could interface compatible using lua, but then I'd end up in a rewrite again, and it makes no sense to spend my live rewriting every few years).

I think that is small price to pay for the gains of not having to worry about every minute detail of OpenType support. Besides doing the shaping with HarfBuzz means that many of the structures exposed by the current font loader will not be even needed, which simplifies things greatly.

it all depends on what one wants to do with fonts, what one wants to control, etc but as said, at some point one can consider (either or not as alternative to a lua based variant) to push node list data (in an already existing callback) to a library and get something back .. of course one then also need to take care of possible interferences (assuming that the backend can load the relevant graphic data from the font)

...

In the end I'd still need to construct and cache lua tables in order to be able to do the same (and more). A nice thing about the current fontforge libs (which taco wants to isolate as independent lib too) is that we also have a open source editor that has a similar structure / view on the font. And that was a delibarate choice!

That is nice to have indeed, but for me having a well tested and robust font loader and feature-full OpenType engine are much more important.

but that will then be a matter of offloading to a wrapper around those libs, won't it? and, if i would use that (who knows) i'd definitely do it in a very controllable way so that it integrates nicely with other alternatives (we have 'base' mode and 'node' mode and that could be 'external' mode for instance), after all, in the end the only thing luatex itself needs is a reference to a slot in a font

...

Concerning a shaper, it would demand list deconstruction and construction, working within the constaints of the shaper that normally operates on different data structures than node lists, where (i guess) it becomes pretty hard then to do intermediate steps, mess around a bit, bypass etc etc. while in luatex we have a more or less consistent view on matters (read: lists). So, in the end one ends up with a patched shaper to deal with some matters which then defeats the purpose.

I don’t quit get this, but I think lots of what ConTeXt does can be done with HarfBuzz or after it is done with the shaping.

maybe, but if i'd have to work within similar constraints as with the typeone/bitmap machinery, i'd probably quickly loose interest tex, because playing with and experimenting with fonts is part of the game

...

If we'd have chose a shaper a few years ago, would we switch now? And in 5 years? And in 10?

Pango was using what is now called HarfBuzz continually since 2001…

...

Of course, when we have the swiglib interface ready (sometime next year), one can always load a library and mess with it, but on purpose we keep libraries outside the core then. So, in that case one could serialize a node list to wherever and push back the result. Of course that could not play well with existing (mixed) font models but that is then up to the user.

That is pretty much what I want, except that I want to be able to use that with ConTeXt or it will be pretty much useless for me (I still get bad dreams for having to use LaTeX to write a font manual because I can’t use XeTeX with ConTeXt and the font would often just break ConTeXt MkIV).

hm, in what sense? I don't claim that all can be done and indeed maybe for some applications offloading makes sense, but I'm pretty sure that there are also cases (and kind of docs) that would work out better with an integrated approach (fonts and features are just one aspect) so there will never be a perfect solution for everything (can also be read as: don't use tex for everything)

...

One important aspect is that when using tex, one also wants flexibility and control and so a font system will have hooks and such. Using a mixed library / lua / tex approach would become pretty complex. Sure, the current lua code is not trivial either but can probably be simplified over time once settled down. One reason for me not to use xetex (plus its libs) is inflexibility. If we hadn't started luatex I might as well have opt out of tex altogether, especially because only with luatex i can do some of our projects within acceotable bounds (dev time, speed of machinery, flexibility).

I don’t think much of LuaTeX’s flexibility (or XeTeX’s inflexibility) is related to the OpenType layout engine used.

I can imagine cases where things happen before shaping that have to be taken into account when shaping, and also that shaping signals things to be done later ... once we start thinking out of the box ... but, i don't see a problem in having multiple shapers (dozens) as long as (in context) they can cooperate, be isolated, controlled, etc. My main point is that I don't like a hard coded choice in luatex (also performance wise). The current core components still resemble tex. But at some points libraries will at least provide a way to play with all that is around (and undoubtely to come ... like plugging in an html rendered). Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On 10/13/2013 10:15 AM, Khaled Hosny wrote:

On 10/12/2013 12:19 PM, Khaled Hosny wrote:

...

The problem is that OpenType is hard, you already know that. ConTeXt will never be able to dedicate enough resources to catch up with development, so it makes much sense to reuse the efforts of other free software projects. HarfBuzz is used by much more software projects than what XeTeX was using before (Android, Mozilla, Chrome, LibreOffice, Pango, EFL, to name few), so it is here to stay. That being said, the

I bet that previous libs and whatever also have that impression ..

I don’t think so, ICU Layout Engine was only used by Java, OpenOffice and XeTeX (as far as free software is concerned), and even for XeTeX it was not adequate and had to be patched. This is different from HarfBuzz, which is a mature piece of code that have been developed and used over a long time and is seeing a wide adoption, my impression is that HarfBuzz is here to stay, just like FreeType; so many people are investing in it.

seen next remark

...

well, in the end I think that open type will fade away too into something else as its design is sort of a compromise. The dev cycles get smaller each year, the claims for 'being the final solution' get more, who can predict ...

I doubt that OpenType is going anywhere, not matter how bad it is, the industry has invested so much into it and switching to any completely new solution will be prohibitively expensive.

I'm around long enough to know that this is a dream. What makes us think that solutions (and the political entities that harbour them) we cook up today are the best and stay forever? I'm pretty sure that many though that about paper books. Just look at what came up the last few hundred years ago and faded away. Look at empires. We can only hope that content can be transferred to new 'standards'. It helps to watch the movie about the linotype machinery: a big invention at that time, claimed to have transformed the world due to more info carried around, but all hardware trashed within a few years. So, what will happen when quantum computing combined with some AI shows up? All our claimed perfect and persistent technologies will disappear within a moment. So ... 'anywhere' == "today + a few years".

...

...

switch in XeTeX did not affect user documents that much (apart from fixing bugs and supporting more OpenType feature, but so does ConTeXt all the time). However, I’m not proposing that LuaTeX be dependant on HarfBuzz or FreeType, but have an optional font loader and shaper that need not even be managed by LuaTeX team.

once we have a more or less standardized lib loader (the swiglib project) one can use such libraries, i.e. there is no need to have something more in the luatex core; after all, it all boils down to passing info around; anything hard plugged into to core (even options) will be hard to fight if one wants something else

An external module is fine by me, I’m not concerned about LuaTeX itself since even the current loader is not integrated, I’m rather concerned about the ability to use the new loader and shaper with ConTeXt.

In principle one can plug in anything if it behaves nicely, but of course one cannot expect existing mechanisms (we have additional features and such) to be dropped in favor of something else without pretty good reason. Btw, for me (that is personal) i think that it's not bad to have an alternative shaper like we have in context if only because a standard (like opentype) is no standard if there is only one (to be used) implementation. Also, keep in mind that one can have similar arguments for all components of luatex: - use javascript instead of lua (if we'd started 15 years ago it would have via perl and python and ruby so if would mean yet another change) - use ghostscript as backend - use graphicmagic for loading graphics - use some html renderer (which one to choose) for rendering math and tables and eventually: move on to english only so that we can also drop complex scripts and stick to ascii latin (thinking of it: scripts and such, including math, are the main reason that tex has survived and that only happened because we could mess with fonts independent from already faded out other approaches and programs ... ok, in special ways, but still the messing around meant survival)

...

at that point i can look into for instance freetype and see if a better loader can be made, who knows ... for me loading and shaping are different things

...

Fonts change, font formats evolve, Knuth-style stability is not really achievable, unless one freezes the source code and the fonts forever, and you can do this with external libraries, too; TeX Live is self-contained, just take a snapshot and freeze it forever, and it should be buildable as long as there are C(++) compilers.

well, practice ... one also needs to freeze the operating system then

but I'm not claiming that long term stability; there was a time that tex was a big system, but nowadays the tds tree is relatively small; unless something magic happens, i think that at some point the complexity of all these big things will explode (one can according to the evangelists get a ruby on rails app up and running in minutes ... but try to update one a few years later) ... with respect to tex: the source tree/build is not trivial (how many folks know all ins-and-outs?) and it makes me already feel quite dependent .. it's the instability of the whole eco system that bothers me

well, the formats don't evolve that much, at least not with respect to what we need in tex ... most features are rather generic, but tex user demands evolve and those will always influence matters; also, in the (context) machinery at some point i want to play with other approaches and then the only thing that matters is having data available (we already have some par optimizing code in place for instance)

(i'm more worried about inconsistencies and a mess in fonts than in the opentype standard ... many characters / scripts / languages have well properties, so in fact designers could do with predefined sets of features and rules ... sort of the reverse of making shapes and then the features: instead of for each font reinventing the wheel, choose a set of logic, make shapes etc ... positioning is probably most of the issue then; but that's another disucssion)

...

There are already few parts of OpenType that I’m not able to use in my fonts for years because they break the fonts with LuaTeX horribly. I understand the priorities of the team, that is why I think that offloading font support is beneficial to everyone.

break in what sense? what features (just curious) ... anyway,

The mark filtering sets we discussed few years ago, for instance; I had to redo the whole font in a much more complex way with few more thousands of glyphs to avoid using them so the font remains usable with ConTeXt, and I still get crashes every now and then that I stopped bothering with reporting them.

Well, I can't fix something if I don't know about it ... one thing I learned the last years, is that one needs examples of usage (and I'm pretty sure that some of the implementation of an opentype engine is not a direct consequence if the standards but of usage). Teaser: if another app doesn't crash, it doesn't mean that it does it right. But its behaviour could become the standard expected. I'm not saying that this is true in your case, but we need to keep this option open. Also, when something is non trivial (and I've had enough discussions to know that the spec is not always that clear) one might wonder about the spec in the first place ... trial and error (due to lack of fonts and examples) is part of opentype implementation development (ok, for much other dev too). Which reminds me: on the agenda is to have a few more fields in glyph nodes so that we can simplify some of the code, but that's a something we will look into end-of-year when some other backend cleanup has happened. I still remember a talk at bachotex where someone went through all version of indesign pointing out in which ones ligatures did / didn't work, workes differently. And look at opentype math ... some reverse engineering is needed. To me opentype is not a real standard but m ore reversed application specification where we have to look at what the inventors do and try to adapt to that.

...

there's always xetex as alternative -)

Except I can’t use it :) even with MkII as there is zero support for XeTeX’s (rudimentary) RTL model.

...

(unless you're referring to wanting to use the microsoft word math renderer trickery -)

I never set math actually :) unless I’m testing a math font, but I can use plain for that.

...

...

...

One important aspect is that when using tex, one also wants flexibility and control and so a font system will have hooks and such. Using a mixed library / lua / tex approach would become pretty complex. Sure, the current lua code is not trivial either but can probably be simplified over time once settled down. One reason for me not to use xetex (plus its libs) is inflexibility. If we hadn't started luatex I might as well have opt out of tex altogether, especially because only with luatex i can do some of our projects within acceotable bounds (dev time, speed of machinery, flexibility).

I don’t think much of LuaTeX’s flexibility (or XeTeX’s inflexibility) is related to the OpenType layout engine used.

I can imagine cases where things happen before shaping that have to be taken into account when shaping, and also that shaping signals things to be done later ... once we start thinking out of the box ... but, i don't see a problem in having multiple shapers (dozens) as long as (in context) they can cooperate, be isolated, controlled, etc. My main point is that I don't like a hard coded choice in luatex (also performance wise). The current core components still resemble tex.

I absolutely agree. All I wanted to know is whether ConTeXt can consider integrating an optional different font machinery, so to not waste time on something that will be of little use to me.

Regards, Khaled ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : http://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

-- ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On 10/12/2013 12:26 AM, Philipp Gesang wrote:

·········

...

FYI,

I didn't know that the font was corrupt. I have over 400 fonts on my machine, and had to reload to pick up the new ones. You'll find that having a large number of fonts will start to become a common situation as more fonts become public (e.g., Google's Free Web Fonts project). This increases the chances that some of those fonts will be corrupt.

sure, but fonts should be chosen with care .. and illegal ripoffs should be avoided (those cd's with hundreds of fonts, nowadays sites) (quality of design / digitization, completeness of coverage, correctness of features ... much can be wrong)

...

It is a low priority, but a corrupt font should not cause a program to hang or crash. The software should exit gracefully, and perhaps strongly suggest to standard error that the user investigate that particular font file.

+1

...

For my particular application, eventually I will allow users to upload custom fonts. If they upload a font that is corrupt (accidentally or intentionally), a run-away process would make for a rainy day.

In that case I'd run a separate font checker, as you never know what users upload. Similar issues can occur with those tagged formats that are in fact linked lists.

That’s a serious problem with Luatex. The fontforge libraries used to import font data simply hang with various corrupt fonts. So far the only viable workaround is to manually create a blacklist of bad fonts. E.g. this is the current list for Luaotfload:

https://github.com/lualatex/luaotfload/blob/master/luaotfload-blacklist.cnf

(Copperplate is going to be added soon.) Unfortunately, Context does not yet have blacklisting functionality (it’s marked as todo in the source) so you’re going to have to filter out bad files from your font directories by hand.

a slippery road .. blacklisting .. this is why it's todo .. (actually in the beginning context had something like that) ... the problem is that such lists can be persistent and users don't know about them (anyhow, I enabled it in the treatments but probably am not going to waste much time on such fonts ... a font not loading is one, but in order to be complete blacklisting would also need to involve fonts with bugged encodings - and i've seen some - and other issues ... i also want to avoid a 'why doesn't this font work, it should because it's not blacklisted' kind of discussions)

Regards, Philipp

___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : http://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

-- ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On Fri, Oct 11, 2013 at 08:05:03PM -0700, Thangalin wrote:

...

In that case I'd run a separate font checker, as you never know what users upload. Similar issues can occur with those tagged formats that are in fact linked lists.

That's a good idea. The TTX font tool was going to be my first stop.

http://sourceforge.net/projects/fonttools/

I'd give Google's OTS a try, both Chrome and Firefox use it to sanitize downloadable webfonts: https://code.google.com/p/ots/ Regards, Khaled

On Sat, Oct 12, 2013 at 11:20 AM, Hans Hagen wrote:

On 10/12/2013 10:52 AM, Khaled Hosny wrote:

...

On Fri, Oct 11, 2013 at 08:05:03PM -0700, Thangalin wrote:

...

In that case I'd run a separate font checker, as you never know what

...

users upload. Similar issues can occur with those tagged formats that are in fact linked lists.

That's a good idea. The TTX font tool was going to be my first stop.

http://sourceforge.net/**projects/fonttools/http://sourceforge.net/projects/fonttools/

I'd give Google's OTS a try, both Chrome and Firefox use it to sanitize downloadable webfonts: https://code.google.com/p/ots/

interesting .. something for luigi to test (swiglib)

https://swiglib.foundry.supelec.fr/ could be into the experimental folder

-- luigi