On 10/12/2013 12:19 PM, Khaled Hosny wrote:
The problem is that OpenType is hard, you already know that. ConTeXt will never be able to dedicate enough resources to catch up with development, so it makes much sense to reuse the efforts of other free software projects. HarfBuzz is used by much more software projects than what XeTeX was using before (Android, Mozilla, Chrome, LibreOffice, Pango, EFL, to name few), so it is here to stay. That being said, the
I bet that previous libs and whatever also have that impression .. well, in the end I think that open type will fade away too into something else as its design is sort of a compromise. The dev cycles get smaller each year, the claims for 'being the final solution' get more, who can predict ...
switch in XeTeX did not affect user documents that much (apart from fixing bugs and supporting more OpenType feature, but so does ConTeXt all the time). However, I’m not proposing that LuaTeX be dependant on HarfBuzz or FreeType, but have an optional font loader and shaper that need not even be managed by LuaTeX team.
once we have a more or less standardized lib loader (the swiglib project) one can use such libraries, i.e. there is no need to have something more in the luatex core; after all, it all boils down to passing info around; anything hard plugged into to core (even options) will be hard to fight if one wants something else at that point i can look into for instance freetype and see if a better loader can be made, who knows ... for me loading and shaping are different things
Fonts change, font formats evolve, Knuth-style stability is not really achievable, unless one freezes the source code and the fonts forever, and you can do this with external libraries, too; TeX Live is self-contained, just take a snapshot and freeze it forever, and it should be buildable as long as there are C(++) compilers.
well, practice ... one also needs to freeze the operating system then but I'm not claiming that long term stability; there was a time that tex was a big system, but nowadays the tds tree is relatively small; unless something magic happens, i think that at some point the complexity of all these big things will explode (one can according to the evangelists get a ruby on rails app up and running in minutes ... but try to update one a few years later) ... with respect to tex: the source tree/build is not trivial (how many folks know all ins-and-outs?) and it makes me already feel quite dependent .. it's the instability of the whole eco system that bothers me well, the formats don't evolve that much, at least not with respect to what we need in tex ... most features are rather generic, but tex user demands evolve and those will always influence matters; also, in the (context) machinery at some point i want to play with other approaches and then the only thing that matters is having data available (we already have some par optimizing code in place for instance) (i'm more worried about inconsistencies and a mess in fonts than in the opentype standard ... many characters / scripts / languages have well properties, so in fact designers could do with predefined sets of features and rules ... sort of the reverse of making shapes and then the features: instead of for each font reinventing the wheel, choose a set of logic, make shapes etc ... positioning is probably most of the issue then; but that's another disucssion)
There are already few parts of OpenType that I’m not able to use in my fonts for years because they break the fonts with LuaTeX horribly. I understand the priorities of the team, that is why I think that offloading font support is beneficial to everyone.
break in what sense? what features (just curious) ... anyway, there's always xetex as alternative -) (unless you're referring to wanting to use the microsoft word math renderer trickery -)
One problem of moving to for instance freetype is that we then need to get at least the same kind of data structures (ok, one could interface compatible using lua, but then I'd end up in a rewrite again, and it makes no sense to spend my live rewriting every few years).
I think that is small price to pay for the gains of not having to worry about every minute detail of OpenType support. Besides doing the shaping with HarfBuzz means that many of the structures exposed by the current font loader will not be even needed, which simplifies things greatly.
it all depends on what one wants to do with fonts, what one wants to control, etc but as said, at some point one can consider (either or not as alternative to a lua based variant) to push node list data (in an already existing callback) to a library and get something back .. of course one then also need to take care of possible interferences (assuming that the backend can load the relevant graphic data from the font)
In the end I'd still need to construct and cache lua tables in order to be able to do the same (and more). A nice thing about the current fontforge libs (which taco wants to isolate as independent lib too) is that we also have a open source editor that has a similar structure / view on the font. And that was a delibarate choice!
That is nice to have indeed, but for me having a well tested and robust font loader and feature-full OpenType engine are much more important.
but that will then be a matter of offloading to a wrapper around those libs, won't it? and, if i would use that (who knows) i'd definitely do it in a very controllable way so that it integrates nicely with other alternatives (we have 'base' mode and 'node' mode and that could be 'external' mode for instance), after all, in the end the only thing luatex itself needs is a reference to a slot in a font
Concerning a shaper, it would demand list deconstruction and construction, working within the constaints of the shaper that normally operates on different data structures than node lists, where (i guess) it becomes pretty hard then to do intermediate steps, mess around a bit, bypass etc etc. while in luatex we have a more or less consistent view on matters (read: lists). So, in the end one ends up with a patched shaper to deal with some matters which then defeats the purpose.
I don’t quit get this, but I think lots of what ConTeXt does can be done with HarfBuzz or after it is done with the shaping.
maybe, but if i'd have to work within similar constraints as with the typeone/bitmap machinery, i'd probably quickly loose interest tex, because playing with and experimenting with fonts is part of the game
If we'd have chose a shaper a few years ago, would we switch now? And in 5 years? And in 10?
Pango was using what is now called HarfBuzz continually since 2001…
Of course, when we have the swiglib interface ready (sometime next year), one can always load a library and mess with it, but on purpose we keep libraries outside the core then. So, in that case one could serialize a node list to wherever and push back the result. Of course that could not play well with existing (mixed) font models but that is then up to the user.
That is pretty much what I want, except that I want to be able to use that with ConTeXt or it will be pretty much useless for me (I still get bad dreams for having to use LaTeX to write a font manual because I can’t use XeTeX with ConTeXt and the font would often just break ConTeXt MkIV).
hm, in what sense? I don't claim that all can be done and indeed maybe for some applications offloading makes sense, but I'm pretty sure that there are also cases (and kind of docs) that would work out better with an integrated approach (fonts and features are just one aspect) so there will never be a perfect solution for everything (can also be read as: don't use tex for everything)
One important aspect is that when using tex, one also wants flexibility and control and so a font system will have hooks and such. Using a mixed library / lua / tex approach would become pretty complex. Sure, the current lua code is not trivial either but can probably be simplified over time once settled down. One reason for me not to use xetex (plus its libs) is inflexibility. If we hadn't started luatex I might as well have opt out of tex altogether, especially because only with luatex i can do some of our projects within acceotable bounds (dev time, speed of machinery, flexibility).
I don’t think much of LuaTeX’s flexibility (or XeTeX’s inflexibility) is related to the OpenType layout engine used.
I can imagine cases where things happen before shaping that have to be taken into account when shaping, and also that shaping signals things to be done later ... once we start thinking out of the box ... but, i don't see a problem in having multiple shapers (dozens) as long as (in context) they can cooperate, be isolated, controlled, etc. My main point is that I don't like a hard coded choice in luatex (also performance wise). The current core components still resemble tex. But at some points libraries will at least provide a way to play with all that is around (and undoubtely to come ... like plugging in an html rendered). Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------