- dev-luatex - NTG Mailing Lists

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 18 May '06

18 May '06

U luatex/src/configure U luatex/src/libs/configure U luatex/src/texk/configure U luatex/src/texk/kpathsea/configure D luatex/src/texk/web2c/pdftexdir D luatex/src/texk/web2c/depend.mak D luatex/src/texk/web2c/win32.mak U luatex/src/texk/web2c/configure U luatex/src/texk/web2c/Makefile.in A luatex/src/texk/web2c/luatexdir A luatex/src/texk/web2c/luatexdir/utils.c A luatex/src/texk/web2c/luatexdir/Makefile.in A luatex/src/texk/web2c/luatexdir/ttf2afm.c A luatex/src/texk/web2c/luatexdir/luatex.ch A luatex/src/texk/web2c/luatexdir/mapfile.c A luatex/src/texk/web2c/luatexdir/writezip.c A luatex/src/texk/web2c/luatexdir/epdf.c A luatex/src/texk/web2c/luatexdir/writeenc.c A luatex/src/texk/web2c/luatexdir/avl.c A luatex/src/texk/web2c/luatexdir/epdf.h A luatex/src/texk/web2c/luatexdir/macnames.c A luatex/src/texk/web2c/luatexdir/depend.mk A luatex/src/texk/web2c/luatexdir/luatex.mk A luatex/src/texk/web2c/luatexdir/avl.h A luatex/src/texk/web2c/luatexdir/image.h A luatex/src/texk/web2c/luatexdir/tex.ch0 A luatex/src/texk/web2c/luatexdir/luastuff.c A luatex/src/texk/web2c/luatexdir/luatexextra.in A luatex/src/texk/web2c/luatexdir/avlstuff.c A luatex/src/texk/web2c/luatexdir/NEWS A luatex/src/texk/web2c/luatexdir/writejpg.c A luatex/src/texk/web2c/luatexdir/luatex.web A luatex/src/texk/web2c/luatexdir/avlstuff.h A luatex/src/texk/web2c/luatexdir/writepng.c A luatex/src/texk/web2c/luatexdir/writet1.c A luatex/src/texk/web2c/luatexdir/pkin.c A luatex/src/texk/web2c/luatexdir/writet3.c A luatex/src/texk/web2c/luatexdir/pdftosrc.1 A luatex/src/texk/web2c/luatexdir/luatex.c A luatex/src/texk/web2c/luatexdir/vfpacket.c A luatex/src/texk/web2c/luatexdir/pdftosrc.rc A luatex/src/texk/web2c/luatexdir/ChangeLog A luatex/src/texk/web2c/luatexdir/ptexmac.h A luatex/src/texk/web2c/luatexdir/luatex.h A luatex/src/texk/web2c/luatexdir/writettf.c A luatex/src/texk/web2c/luatexdir/ptexlib.h A luatex/src/texk/web2c/luatexdir/README A luatex/src/texk/web2c/luatexdir/subfont.txt A luatex/src/texk/web2c/luatexdir/writettf.h A luatex/src/texk/web2c/luatexdir/pdftoepdf.cc A luatex/src/texk/web2c/luatexdir/luatexlib.mk A luatex/src/texk/web2c/luatexdir/luatex-api.h A luatex/src/texk/web2c/luatexdir/subfont.c A luatex/src/texk/web2c/luatexdir/writefont.c A luatex/src/texk/web2c/luatexdir/change-files.txt A luatex/src/texk/web2c/luatexdir/ttf2afm.rc A luatex/src/texk/web2c/luatexdir/pdftosrc.cc A luatex/src/texk/web2c/luatexdir/writeimg.c A luatex/src/texk/web2c/luatexdir/lua.ch A luatex/src/texk/web2c/luatexdir/luatex.defines U luatex/src/texk/web2c/configure.in U luatex/src/texk/web2c/texmfmp-help.h U luatex/src/texk/web2c/web2c/convert U luatex/src/texk/web2c/web2c/splitup.c U luatex/src/texk/web2c/web2c/coerce.h U luatex/src/texk/web2c/web2c/common.defines U luatex/src/texk/web2c/withenable.ac U luatex/src/texk/web2c/texmfmp.h D luatex/NEWS A luatex/NEWS D luatex/README A luatex/README Checked out revision 23.

1 0

Re: [Dev-luatex] Cron <luatex@server> /home/luatex/luatex-svn
by Taco Hoekwater 17 May '06

17 May '06

Hi, I have set up a script that does a checkout every hour, and that script mails to this list. Hans, can you add the sender (root(a)aanhet.net) to the 'allow' filter? Cheers, Taco Cron Daemon wrote: > U luatex/INSTALL > Checked out revision 17.

1 0

Progress
by Taco Hoekwater 17 May '06

17 May '06

Hi, I've just committed a new version of luatex that has the official lua51 library and the pdftex-1.40.0-beta-20060213. There is no new functionality except that the lua IO and OS libraries now should work as advertised. This is the first of a series of commits to come in the coming weeks. Aside from Hans' requests for functionality, I intend to change the engine & executable name to 'luatex', and embed the pool file. More about that later . Cheers, taco

1 0

next steps in development
by Hans Hagen 07 May '06

07 May '06

Hi, In order to experiment we need: (1) \luadef\cs goal: starts an lua instance usage: \luadef\lua (2) \luastring{...} goal: escapes problematic chars usage: \lua{str = "\luastring{blabla "bla"}"} (3) \immediate\luacs goal: feeds back into the token scanner (no need for scantokens, faster due to less parameter passing) usage: \immediate\lua{tex.print("$x$")} (4) tables: tex.print pdf.print (literal, so we need something pdf.print(str,optional_type) in order to handle the now keywords to \pdfliteral hartmut may have ideas on other other pdf entries (since he already has done some experiments) further, taco and i discussed a lua extension using a variant of strings using 32 bit entries, partly for the unicode chars and partly for the \cs etc pointers so that we can do stringlike thinsg with token lists well, we discussed more, but i need to search my memory -) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

2 2

[Fwd: LuaTeX]
by Taco Hoekwater 05 May '06

05 May '06

Hi, Comments are welcome before I reply. Personally I doubt much indirection is feasible, though. Taco -------- Original Message -------- Subject: LuaTeX Date: Sun, 30 Apr 2006 08:52:03 -0700 (PDT) From: James Quirk <jjq(a)galcit.caltech.edu> To: Taco Hoekwater <taco(a)elvenkind.com> Dear Taco, I understand from the LuaTeX FAQ that third-party feedback is currently not encouraged. However, I have one observation that I hope you might consider. Specifically, I would have thought it possible, and desirable, to introduce a lightweight intermediate layer to broker communications between TeX and Lua. The situation would be analagous to converting between graphics formats. Rather than writing N^2 conversion programs (a2b, a2c, b2a, b2c, c2a, c2b etc.) it is possible to get away with the 2N conversion programs (a2xyz, b2xyz, c2xyz, xyz2a, xyz2b, xyz2c etc.). The motivation being that when a new format comes along one need only construct xyz2new and new2xyz to be able to convert to any one of the other formats. So instead of having tex2lua and lua2tex communications one would have {tex,lua}2broker and broker2{tex,lua}. That way it would be be straightforward for a third-party to add their own favourite scripting language, as most of the machinations live with the broker and do not change with the choice of embedded language (at least not for languages implemented in C, which covers perl python, ruby etc.). I realise, from arguments of portablity, that you might not want to have a situation where there are 57 varieties of embedded interpreter. But the broker-design would provide a springboard for whatever language developments might occur down the road without needing to be rewritten. As for the cost of the overhead, it would not be great and with the advent of multicore processors I'm not sure it is something to worry about. Afterall, much of TeX's programming clunkiness stems from decisions that were too focussed on the machine's of the day; a mistake that crops up time and time again. In any event. I look forward to trying out LuaTeX when it is made publicly available. All the best with your project, James Quirk

2 1

new parbreaking algorthm prototype
by Karel Skoupý 03 Oct '05

03 Oct '05

Dear all, I'm sending the China version of my free-shape text breaker in case you wanna see what it does. If you really wanna compile and run it, then you need to install 2 other packages, but read till the end before you do it. (1) Objective Caml: http://caml.inria.fr/ It's a standard package in linux distros (at least in Debian), otherwise you can take binaries from the Download section at that site (2) ADvi: http://pauillac.inria.fr/advi/ My prototype links with the advi modules (v 1.6.0). Follow the download and build instructions at the site. (3) Dyntyp: dyntyp-src-050823-1121.tar.gz attached This is my prototype The actual algorithm is inside xbreak.ml. There is also break.ml which contains earlier version of the algorithm which doesn't really work. In order to build it, you need to install (1) and to untar and build (2). Then untar my package (3), preferably with the following directory layout: <SOME DIRECTORY> +--adk | +--advi-1.6.0 | +--camlimages-2.2 +--<SOME DIRECTORY> (dyntyp, but I should think of something better) +--src because (2) is referenced from (3) in Makefile.config and it expects adk in ../../adk. If you need different structure, change Makefile.config in src. Oops, I think, advi doesn't work on Windows, so the whole prototype is only for linux (unix) at the moment (but it will change soon). OK, so inside src (of (3)), do make, it should create dyntyp.opt. The shellscript show.sh runs a demo. Well, linking with ADvi was a quick hack for getting something visible. My next plan is to change it for a TeX plugin as it was announced. For that I need: * parsing of \showlists output, not very difficult * accessing tfm, it will take some time, I have to rewrite my NTS module * outputing TeX input So it will all take some time, but then it won't be dependent on ADvi and it will work under Windows too. So it makes more sense not to install ADvi and to wait few weeks. Cheers, --ksk

1 0

Karel's presentation on TUG 2005
by Karel Skoupý 29 Sep '05

29 Sep '05

Hi All, reacting to Thanh's request, I put my TUG 2005 presentation on http://www.inf.ethz.ch/personal/kaskoupy/slides/FreeShapeTextFormatting.pdf I doubt that there can be anything useful understood just from those slides. I should still write a proper paper in next few days, that should be much more clear and complete, you can safely wait for that. I'll also send you the actual prototype with the instructions how to get it running, hopefully tonight. Regards, --ksk

1 0

Re: [Dev-luatex] plugin for external formatting
by Thanh Han The 25 Sep '05

25 Sep '05

Hi, this is a long thread and too many things are discussed at the same time so I will need some time to read and understand what is going on. My first thought is that some small modifications to \showlist and \showbox will help a lot. It's easy to write additional info like dimensions of each item in the list, or in case of characters the filename of a tfm with fontsize (or we may write the dimensions of each char as Hans suggested, but this is an overkill IMHO). My feeling is that we need to work out the specification and format of the `` node list'' first. In the first step, I would prefer to have only node-specific things, eg only what comes out after a box construction. I also got a similar request: to provide a primitive that writes out the content of a box and another primitive to re-construct that box back from the output. We can start with this and make further extensions later on. At the moment I cannot see clearly what is needed, but I am willing to write some extensions so that we can experiment with to see what is really needed and perhaps change what have been done. Thanh On Tue, Sep 20, 2005 at 11:58:08AM +0200, Karel Skoupy wrote: > Hi all, > > as Hans has already mentioned, my concern with luatex is to have some > interface/protocol for formatting the TeX stuff externally. > > Without going to details now, I'm interested in alternative algorithms for > formatting not only paragraphs, but the whole stream. For TUG 2005 I have > written a prototype which doesn't use any TeX code at all (it just > parasities on ADvi code for getting some metric information and showing the > results). For long time I planned to make a whole new system from scratch, > but for several reasons, that was reconsidered and Hans proposed a way > (plugin mechanism for external engine), how to cooperate with TeX, so TeX > could benefit from the new algorithm and I can concentrate on the core > stuff. > > So basically I need a stream of (character) boxes, glues, penalties, ... > (is there a simple unambiguous notion for all that?) in a preprocessed > form (I don't care about input and macro handling) plus some parameters > (standard paragraph breaking parameters and the new special ones) and I > will return a stream of fixed boxes. > > 'I' will often mean 'the engine' depending on the context :-) > > In the first stage, I won't need lua (or any changes to TeX) at all. I > plan to use \showlists for my input stream and to generate a standard > TeX input file for reading the result back. Of course, it won't be so > simple, there will be some macro programming and trickery, which will > make the whole thing complicated, fragile, unreliable, and inefficient for > real use. Therefore some hooks from the actively developed TeX will be > probably useful for making the cooperation of TeX and the external > engine smooth. It might use lua or not, we will see, in any case I would > like to keep the plugin support generic and (complete but) minimal. > > I will now list the aspects of the communications between TeX and the > engine which I have thought of so far. I will be glad if you can just think > about it for the moment and give me some feedback if you will. > > * single paragraph stuff > > I need: > (1) complete representation of all the stuff which is to be returned formatted > (2) sizes of all the objects which are involved in formatting > (3) properties which influence the formatting (breakable, discardable, ...) > > It seems that the standard output of \showlists (or \showbox) will mostly do. > (1) is fulfilled I guess (the returned input needs to be only slightly > modified to fit TeX). > > (2) is little bit tricky, because for the characters I get only an id of > the font. So I will need to know the exact reference to a real font to get > the metrics information. This can be learned by eg. \show\tenrm. But of > course it is not know in advance what fonts are used in the paragraph, so > either all fonts can be listed at the beginning -- but where to get the list of > all font definitions, and the definitions can actually change in the middle > of the paragraph -- or I can make a first pass, collect the font ids and > ask for them in the second pass. It will be bit tricky and won't be > reliable due to redefinitions (I can also change the current id using \let > and lose the old id (still used in the log), right?), so it will be OK for > experimenting but for a real version, I will need a better support from > TeX. > > (3) is implicit, right? > > * stream of paragraphs > > I can need even the whole chapter, because I want to treat > - shapes and layouts, which are relative to page and not to a particular > paragraph > - pagination, floats placement > > For the basic experimenting I can redefine \par to something like > \hfil\break\indent but it will restrict all kinds of things which can > happen between the paragraphs (in vertical mode). Of course, the whole > thing will never be compatible to TeX, because TeX expect after \par that > the last paragraph was formatted and placed on the vertical list. So it > will be responsibility of the user/macro-programmer to bear the > consequences of using the alternative mechanism. Nevertheless, the > consequences should be as small as possible. > > So for the prototyping I can redefine \par or perhaps I can store the whole > paragraphs in infinite hboxes (redefining \hsize?) or maybe I can use some > \specials for tagging, but for the production version, this will be a very > tricky part. Not so much for the engine, but mainly on the TeX side. It > should be of a great concern for people who would want to use the new > algorithms in their systems (Hans?), (after those ideas are first tested by > a prototype :-). > > * passing the parameters specific to the new algorithms > > - layouts, shapes > - maybe others, like weights for resolving paragraph contra page breaking > > This will be a new thing so I hope that there is no compatibility burden. > > * hyphenation > > It will be a lot of additional work, but I think that I should handle it > locally. There are two reasons: > > (1) the protocol for failing and getting the list with new discretionaries > (TeX's 2nd pass) for every individual paragraph would be extremely > complicated, in the end it might be more difficult than handling it locally. > > (2) TeX's hyphenation mechanism is IMHO one of the crappiest parts of TeX. > I mean the way how the (non)ligatures are screwed up for discretionaries > which are not used in the end. So if it is handled locally, it will be IMO > simpler and more correct. There are also some research results concerning > hyphenation, which are not implemented in TeX, because it would be too > complicated. > > At the first stage, I'll omit the hyphenation completelly. > > At the moment, I don't remember anything else. I'm looking forward for your > feedback. > > --ksk > _______________________________________________ > Dev-luatex mailing list > Dev-luatex(a)ntg.nl > http://www.ntg.nl/mailman/listinfo/dev-luatex >

3 3

Re: [Dev-luatex] plugin for external formatting
by Thanh Han The 23 Sep '05

23 Sep '05

On Thu, Sep 22, 2005 at 09:02:43AM +0200, Hans Hagen wrote: > Thanh Han The wrote: > > >My feeling is that we need to work out the specification and > >format of the `` node list'' first. In the first step, I > >would prefer to have only node-specific things, eg only what > >comes out after a box construction. I also got a similar > >request: to provide a primitive that writes out the content > >of a box and another primitive to re-construct that box back > >from the output. We can start with this and make further > >extensions later on. > > > >At the moment I cannot see clearly what is needed, but I am > >willing to write some extensions so that we can experiment with > >to see what is really needed and perhaps change what have been done. > > > > > i'd say ... go ahead, so that we get a picture; at least we then hav ea > starting point for karl's work err -- we need to work out how the node list will be output and input back first. I am still waiting for Karel's opinion. Thanh

1 0

pdfliteral after text/rules
by Hartmut Henkel 22 Sep '05

22 Sep '05

Hi, here ist the problem, that the "page" mode of \pdfliteral and \latelua does not by itself guarantee, that the result goes relative to the lower left corner (the PDF origin): \nopagenumbers \pdfcompresslevel=0 X\latelua{ x={};y={}; c=72/72.27/65536; x[1]=pdf.geth()*c; y[1]=pdf.getv()*c } \input tufte X\latelua page{ x[2]=pdf.geth()*c; y[2]=pdf.getv()*c } \hrule \latelua page{ tex.print(x[1], " ", y[1], " m ", x[2], " ", y[2], " l S") } \input tufte \bye The above example works as intended only, if you remove the \hrule. Reason is, that for setting a rule, a transform "cm" is done away from the origin, then the rule is placed, but there is no transforms back to the origin. The next placements are done after a transform relative to the placement point of the rule... Text is always typeset with the transform matrix "cm" set to the origin, and inside the BT...ET group the Td operator does the local transform. There seems to be currently no \pdfliteral (or \latelua) mode that guarantees positioning relative to the PDF origin. But positioning relative to the origin would be nice to have since we now have the pdf.geth/v() lua functions. Else one would have to think what was the state (text or rule?) just before using \pdfliteral page... To solve this ambiguity, i have tried the attached patch, which puts any rule into a qQ group and does "cm" transforms inside this group. So the rules are handled like text, which is also grouped (inside BT...ET). That means, that the global working reference point is (hopefully :) always (?) at the origin. With the rule.ch patch the above example works apparently ok. This also means, that when using \pdfliteral page, one gets stuff placed at the origin, and still without transform (which should be ok). This would spare to invent yet another \pdfliteral mode. No idea how general it is. And wonder, how backward-compatible? We would need to check also \pdfrefximage... Just for discussion... Regards, Hartmut P.S. Let's see how the attachment is handled. Never tried this before.

3 5