April 2007 - dev-luatex - NTG Mailing Lists

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 13 Apr '07

13 Apr '07

U luatex/src/libs/luapeg/lpeg.c U luatex/src/texk/web2c/luatexdir/image/writeimg.c U luatex/src/texk/web2c/luatexdir/luatex.web U luatex/src/texk/web2c/luatexdir/lua/lcallbacklib.c U luatex/src/texk/web2c/luatexdir/lua/luatex.c U luatex/src/texk/web2c/luatexdir/lua/luatoken.c U luatex/src/texk/web2c/luatexdir/lua/ltokenlib.c U luatex/src/texk/web2c/luatexdir/lua/luanode.c U luatex/src/texk/web2c/luatexdir/lua/luastuff.c U luatex/src/texk/web2c/luatexdir/luatex.h U luatex/src/texk/web2c/luatexdir/luatex-api.h U luatex/src/texk/web2c/luatexdir/font/writetype0.c U luatex/src/texk/web2c/luatexdir/font/mapfile.c U luatex/src/texk/web2c/luatexdir/font/dofont.c U luatex/src/texk/web2c/luatexdir/font/writet1.c U luatex/src/texk/web2c/luatexdir/font/writet3.c U luatex/src/texk/web2c/luatexdir/font/writettf.c U luatex/src/texk/web2c/luatexdir/font/subfont.c U luatex/src/texk/web2c/luatexdir/font/vfovf.c U luatex/src/texk/web2c/luatexdir/font/tfmofm.c U luatex/src/texk/web2c/luatexdir/luatex.defines Checked out revision 405.

1 0

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 11 Apr '07

11 Apr '07

U luatex/src/libs/luapeg/lpeg.c U luatex/src/texk/web2c/luatexdir/luatex.web U luatex/src/texk/web2c/luatexdir/lua/luawhatsits.c U luatex/src/texk/web2c/luatexdir/lua/nodes.h U luatex/src/texk/web2c/luatexdir/lua/luanode.c Checked out revision 399.

1 0

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 09 Apr '07

09 Apr '07

U luatex/src/texk/web2c/luatexdir/luatex.web U luatex/src/texk/web2c/luatexdir/lua/luawhatsits.c U luatex/src/texk/web2c/luatexdir/lua/luatoken.c U luatex/src/texk/web2c/luatexdir/lua/luanode.c U luatex/src/texk/web2c/luatexdir/luatex.defines Checked out revision 397.

1 0

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 09 Apr '07

09 Apr '07

U luatex/src/texk/web2c/luatexdir/lua/nodes.h Checked out revision 392.

1 0

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 08 Apr '07

08 Apr '07

U luatex/src/libs/lua51/Makefile U luatex/src/texk/web2c/luatexdir/luatex.web U luatex/src/texk/web2c/luatexdir/lua/luawhatsits.c U luatex/src/texk/web2c/luatexdir/lua/luatoken.c U luatex/src/texk/web2c/luatexdir/lua/luanode.c Checked out revision 391.

1 0

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 06 Apr '07

06 Apr '07

U luatex/src/texk/web2c/luatexdir/Makefile.in U luatex/src/texk/web2c/luatexdir/luatex.web U luatex/src/texk/web2c/luatexdir/lua/lcallbacklib.c A luatex/src/texk/web2c/luatexdir/lua/luawhatsits.c A luatex/src/texk/web2c/luatexdir/lua/nodes.h U luatex/src/texk/web2c/luatexdir/lua/luatoken.c U luatex/src/texk/web2c/luatexdir/lua/ltokenlib.c A luatex/src/texk/web2c/luatexdir/lua/luanode.c U luatex/src/texk/web2c/luatexdir/font/ttflua.c U luatex/src/texk/web2c/luatexdir/font/ttfotf.c U luatex/src/texk/web2c/luatexdir/font/splinefont.h U luatex/src/texk/web2c/luatexdir/luatex.defines U luatex/manual/luatexref-t.tex Checked out revision 386.

1 0

Minimal example of using OTF/TTF?
by Geoffrey Alan Washburn 06 Apr '07

06 Apr '07

There is probably a really simple answer to this, but what is the current method for loading OpenType/TrueType fonts in luatex? I've tried a few things that do not appear to work, and I haven't been able to find any complete examples of how to do so. \pdfoutput=1 \font\myfont={Vera.ttf} % This doesn't work %\font\myfont={cmr10} % This does work {\myfont Testing} \end % luatex --fmt=plain foo.tex This is luaTeX, Version 3.141592-snapshot-2007040613 (Web2C 7.5.6) (foo.tex ! Font \myfont=Vera.ttf not loadable: Metric (TFM/OFM) file not found or bad. <to be read again> \par l.4 Where Vera.ttf is in current directory. My impression from the manual (page 38) was that luatex could construct metrics directly from the font without needing something like otftotfm. I also tried registering a define_font callback that called the font.read_ttf function, but that gave similar results. I'm using luatex build from the subversion repository (currently at revision 386). -- [Geoff Washburn|geoffw@cis.upenn.edu|http://www.cis.upenn.edu/~geoffw/]

2 1

Benchmark.
by David Kastrup 06 Apr '07

06 Apr '07

Hi, latex -ini latex.ltx gives real 0m0.498s user 0m0.080s sys 0m0.408s luatex -ini latex.8bit (where the latter just sets up the bytes->utf8 conversion through \directlua0{ callback.register("process_input_buffer", function(buf) return unicode.utf8.char(unicode.latin1.byte(buf,1,-1)) end)} like in the last mail and then loads LaTeX) is real 0m2.730s user 0m1.836s sys 0m0.700s Sadly, I have no good way to see how much of this is caused by the callback, and how much is due to other LuaTeX particularities. However, one can run tex.tex through LuaTeX with and without this translation, and just run the normal TeX engine, too. tex tex.tex gives us real 0m1.253s user 0m0.496s sys 0m0.664s luatex tex.tex gives us real 0m14.329s user 0m13.413s sys 0m0.872s and using time luatex "&plain" '\directlua0{callback.register("process_input_buffer",function(buf)return unicode.utf8.char(unicode.latin1.byte(buf,1,-1))end)}\input tex' gives us real 0m14.801s user 0m12.709s sys 0m1.304s So the good news is that using the callback makes LuaTeX faster (more probably the difference gets lost in the noise). And the bad news is that it is about a factor of 25 slower than the normal TeX executable in either case. Some of it might be the difference in table sizes for the plain TeX executable. But the factor of 25 seems to fit rather well also with the LaTeX format test. Any idea where the bulk of this would be from? What would somebody wanting to use LuaTeX in a production environment do (apart from getting his head examined, I mean)? -- David Kastrup

3 23

buffer size musings
by David Kastrup 05 Apr '07

05 Apr '07

Hi, since I am off for Easter, I just want to spread the implementation idea for dealing with arbitrary long lines, so that I can get some feedback before I get to actual coding. Since LuaTeX has its own complications to take care off with regard to utf8, I actually would want to prepare this as a LuaTeX patch. Backporting to PDFTeX should be straightforward. I'll also try thinking about some input encoding implementation place: the process_input_buffer callback has several drawbacks: for one thing, it can't expect to get complete lines if I prepare lines peacemeal. So it would probably have to get another argument "partial". For another, the line end needs to get detected in the first place. But line detection _depends_ on the encoding, at least if we are talking about utf16 flavors. And in particular if process_input_buffer gets partial lines, I'd like it to get lines that don't contain partial utf-8 sequences. So I do see a need for some more stuff specific to LuaTeX, and I don't want to design something that would be hard to port to it. So much for some background. The basic design would be the following: buffer happens to be a single buffer. This could probably be chosen with a total size of 32k (naturally, people will disagree here, but that will stay configurable). Before reading material from a file, TeX will _start_ by placing the \endlinechar before any other material, as a fixed 4-byte long utf-8 sequence (alternatively, it may be stored as part of the file data structure): its setting at the time of reading the file needs to get preserved until we finally reach the end of the file. Then the next line gets read into buffer, either until end of line is reached, or until the buffer read limit (2k sounds reasonable, could conceivably be configurable since it influences things like maximum size of \csname ...\endcsname) is hit. When processing material, we usually check for end of line condition, anyway. When those checks turn out true, we do another check for "really" end of line or just end of the buffered part. If it is just the end of the buffered part, then sufficient material from the end of the buffered part is copied to the front of the buffered part, more stuff is read in according to the buffer read limit (and possibly tacking on the buffered end line character at the end), and we resume. "sufficient material" means the maximum of a) 40 characters (probably 160 bytes will do) of error context for the input line context part of error messages. b) if we are in the middle of scanning a control sequence name, the beginning of the control sequence. If this copying process would not result in any more available space (making it possible to actually read in new material), we get the dreaded buffer overflow. Basically this concept appears sound to me. It would, however, be strictly restricted to file reading. Things like \scantokens and \csname (which also use the buffer) would still be required to have their argument fit in one piece. But I guess that the file reading should cover the largest problem area. Something like that. I hope I'll have something to show before EuroTeX. But as I said, I am away without net access (and without computer) for the next week. All the best, David -- David Kastrup

3 6

\endlinechar can't be set to non-ASCII values.
by David Kastrup 04 Apr '07

04 Apr '07

Hi, the following program barfs and gets confused: \endlinechar128 \relax \relax \end This is luaTeX, Version 3.141592-snapshot-2007040210 (Web2C 7.5.6) (INITEX) (/tmp/junk.tex ! Buffer contains an invalid utf-8 sequence. l.2 \ relax ? ! Undefined control sequence. l.2 \rel ax ? ! Buffer contains an invalid utf-8 sequence. l.2 \relax ? ! Buffer contains an invalid utf-8 sequence. l.3 \ relax ? ! Undefined control sequence. l.3 \rel ax ? ! Buffer contains an invalid utf-8 sequence. l.3 \relax ? ! Buffer contains an invalid utf-8 sequence. l.4 \ end ? ! Undefined control sequence. l.4 \e nd ? ! Buffer contains an invalid utf-8 sequence. l.4 \end ? ) * ! Emergency stop. <*> /tmp/junk.tex No pages of output. Transcript written on junk.log. Maybe it would be better if buffer was an array of ASCII rather than packedASCII, namely encoded in UTF21 (or whatever) internally rather than UTF8. As it stands, dealing with \endlinechar and the end of buffer will not exactly be a lot of fun. Or one will need a function that is going to place a single character into buffer, and one needs to take care that at all times at least 4 (rather than one) characters remain free whenever \endlinechar has not yet been appended. -- David Kastrup

2 2