- dev-luatex - Mailing Lists

UTF-16 in \pdfoutline
by Jonathan Sauer 11 Dec '07

11 Dec '07

% The following code converts a string to UTF-16 big endian with BOM % and outputs it using \message: % We change the catcode of '%' so we can use it for modulo calculations: \begingroup \catcode`\%=12 \directlua0{\unexpanded{ function convertToUTF16(str) local result = string.char(0xFE) .. string.char(0xFF) for c in string.utfvalues(str) do if c < 0x10000 then result = result .. string.char(c / 256) .. string.char(c % 256) else c = c - 0x10000 local c1 = c / 1024 + 0xD800 local c2 = c % 1024 + 0xDC00 result = result .. string.char(c1 / 256) .. string.char(c1 % 256) .. string.char(c2 / 256) .. string.char(c2 % 256) end end tex.print('\\message{' .. result .. '}') end convertToUTF16('AäöüB!') }} \endgroup \bye This fails with 'Text line contains an invalid utf-8 sequence.' (not surprising, since the text is UTF-16 big endian). If I want to pass the UTF-16-encoded string i.e. to \pdfoutline (since PDF bookmarks can be encoded in UTF-16), how do I do this? (Maybe a callback would be useful, i.e. `convert_pdf_text') Thanks in advance, Jonathan

4 3

Hyphenation pattern information are not logged?
by Jonathan Sauer 11 Dec '07

11 Dec '07

Hello, LuaTeX does not seem to log the hyphenation memory usage anymore when in IniTeX mode. pdfTeX: ------------------------------------------------------------------------ This is pdfTeX, Version 3.141592-1.40.3 (Web2C 7.5.6) (INITEX) entering extended mode [...] Beginning to dump on file ../web2c/ktex.fmt (format=ktex 2007.12.8) 3245 strings of total length 50066 8638 memory locations dumped; current usage is 64&8395 1676 multiletter control sequences \font\nullfont=nullfont 0 words of font info for 0 preloaded fonts 14 hyphenation exceptions Hyphenation trie of length 13987 has 388 ops out of 35111 207 for language 1 181 for language 0 ------------------------------------------------------------------------ LuaTeX: ------------------------------------------------------------------------ This is LuaTeX, Version snapshot-0.20.0-2007120716 (Web2C 7.5.6) (INITEX) [...] Beginning to dump on file ../web2c/luaktex.fmt (format=luaktex 2007.12.8) 2970 strings of total length 47739 45428 memory locations dumped; current usage is 48&8787 1788 multiletter control sequences \font\nullfont=nullfont 0 preloaded fonts 0 words of active ocps \ocp\nullocp=nullocp 0 preloaded ocps \ocplist\nullocplist=[] ------------------------------------------------------------------------ I do not know if the new hyphenation mechanism still allows for logging its memory usage, and if it is still deemed useful/necessary. I just noticed this change since 0.11.2. Jonathan

2 1

Cron <luatex@www> /var/www/luatex.org/www/bin/luatex-svn
by root＠www.bittext.nl 09 Dec '07

09 Dec '07

U svn/trunk/build.sh.cross U svn/trunk/build.sh.linux U svn/trunk/build.sh Checked out revision 951.

1 0

LuaTeX dependencies
by Joel C. Salomon 09 Dec '07

09 Dec '07

I'm considering porting LuaTεχ to a new, vaguely Uɴɪx-like, operating system. (Plan 9 from Bell Labs, if anyone cares.) What libraries/compilers/interpreters does the system have to have for this to work? E.g., I know that texexec is currently written in Ruby (which Plan 9 lacks), but Lua is straight C (which Plan 9 is built around). (If you tell me to wait a few months until things stabilize, that's fine too.) --Joel

6 8

How to insert at start of hlist/vlist?
by Heiko Oberdiek 09 Dec '07

09 Dec '07

Hello, I have some trouble to insert something at the start of a list. Example (complete example file after PS): \setbox0=\hbox{XY} \directlua0{ do local hlist = tex.box[0] local pdf_literal = node.new(8,8)% \pdfliteral pdf_literal.mode = 2 % direct pdf_literal.data = "1 0 0 rg 1 0 0 RG" local head = hlist.list % local current = hlist.list.next % works local current = hlist.list % fails hlist.list = node.insert_before(head, current, pdf_literal) end } ! LuaTeX error Attempt to node.insert_before() a non-existing node. Have I misunderstood node.insert_before()? Yours sincerely Heiko <oberdiek(a)uni-freiburg.de> PS: Complete example file that can be used with iniTeX, plain-TeX and LaTeX: \ifnum\catcode`\{=1 \else \catcode`\{=1 \catcode`\}=2 \hsize=6.5in \vsize=8.9in \parfillskip=0pt plus 1fil \fi \font\f=cmr10 \f \pdfoutput=1 \setbox0=\hbox{XY} \directlua0{ do local hlist = tex.box[0] local pdf_literal = node.new(8,8)% \pdfliteral pdf_literal.mode = 2 % direct pdf_literal.data = "1 0 0 rg 1 0 0 RG" local head = hlist.list % local current = hlist.list.next % works local current = hlist.list % fails hlist.list = node.insert_before(head, current, pdf_literal) end } \shipout\hbox{[\box0]} \csname @@end\endcsname \end

3 4

Cron <luatex@www> /var/www/luatex.org/www/bin/luatex-svn
by root＠www.bittext.nl 07 Dec '07

07 Dec '07

U svn/trunk/src/texk/web2c/luatexdir/luatex.web U svn/trunk/src/texk/web2c/luatexdir/luatex.h U svn/trunk/src/texk/web2c/luatexdir/tex/postlinebreak.c U svn/trunk/src/texk/web2c/luatexdir/luatex.defines Checked out revision 944.

1 0

Cron <luatex@www> /var/www/luatex.org/www/bin/luatex-svn
by root＠www.bittext.nl 06 Dec '07

06 Dec '07

U svn/trunk/manual/luatexref-t.tex U svn/trunk/manual/luatexref-t.pdf Checked out revision 941.

1 0

Assertion while typesetting: vlink(alink(r))==r
by Jonathan Sauer 06 Dec '07

06 Dec '07

Hello, while typesetting a document of mine (just to check if my documents still work with the new LuaTeX beta), I got the following result (I used Arthur's build for OS X): ----------------------------------------------------------------- This is LuaTeX, Version snapshot-0.20.0-2007120515 (Web2C 7.5.6) (Schwarzfahren.tex (/Dokumente/Writing/trunk/Writing/DerProzess/TeX/tex/prozess.kprj [more included files] (Schwarzfahren.aux../../../../src/texk/web2c/luatexdir/tex/linebreak.c:215: failed assertion `vlink(alink(r))==r' ))Abort trap ----------------------------------------------------------------- I was unable to determine what exactly caused the assertion. Unfortunately, the document uses neither LaTeX nor Context nor PlainTeX, so I am also unable to post it here (unless I would post the macros as well). The raw text (without umlauts and headings) can be typeset correctly using PlainTeX. A more detailed log (with TeX's tracing turned on): ----------------------------------------------------------------- \BT:centerLine #1->\:noindent \BT:strut \:hfill #1\:hfill \BT:strut \:par #1<-Schwarzfahren {\noindent} \BT:strut ->\:relax \:ifmmode \:copy \:strutbox \:else \:unhcopy \:strutbox \:f i {horizontal mode: \relax} {\ifmmode} {false} {\unhcopy} {\fi} {\hfill} {the letter S} {\hfill} \BT:strut ->\:relax \:ifmmode \:copy \:strutbox \:else \:unhcopy \:strutbox \:f i {\relax} {\ifmmode} {false} {\unhcopy} {\fi} {\par} @firstpass []| \FS:O@font@t1/lmss/r/r/n/n/10@font Schwarz-fah-ren @ via @@0 b=0 p=0 d=100 @@1: line 1.2 t=100 -> @@0 | @\par via @@0 b=0 p=-10000 d=100 @\par via @@1 b=0 p=-10000 d=100 @@2: line 1.2- t=100 -> @@0 ../../../../src/texk/web2c/luatexdir/tex/linebreak.c:215: failed assertion `vlink(alink(r))==r' Abort trap ----------------------------------------------------------------- Also a little bit above this trace, while creating the current strut box, there are some weird "reassigning" messages: ----------------------------------------------------------------- {\setbox} {entering hbox group (level 2) at line 33} {reassigning [unknown integer parameter!]=0} {reassigning [unknown integer parameter!]=0} {reassigning [unknown integer parameter!]=0} {changing [level_local_dir]=1} {into [level_local_dir]=3} {restricted horizontal mode: \vrule} ----------------------------------------------------------------- If I disable font switches, a little bit more text is typeset before LuaTeX aborts with the same assertion. The text fragment typeset is (the \section results in "Einziges Kapitel: Berufswahl..." as seen below): ----------------------------------------------------------------- \chapter{Schwarzfahren}< preamble= {Februar `12}, label= {12.Schwarzfahren}, > \section{Berufswahl per Ausschlußverfahren}< preamble= {Worin ein ungestörtes Plätzchen (noch) eine Straßenbahnreise entfernt ist; ein Generalverdacht wird bedenkenlos akzeptiert; ein Glücksspieler verliert das Spiel; und eine Frage wird lieber nicht gestellt.}, datewritten={????/??/??, 2007/11/26}, > ----------------------------------------------------------------- And the trace (also note how TeX's primitives are traced with their original name and not the name they have in the macros): ----------------------------------------------------------------- [now the \chapter's title is typeset] \BT:centerLine #1->\:noindent \BT:strut \:hfill #1\:hfill \BT:strut \:par #1<-Schwarzfahren {\noindent} \BT:strut ->\:relax \:ifmmode \:copy \:strutbox \:else \:unhcopy \:strutbox \:f i {horizontal mode: \relax} {\ifmmode} {false} {\unhcopy} {\fi} {\hfill} {the letter S} {\hfill} \BT:strut ->\:relax \:ifmmode \:copy \:strutbox \:else \:unhcopy \:strutbox \:f i {\relax} {\ifmmode} {false} {\unhcopy} {\fi} {\par} @firstpass [] Schwarz-fah-ren @\par via @@0 b=0 p=-10000 d=100 @@1: line 1.2- t=100 -> @@0 %% goal height=431.24408, max depth=4.0 % t=0.0 g=431.24408 b=10000 p=0 c=100000# \BT:suppressNextParIndent ->\:begingroup \:edef \:Mi {\:the \:everypar }\:expan dafter \:endgroup \:ifx \:Mi \:empty \:global \:everypar {\:global \:everypar { }{\:setbox \:z \:lastbox }}\:else \:global \:everypar \:expandafter {\:expandaf ter \:global \:expandafter \:everypar \:expandafter {\:the \:everypar }{\:setbo x \:z \:lastbox }\:the \:everypar }\:fi [about 10000 lines of trace inbetween, mostly key-value processing] [now the \section's title immediately after the \chapter is typeset] \CF:first:b #1->#1 #1<-: Berufswahl per Ausschlußverfahren {blank space } {the letter B} {blank space } {the letter p} {blank space } {the letter A} ß->\EX:protect \OE:ssharp \EX:protect -> \OE:ssharp ->\:char 255 {\par} @firstpass []Ein-zi-ges @ via @@0 b=0 p=0 d=100 @@1: line 1.2 t=100 -> @@0 Ka-pi-tel: @ via @@0 b=0 p=0 d=100 @ via @@1 b=0 p=0 d=100 @@2: line 1.2 t=100 -> @@0 Be-rufs-wahl @ via @@0 b=0 p=0 d=100 @ via @@1 b=0 p=0 d=100 @ via @@2 b=0 p=0 d=100 @@3: line 1.2 t=100 -> @@0 per @ via @@0 b=0 p=0 d=100 @ via @@1 b=0 p=0 d=100 @ via @@2 b=0 p=0 d=100 @ via @@3 b=0 p=0 d=100 @@4: line 1.2 t=100 -> @@0 Aus-schluÿ-ver-fah-ren @\par via @@0 b=0 p=-10000 d=100 @\par via @@1 b=0 p=-10000 d=100 @\par via @@2 b=0 p=-10000 d=100 @\par via @@3 b=0 p=-10000 d=100 @\par via @@4 b=0 p=-10000 d=100 @@5: line 1.2- t=100 -> @@0 ../../../../src/texk/web2c/luatexdir/tex/linebreak.c:215: failed assertion `vlink(alink(r))==r' Abort trap ----------------------------------------------------------------- (LuaTeX beta 0.11.2 worked correctly) @Taco: What do you need to narrow this down? I can try to create a PlainTeX document exhibiting the same behaviour. Or maybe you could include some more tracing of the line breaking algorithm (just temporarily, in the trunk), and I send you the log file of the run. The document uses the "process_input_buffer" callback to convert vom MacRoman to UTF-8. It uses Latin Modern, T1 encoding. It uses a simple OTR with a single column as well as a simple header and footer. It does not use any specials, but the pdfTeX color stack. It uses Context's UTF-8 hyphenation patterns for traditional german, slightly modified to include "y diaresis" as a synonym for "s sharp". Jonathan

6 9

Cron <luatex@www> /var/www/luatex.org/www/bin/luatex-svn
by root＠www.bittext.nl 06 Dec '07

06 Dec '07

U svn/trunk/src/texk/web2c/luatexdir/lang/texlang.c U svn/trunk/src/texk/web2c/luatexdir/lua/luainit.c U svn/trunk/src/texk/web2c/luatexdir/font/texfont.c Checked out revision 940.

1 0

Re: [Dev-luatex] Assertion while typesetting: vlink(alink(r))==r
by Jonathan Sauer 06 Dec '07

06 Dec '07

Hello, > > This is LuaTeX, Version snapshot-0.20.0-2007120515 (Web2C 7.5.6) > > (Schwarzfahren.tex > > (/Dokumente/Writing/trunk/Writing/DerProzess/TeX/tex/prozess.kprj > > [more included files] > > > (Schwarzfahren.aux../../../../src/texk/web2c/luatexdir/tex/lin > ebreak.c:215: failed assertion `vlink(alink(r))==r' > > ))Abort trap > > I hope you don't mind some unrelated, off-topic jokes :) > > I guess it fails because of the title (and/or content). You > really aren't supposed to take public lines or connections > (links) between them when "Schwarzfahren" :p Well, the protagonists just talk about "Schwarzfahren", they of course have a ticket. So I think it a bit unfair of LuaTeX to simply *assert* they don't. Note that it also failed with a file called "UeberDemMensch.tex". Guess LuaTeX does not like Nietzsche either. But why? "What is Word to TeX? A laughing-stock, a thing of shame. And just the same shall TeX be to LuaTeX: a laughing-stock, a thing of shame." ;-) > Mojca Jonathan

4 8