- dev-luatex - Mailing Lists

PNG file inclusion
by Reinhard Kotucha 05 Jan '20

05 Jan '20

Hi, I just encountered a problem with a broken PNG file. The PNG file was created with Microsoft's "Snipping Tool". The problem affects both, LuaTeX and pdfTeX and proably XeTeX too (not tested). When I include a PNG file created by Microsoft's "Snipping Tool", Okular crashes. No Problem when I convert PNG->PNM->PNG with ImageMagick. In both cases the header is: 1 0 obj << /Type /XObject /Subtype /Image /Width 1942 /Height 878 /BitsPerComponent 8 /ColorSpace /DeviceRGB /Length 5115228 >> stream .... endstream endobj I would expect that the number of bytes between "stream" and "endstream" is exactly the number of bytes denoted by /Length. When I use the original PNG file, 6820463 bytes are inserted between stream and endstream, with the converted file exactly 5115228 bytes are written. The value of /Length is obviously derived from the header: 1942 × 878 × 3 = 5115228 Assuming that a PDF viewer uses the value of /Length to demermine the amount of memory to allocate, crashes are quite likely and maybe this behavior can be abused to inject code into unallocated memory. Assuming that libpng is used for PNG decompression, it's probably the culprit. But I don't know whether the PNG standard allows to add additional information to the file. In this case the size cannot be determined from the header. My suggestion is, instead of relying entirely on external libraries, to make sure that (pdf|Lua)TeX always inserts exactly /Length bytes to the stream. If this is not possible, abort with an meaningful error message. At the end TeX has to create valid PDF and without such a check such problems get undetected. I've got the PNG file from someone writing his master thesis on Windows. He didn't encounter any problem himself. Here only Okular crashed, Evince and two versions of Adobe Reader didn't complain. And a minimal example which contains only the PNG file is accepted by Okular. Thus I can't provide a minimal example. But I've copied the PNG and PDF files to http://ms25.ddns.net/texlive/png-inclusion.tar.xz The PDF files were created by pdfTeX, using only the \pdfximage and \pdfrefximage primitives. Regards, Reinhard -- ------------------------------------------------------------------ Reinhard Kotucha Phone: +49-511-3373112 Marschnerstr. 25 D-30167 Hannover mailto:reinhard.kotucha@web.de ------------------------------------------------------------------

2 1

Allow skippig the font cache in font.getfont
by Marcel Fabian Krüger 12 Dec '19

12 Dec '19

Hello, could font.getfont get an option to always return a new font table instead of returning the current one? While for many fonts a cached table makes sense, sometimes it would be useful to also have access to the ooriginal table, e.g. to see which characters are `used`, inspect the result of `font.addcharacters` or inspect other changes which might be applied to the underlying font from TeX. I attached a patch implementing this by adding a `nocache` parameter to `font.getfont`, such that `font.getfont(true, font_id)` always generates a new table. IMO `font.getfont(font_id, true)` would look nicer, but that wouldn't be compatible to older LuaTeX versions which always expect the font is as last parameter. Best regards, Marcel

2 1

commands with a non-letter in token.set_macro
by Ulrike Fischer 11 Dec '19

11 Dec '19

Commands that should be use in token.set_macro must already exist. But while it is easy to predefine a command like \blub it seems not to be possible to do it with commands consisting of a \ with a non-letter. E.g. I put this in a luatest.lua token.set_macro("foo","\\bar\\blub\\!") and then compile \def\!{hallo} \def\blub{hallo} \directlua{require"luatest"} \show\foo \bye I get: > \foo=macro: ->\bar \blub BAD!. l.254 \show\foo so the \blub works, but not the \!. How can one repair this? -- Ulrike Fischer https://www.troubleshooting-tex.de/

2 1

Re: [Dev-luatex] Request for bugfix - user defined whatsit node
by luigi scarso 05 Dec '19

05 Dec '19

On Tue, Dec 3, 2019 at 6:16 PM Hans Hagen <j.hagen(a)xs4all.nl> wrote: > On 11/15/2019 12:35 PM, Sigitas Tolusis wrote: > > Noticed two bugs. > > > > 1. Mised case for user defined whatsit node type 108 (‘l’ Lua value) > > in function check_node_wrapup_core (texnodes.c:2482). > > Possible fix: just add the same as cases for types ’s’ and ‘d’. > > Insert line at texnodes.c:2500 > > case ‘l': > > > > 2. In function flush_node_wrapup_core (texnodes.c:2201) for user defined > > whatsit node type ‘l’ we have: free_user_lua(user_node_value(p)). > > But in luanode.c:409 free_user_lua is defined as: > > > > void free_user_lua(pointer p) > > { > > if (user_node_value(p) != 0) { > > luaL_unref(Luas, LUA_REGISTRYINDEX, user_node_value(p)); > > } > > } > > > > So we have two times user_node_value and it generates segmentation fault. > > Possible fix: free_user_lua(p); // (texnodes.c:2223) > fixed in next luatex beta buit best test then > > Hans > Done in rev. 7233 . -- luigi

1 0

Revive Type 3 fonts
by Marcel Fabian Krüger 15 Nov '19

15 Nov '19

Hey, LuaTeX currently supports only a very special kind of Type 3 fonts: They are generated for traditional TeX bitmap fonts. While this used to be the most common use case for Type3 fonts, the fully general concept of these fonts has recently been reinvented under the name of SVG fonts (and friends). While they can be implemented in LuaTeX to some degree through virtual fonts/node processing etc., converting them into "real" Type 3 fonts in the PDF file would give some important advantages: Such fonts are called like other fonts and can get tounicode vectors, allowing better accessibillity and better Copy&Paste behaviour. Also viewers generally treat them as text instead of graphics, allowing proper interaction with features of PDF viewers. Beside from supporting SVG fonts, it also allows to use general MetaPost fonts which might not be easily convertable into normal formats for vector fonts. As far as I can tell there used to be an implementation of general Type 3 fonts for pdfTeX which was dropped in LuaTeX, mostly because it was never actually used. My suggested new implementation is a bit more flexible by not requiring external files defining the font but instead generating each font character based on individual node lists for every glyph. One way to make this work would be to add a special `format` named e.g. `node` (given that `type3` is already used to imply bitmap fonts) and add a field `node`/`list`/`head` to the character table referencing a node list which should become the font glyph. The detaily like bounding box calculations etc. would be the responsibility of the Lua code loading the font. The intended usage would be e.g. \directlua{ local l = token.scan_list() local wi = node.new('whatsit', 'pdf_literal') wi.mode = 3 wi.data = math.floor(l.width / tex.sp'1bp') .. ' 0 d0' wi.next = l.head l.head = wi font.current(font.define{ name = "exp-nodefont", format = "node", designsize = tex.sp("10pt"), size = tex.sp("10pt"), characters = { [42] = { width = l.width, height = l.height, depth = l.depth, node = node.direct.todirect(l), } } })}\hbox{ABC} \char42 \char42 \bye What do you think? (I'll attach a patch I used for experiments (mostly with MetaPost fonts) which implements this in a slightly hacky way.) -- Marcel

6 7

Improvement for `glyph_info`
by Marcel Fabian Krüger 15 Nov '19

15 Nov '19

Hi, the new `glyph_info` callback is called in two pretty different cases: in `short_display(_n)` where a pretty short representation of the actual output characters is required, and in `print_font_and_char`, which creates the representation for `print_node_list`. Here, more details can be useful, e.g.: Which variant of a character is used, are displacements applied etc. These could be added through `glyph_info`, but then these additional fields are also printed in `short_display`, leading to significantly worse overfull warnings etc. This could be improved if `glyph_info` could return different values in different situations. So I suggest to add an additional parameter to the `glyph_info` callback which states if the current context could handle longer output. A possible implementation is attached. Best regards, Marcel

4 4

require doesn't get recorded
by Marcel Fabian Krüger 08 Nov '19

08 Nov '19

Hi, based on an observation by Karl Berry on the TeX Live list I discovered an inconsistency in LuaTeX's handling of `-recorder`: While LuaTeX does record files loaded through `loadfile`, files loaded through `require` are ignored: For example if the current directory contains two empty files l1.lua and l2.lua, then the document \directlua{ loadfile'l1.lua'() require'l2' } \bye compiled with `luatex -recorder recorded` IMO should write to recorded.fls the entries PWD /some/weird/pwd INPUT /usr/local/texlive/2019/texmf-var/web2c/luatex/luatex.fmt INPUT ./recorded.tex OUTPUT recorded.log INPUT l1.lua INPUT ./l2.lua but it does write PWD /some/weird/pwd INPUT /usr/local/texlive/2019/texmf-var/web2c/luatex/luatex.fmt INPUT ./recorded.tex OUTPUT recorded.log INPUT l1.lua I do not think that this is intentional, so I added a patch to add recorder entries iff the file is loaded successfully (similar to `loadfile`) -- Marcel

3 6

Re: [Dev-luatex] Broken \adjdemerits on Linux due to unspecified enumeration type
by Marcel Fabian Krüger 05 Nov '19

05 Nov '19

Hi, thanks for fixing the \adjdemerits problem. One remark about the fix: The line if ( (fit_class>(fitness(r)+1)) || (fitness(r)>(fit_class+1)) ) is indented by one tab "\t" and four spaces, while all surrounding code is only indented with spaces. (Confusingly, tabs are also used as indentation in line 117-119) This leads to confusing appearence if the file is viewed in an editor which does not use 8 spaces for one tab. Maybe this could be changed to be a bit more consistant? Best regards, Marcel

2 1

TTC handling
by Marcel Fabian Krüger 31 Oct '19

31 Oct '19

Hey, in `writefont0.c` and `writefont2.c`, `ff_get_ttc_index` is called to determine the index of the font in a font collection. This uses the original filename from fd->fm->ff_name and therefore does not respect lookups done using the find_opentype_file etc. callbacks. In writefont0.c, this could be fixed relativly easily by passing using cur_file_name, the name after the callback, instead. But even than there are remianing problems because this still wouldn't respect read_..._file callbacks. So implementing this correctly is quite tricky and it also isn't really necessary in most cases: Especially when using Lua-loaded fonts, the code creating the font needs to know the index anyway. So maybe an approach like in Khaled Hosny's HarfTeX could be used: Allow a Lua-loaded font to explicitly set the index and only use ff_get_ttc_index if this index isn't set. Then backward-compatibility is kept but the actually used index selection can be influenced using Lua. (HarfTeX implemented this in https://github.com/khaledhosny/harftex/commit/caded3e80f106687c84f9613aa318…) Best regards, Marcel

2 1

Broken \adjdemerits on Linux due to unspecified enumeration type
by Marcel Fabian Krüger 30 Oct '19

30 Oct '19

Hi, current versions of LuaTeX from TeXLive for Linux (Windows is not affected) always add `\adjdemerits` when a line is more loose than the previous one, even if it is only one step more loose: Take \tracingparagraphs1 \tracingonline1 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa \bye This gives @firstpass []\tenrm aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @ via @0 b=96 p=0 d=1011236 @@1: line 1.1 t=21236 -> @0 aaaaaa @ via @0 b=83 p=0 d=8649 @@2: line 1.3 t=8649 -> @0 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aa aaaa aaaaaa @ via @1 b=3 p=0 d=169 @@3: line 2.2 t=1011405 -> @1 aaaaaa @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @\par via @3 b=0 p=-10000 d=100 @\par via @4 b=0 p=-10000 d=100 @@5: line 3.2- t=1008918 -> @4 Here, the relevant part is @@2: line 1.3 t=8649 -> @0 ... @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 So breakpoint @2 is .3 aka. "tight", while @4 is .2 aka. "decent". This break gives demerits d=100169=1000000+(10+3)^2. So \adjdemerits are inserted which is wrong because tight and decent lines are "visually compatible" as defined in the TeXbook. The problem is in linebreak.c. There the demerits are inserted by if (abs(fit_class - fitness(r)) > 1) d = d + adj_demerits; Both fit_class and fitness(r) have type fitness_value which is an enum with four entries. These enums have an implementation defined type and the bug is triggered if LuaTeX is compiled with a compiler which uses an unsigned type: Then `fit_class - fitness(r)` is a difference of unsigned values and therefore unsigned. If fitness(r) is bigger than fit_class, the attempt to store a negative difference in a unsigned value wraps around to a high positive value which is bigger than 1. So if fitness(r) is bigger than fit_class, the condition is always true. This can be fixed by explicitly casting fit_class and fitness(r) to a signed type. A possible patch is attached. Best regards, Marcel

3 4