- dev-luatex - NTG Mailing Lists

Updates for (lua)HarfBuzz
by Marcel Fabian Krüger 25 Jan '20

25 Jan '20

Hello, would it be possible to pull a new version of luaharfbuzz in before TeXLive 2020 is frozen? There hasn't been an official release, but there are some new interfaces which would be very useful for luaotfload. There are also at least two bugs in HarfBuzz affecting luaotfload for which the fixes are in the repo but not yet released, but I guess the HarfBuzz version is controlled by the TeXLive team? Best regards, Marcel

3 4

Request for bugfix - rule nodes (subtypes: 1 = box, 2 = image, 3 = empty, 4 = user) in DVI mode
by Sigitas Tolusis 24 Jan '20

24 Jan '20

Three things to be fixed for the rule nodes with subtypes 1 to 4 in DVi mode. (Function dvi_place_rule from dvigen.c file) void dvi_place_rule(PDF pdf, halfword q, scaledpos size) { synch_dvi_with_pos(pdf->posstruct->pos); if ((subtype(q) >= box_rule) && (subtype(q) <= user_rule)) { /*tex place nothing, only take space */ if (textdir_is_L(pdf->posstruct->dir)) dvi.h += size.h; } else { /*tex normal_rule or >= 100 being a leader rule */ if (textdir_is_L(pdf->posstruct->dir)) { /*tex movement optimization for |dir_*L*| */ dvi_out(set_rule); dvi.h += size.h; } else dvi_out(put_rule); } dvi_four(size.v); dvi_four(size.h); } Problem 1. Bad dvi file due the arguments output without dvi command (misplaced the two last lines in the code). Possible fix: void dvi_place_rule(PDF pdf, halfword q, scaledpos size) { synch_dvi_with_pos(pdf->posstruct->pos); if ((subtype(q) >= box_rule) && (subtype(q) <= user_rule)) { /*tex place nothing, only take space */ if (textdir_is_L(pdf->posstruct->dir)) dvi.h += size.h; } else { /*tex normal_rule or >= 100 being a leader rule */ if (textdir_is_L(pdf->posstruct->dir)) { /*tex movement optimization for |dir_*L*| */ dvi_out(set_rule); dvi.h += size.h; } else dvi_out(put_rule); dvi_four(size.v); dvi_four(size.h); } } Problem 2. Not reserved space for the rule in the output. Possible fix: void dvi_place_rule(PDF pdf, halfword q, scaledpos size) { synch_dvi_with_pos(pdf->posstruct->pos); if ((subtype(q) >= box_rule) && (subtype(q) <= user_rule)) { /*tex place nothing, only take space */ if (textdir_is_L(pdf->posstruct->dir)) dvi.h += size.h; movement(size.h, right1); } else { /*tex normal_rule or >= 100 being a leader rule */ if (textdir_is_L(pdf->posstruct->dir)) { /*tex movement optimization for |dir_*L*| */ dvi_out(set_rule); dvi.h += size.h; } else dvi_out(put_rule); dvi_four(size.v); dvi_four(size.h); } } Problem 3. Broken consistency with PDF mode as process_rule callback is missing in DVI mode. Possible fix: void dvi_place_rule(PDF pdf, halfword q, scaledpos size, int callback_id) { synch_dvi_with_pos(pdf->posstruct->pos); if ((subtype(q) >= box_rule) && (subtype(q) <= user_rule)) { /*tex place nothing, only take space */ if (textdir_is_L(pdf->posstruct->dir)) dvi.h += size.h; if (subtype(q) == user_rule && callback_id != 0) { run_callback(callback_id, “Ndd->”, q, size.h, size.v); } movement(size.h, right1); } else { /*tex normal_rule or >= 100 being a leader rule */ if (textdir_is_L(pdf->posstruct->dir)) { /*tex movement optimization for |dir_*L*| */ dvi_out(set_rule); dvi.h += size.h; } else dvi_out(put_rule); dvi_four(size.v); dvi_four(size.h); } } and fix in dvigen.h < extern void dvi_place_rule(PDF pdf, halfword q, scaledpos size); > extern void dvi_place_rule(PDF pdf, halfword q, scaledpos size, int callback_id); Regards, Sigitas P.S. Some point for the discussion. For the wider usabillity maybe the rules with zero dimensions (width, height+depth) can be allowed in the output.

2 1

wrong no-file-line-error option
by Hironobu Yamashita 17 Jan '20

17 Jan '20

Hi, The command-line option -no-file-line-error behaves exactly the same as -file-line-error. hironobu$ cat test.tex \x hironobu$ echo x | luatex test This is LuaTeX, Version 1.10.0 (TeX Live 2020/dev) restricted system commands enabled. (./test.tex ! Undefined control sequence. l.1 \x ? warning (pdf backend): no pages of output. Transcript written on test.log. hironobu$ echo x | luatex -file-line-error test This is LuaTeX, Version 1.10.0 (TeX Live 2020/dev) restricted system commands enabled. (./test.tex ./test.tex:1: Undefined control sequence. l.1 \x ? warning (pdf backend): no pages of output. Transcript written on test.log. hironobu$ echo x | luatex -no-file-line-error test This is LuaTeX, Version 1.10.0 (TeX Live 2020/dev) restricted system commands enabled. (./test.tex ./test.tex:1: Undefined control sequence. l.1 \x ? warning (pdf backend): no pages of output. Transcript written on test.log. hironobu$ Best, Hironobu

1 1

LuaTeX 1.10.0 cannot embed properly PDFs with 4-bit images
by L. E. Segovia 17 Jan '20

17 Jan '20

Hello, I would like to report that since TeX Live 2019, LuaTeX is unable to embed properly a certain type of image I have. These sort of images are embedded in the PDFs generated by the tool "Comprobantes en línea" of Argentina's IRS (Administración Federal de Ingresos Públicos, https://www.afip.gob.ar) This is an example: https://comunadealvear.gob.ar/wp-content/uploads/2017/03/30710518803_01_000… (or just search online for the string "Esta Administración Federal no se responsabiliza por los datos ingresados en el detalle de la operación"). Embed this file using the following snippet: \documentclass{article} \usepackage{luatex85} \usepackage{incgraph} \begin{document} \incgraph{30710518803_01_0002_00000009.pdf} \end{document} Where the white-and-grey AFIP logo appears in the original, LuaTeX instead embeds a grey-on-grey image, probably by messing with the alpha channel. I have consistently reproduced this on every update since the 2019 release. XeLaTeX and pdfTeX do not have this issue. Please let me know if I can be of more assistance. I hope I've posted to the right mailing list. Best regards, Leonardo -- Leonardo E. Segovia https://www.amyspark.me

3 6

PNG file inclusion
by Reinhard Kotucha 05 Jan '20

05 Jan '20

Hi, I just encountered a problem with a broken PNG file. The PNG file was created with Microsoft's "Snipping Tool". The problem affects both, LuaTeX and pdfTeX and proably XeTeX too (not tested). When I include a PNG file created by Microsoft's "Snipping Tool", Okular crashes. No Problem when I convert PNG->PNM->PNG with ImageMagick. In both cases the header is: 1 0 obj << /Type /XObject /Subtype /Image /Width 1942 /Height 878 /BitsPerComponent 8 /ColorSpace /DeviceRGB /Length 5115228 >> stream .... endstream endobj I would expect that the number of bytes between "stream" and "endstream" is exactly the number of bytes denoted by /Length. When I use the original PNG file, 6820463 bytes are inserted between stream and endstream, with the converted file exactly 5115228 bytes are written. The value of /Length is obviously derived from the header: 1942 × 878 × 3 = 5115228 Assuming that a PDF viewer uses the value of /Length to demermine the amount of memory to allocate, crashes are quite likely and maybe this behavior can be abused to inject code into unallocated memory. Assuming that libpng is used for PNG decompression, it's probably the culprit. But I don't know whether the PNG standard allows to add additional information to the file. In this case the size cannot be determined from the header. My suggestion is, instead of relying entirely on external libraries, to make sure that (pdf|Lua)TeX always inserts exactly /Length bytes to the stream. If this is not possible, abort with an meaningful error message. At the end TeX has to create valid PDF and without such a check such problems get undetected. I've got the PNG file from someone writing his master thesis on Windows. He didn't encounter any problem himself. Here only Okular crashed, Evince and two versions of Adobe Reader didn't complain. And a minimal example which contains only the PNG file is accepted by Okular. Thus I can't provide a minimal example. But I've copied the PNG and PDF files to http://ms25.ddns.net/texlive/png-inclusion.tar.xz The PDF files were created by pdfTeX, using only the \pdfximage and \pdfrefximage primitives. Regards, Reinhard -- ------------------------------------------------------------------ Reinhard Kotucha Phone: +49-511-3373112 Marschnerstr. 25 D-30167 Hannover mailto:reinhard.kotucha@web.de ------------------------------------------------------------------

2 1

Allow skippig the font cache in font.getfont
by Marcel Fabian Krüger 12 Dec '19

12 Dec '19

Hello, could font.getfont get an option to always return a new font table instead of returning the current one? While for many fonts a cached table makes sense, sometimes it would be useful to also have access to the ooriginal table, e.g. to see which characters are `used`, inspect the result of `font.addcharacters` or inspect other changes which might be applied to the underlying font from TeX. I attached a patch implementing this by adding a `nocache` parameter to `font.getfont`, such that `font.getfont(true, font_id)` always generates a new table. IMO `font.getfont(font_id, true)` would look nicer, but that wouldn't be compatible to older LuaTeX versions which always expect the font is as last parameter. Best regards, Marcel

2 1

commands with a non-letter in token.set_macro
by Ulrike Fischer 11 Dec '19

11 Dec '19

Commands that should be use in token.set_macro must already exist. But while it is easy to predefine a command like \blub it seems not to be possible to do it with commands consisting of a \ with a non-letter. E.g. I put this in a luatest.lua token.set_macro("foo","\\bar\\blub\\!") and then compile \def\!{hallo} \def\blub{hallo} \directlua{require"luatest"} \show\foo \bye I get: > \foo=macro: ->\bar \blub BAD!. l.254 \show\foo so the \blub works, but not the \!. How can one repair this? -- Ulrike Fischer https://www.troubleshooting-tex.de/

2 1

Re: [Dev-luatex] Request for bugfix - user defined whatsit node
by luigi scarso 05 Dec '19

05 Dec '19

On Tue, Dec 3, 2019 at 6:16 PM Hans Hagen <j.hagen(a)xs4all.nl> wrote: > On 11/15/2019 12:35 PM, Sigitas Tolusis wrote: > > Noticed two bugs. > > > > 1. Mised case for user defined whatsit node type 108 (‘l’ Lua value) > > in function check_node_wrapup_core (texnodes.c:2482). > > Possible fix: just add the same as cases for types ’s’ and ‘d’. > > Insert line at texnodes.c:2500 > > case ‘l': > > > > 2. In function flush_node_wrapup_core (texnodes.c:2201) for user defined > > whatsit node type ‘l’ we have: free_user_lua(user_node_value(p)). > > But in luanode.c:409 free_user_lua is defined as: > > > > void free_user_lua(pointer p) > > { > > if (user_node_value(p) != 0) { > > luaL_unref(Luas, LUA_REGISTRYINDEX, user_node_value(p)); > > } > > } > > > > So we have two times user_node_value and it generates segmentation fault. > > Possible fix: free_user_lua(p); // (texnodes.c:2223) > fixed in next luatex beta buit best test then > > Hans > Done in rev. 7233 . -- luigi

1 0

Revive Type 3 fonts
by Marcel Fabian Krüger 15 Nov '19

15 Nov '19

Hey, LuaTeX currently supports only a very special kind of Type 3 fonts: They are generated for traditional TeX bitmap fonts. While this used to be the most common use case for Type3 fonts, the fully general concept of these fonts has recently been reinvented under the name of SVG fonts (and friends). While they can be implemented in LuaTeX to some degree through virtual fonts/node processing etc., converting them into "real" Type 3 fonts in the PDF file would give some important advantages: Such fonts are called like other fonts and can get tounicode vectors, allowing better accessibillity and better Copy&Paste behaviour. Also viewers generally treat them as text instead of graphics, allowing proper interaction with features of PDF viewers. Beside from supporting SVG fonts, it also allows to use general MetaPost fonts which might not be easily convertable into normal formats for vector fonts. As far as I can tell there used to be an implementation of general Type 3 fonts for pdfTeX which was dropped in LuaTeX, mostly because it was never actually used. My suggested new implementation is a bit more flexible by not requiring external files defining the font but instead generating each font character based on individual node lists for every glyph. One way to make this work would be to add a special `format` named e.g. `node` (given that `type3` is already used to imply bitmap fonts) and add a field `node`/`list`/`head` to the character table referencing a node list which should become the font glyph. The detaily like bounding box calculations etc. would be the responsibility of the Lua code loading the font. The intended usage would be e.g. \directlua{ local l = token.scan_list() local wi = node.new('whatsit', 'pdf_literal') wi.mode = 3 wi.data = math.floor(l.width / tex.sp'1bp') .. ' 0 d0' wi.next = l.head l.head = wi font.current(font.define{ name = "exp-nodefont", format = "node", designsize = tex.sp("10pt"), size = tex.sp("10pt"), characters = { [42] = { width = l.width, height = l.height, depth = l.depth, node = node.direct.todirect(l), } } })}\hbox{ABC} \char42 \char42 \bye What do you think? (I'll attach a patch I used for experiments (mostly with MetaPost fonts) which implements this in a slightly hacky way.) -- Marcel

6 7

Improvement for `glyph_info`
by Marcel Fabian Krüger 15 Nov '19

15 Nov '19

Hi, the new `glyph_info` callback is called in two pretty different cases: in `short_display(_n)` where a pretty short representation of the actual output characters is required, and in `print_font_and_char`, which creates the representation for `print_node_list`. Here, more details can be useful, e.g.: Which variant of a character is used, are displacements applied etc. These could be added through `glyph_info`, but then these additional fields are also printed in `short_display`, leading to significantly worse overfull warnings etc. This could be improved if `glyph_info` could return different values in different situations. So I suggest to add an additional parameter to the `glyph_info` callback which states if the current context could handle longer output. A possible implementation is attached. Best regards, Marcel

4 4