- dev-luatex - NTG Mailing Lists

commands with a non-letter in token.set_macro
by Ulrike Fischer 10 Dec '19

10 Dec '19

Commands that should be use in token.set_macro must already exist. But while it is easy to predefine a command like \blub it seems not to be possible to do it with commands consisting of a \ with a non-letter. E.g. I put this in a luatest.lua token.set_macro("foo","\\bar\\blub\\!") and then compile \def\!{hallo} \def\blub{hallo} \directlua{require"luatest"} \show\foo \bye I get: > \foo=macro: ->\bar \blub BAD!. l.254 \show\foo so the \blub works, but not the \!. How can one repair this? -- Ulrike Fischer https://www.troubleshooting-tex.de/

2 1

Re: [Dev-luatex] Request for bugfix - user defined whatsit node
by luigi scarso 05 Dec '19

05 Dec '19

On Tue, Dec 3, 2019 at 6:16 PM Hans Hagen <j.hagen(a)xs4all.nl> wrote: > On 11/15/2019 12:35 PM, Sigitas Tolusis wrote: > > Noticed two bugs. > > > > 1. Mised case for user defined whatsit node type 108 (‘l’ Lua value) > > in function check_node_wrapup_core (texnodes.c:2482). > > Possible fix: just add the same as cases for types ’s’ and ‘d’. > > Insert line at texnodes.c:2500 > > case ‘l': > > > > 2. In function flush_node_wrapup_core (texnodes.c:2201) for user defined > > whatsit node type ‘l’ we have: free_user_lua(user_node_value(p)). > > But in luanode.c:409 free_user_lua is defined as: > > > > void free_user_lua(pointer p) > > { > > if (user_node_value(p) != 0) { > > luaL_unref(Luas, LUA_REGISTRYINDEX, user_node_value(p)); > > } > > } > > > > So we have two times user_node_value and it generates segmentation fault. > > Possible fix: free_user_lua(p); // (texnodes.c:2223) > fixed in next luatex beta buit best test then > > Hans > Done in rev. 7233 . -- luigi

1 0

Revive Type 3 fonts
by Marcel Fabian Krüger 15 Nov '19

15 Nov '19

Hey, LuaTeX currently supports only a very special kind of Type 3 fonts: They are generated for traditional TeX bitmap fonts. While this used to be the most common use case for Type3 fonts, the fully general concept of these fonts has recently been reinvented under the name of SVG fonts (and friends). While they can be implemented in LuaTeX to some degree through virtual fonts/node processing etc., converting them into "real" Type 3 fonts in the PDF file would give some important advantages: Such fonts are called like other fonts and can get tounicode vectors, allowing better accessibillity and better Copy&Paste behaviour. Also viewers generally treat them as text instead of graphics, allowing proper interaction with features of PDF viewers. Beside from supporting SVG fonts, it also allows to use general MetaPost fonts which might not be easily convertable into normal formats for vector fonts. As far as I can tell there used to be an implementation of general Type 3 fonts for pdfTeX which was dropped in LuaTeX, mostly because it was never actually used. My suggested new implementation is a bit more flexible by not requiring external files defining the font but instead generating each font character based on individual node lists for every glyph. One way to make this work would be to add a special `format` named e.g. `node` (given that `type3` is already used to imply bitmap fonts) and add a field `node`/`list`/`head` to the character table referencing a node list which should become the font glyph. The detaily like bounding box calculations etc. would be the responsibility of the Lua code loading the font. The intended usage would be e.g. \directlua{ local l = token.scan_list() local wi = node.new('whatsit', 'pdf_literal') wi.mode = 3 wi.data = math.floor(l.width / tex.sp'1bp') .. ' 0 d0' wi.next = l.head l.head = wi font.current(font.define{ name = "exp-nodefont", format = "node", designsize = tex.sp("10pt"), size = tex.sp("10pt"), characters = { [42] = { width = l.width, height = l.height, depth = l.depth, node = node.direct.todirect(l), } } })}\hbox{ABC} \char42 \char42 \bye What do you think? (I'll attach a patch I used for experiments (mostly with MetaPost fonts) which implements this in a slightly hacky way.) -- Marcel

6 7

Improvement for `glyph_info`
by Marcel Fabian Krüger 15 Nov '19

15 Nov '19

Hi, the new `glyph_info` callback is called in two pretty different cases: in `short_display(_n)` where a pretty short representation of the actual output characters is required, and in `print_font_and_char`, which creates the representation for `print_node_list`. Here, more details can be useful, e.g.: Which variant of a character is used, are displacements applied etc. These could be added through `glyph_info`, but then these additional fields are also printed in `short_display`, leading to significantly worse overfull warnings etc. This could be improved if `glyph_info` could return different values in different situations. So I suggest to add an additional parameter to the `glyph_info` callback which states if the current context could handle longer output. A possible implementation is attached. Best regards, Marcel

4 4

require doesn't get recorded
by Marcel Fabian Krüger 08 Nov '19

08 Nov '19

Hi, based on an observation by Karl Berry on the TeX Live list I discovered an inconsistency in LuaTeX's handling of `-recorder`: While LuaTeX does record files loaded through `loadfile`, files loaded through `require` are ignored: For example if the current directory contains two empty files l1.lua and l2.lua, then the document \directlua{ loadfile'l1.lua'() require'l2' } \bye compiled with `luatex -recorder recorded` IMO should write to recorded.fls the entries PWD /some/weird/pwd INPUT /usr/local/texlive/2019/texmf-var/web2c/luatex/luatex.fmt INPUT ./recorded.tex OUTPUT recorded.log INPUT l1.lua INPUT ./l2.lua but it does write PWD /some/weird/pwd INPUT /usr/local/texlive/2019/texmf-var/web2c/luatex/luatex.fmt INPUT ./recorded.tex OUTPUT recorded.log INPUT l1.lua I do not think that this is intentional, so I added a patch to add recorder entries iff the file is loaded successfully (similar to `loadfile`) -- Marcel

3 6

Re: [Dev-luatex] Broken \adjdemerits on Linux due to unspecified enumeration type
by Marcel Fabian Krüger 05 Nov '19

05 Nov '19

Hi, thanks for fixing the \adjdemerits problem. One remark about the fix: The line if ( (fit_class>(fitness(r)+1)) || (fitness(r)>(fit_class+1)) ) is indented by one tab "\t" and four spaces, while all surrounding code is only indented with spaces. (Confusingly, tabs are also used as indentation in line 117-119) This leads to confusing appearence if the file is viewed in an editor which does not use 8 spaces for one tab. Maybe this could be changed to be a bit more consistant? Best regards, Marcel

2 1

TTC handling
by Marcel Fabian Krüger 31 Oct '19

31 Oct '19

Hey, in `writefont0.c` and `writefont2.c`, `ff_get_ttc_index` is called to determine the index of the font in a font collection. This uses the original filename from fd->fm->ff_name and therefore does not respect lookups done using the find_opentype_file etc. callbacks. In writefont0.c, this could be fixed relativly easily by passing using cur_file_name, the name after the callback, instead. But even than there are remianing problems because this still wouldn't respect read_..._file callbacks. So implementing this correctly is quite tricky and it also isn't really necessary in most cases: Especially when using Lua-loaded fonts, the code creating the font needs to know the index anyway. So maybe an approach like in Khaled Hosny's HarfTeX could be used: Allow a Lua-loaded font to explicitly set the index and only use ff_get_ttc_index if this index isn't set. Then backward-compatibility is kept but the actually used index selection can be influenced using Lua. (HarfTeX implemented this in https://github.com/khaledhosny/harftex/commit/caded3e80f106687c84f9613aa318…) Best regards, Marcel

2 1

Broken \adjdemerits on Linux due to unspecified enumeration type
by Marcel Fabian Krüger 30 Oct '19

30 Oct '19

Hi, current versions of LuaTeX from TeXLive for Linux (Windows is not affected) always add `\adjdemerits` when a line is more loose than the previous one, even if it is only one step more loose: Take \tracingparagraphs1 \tracingonline1 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa \bye This gives @firstpass []\tenrm aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @ via @0 b=96 p=0 d=1011236 @@1: line 1.1 t=21236 -> @0 aaaaaa @ via @0 b=83 p=0 d=8649 @@2: line 1.3 t=8649 -> @0 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa aa aaaa aaaaaa @ via @1 b=3 p=0 d=169 @@3: line 2.2 t=1011405 -> @1 aaaaaa @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 aaaaaa aaaaaa aaaaaa aaaaaa aaaaaa @\par via @3 b=0 p=-10000 d=100 @\par via @4 b=0 p=-10000 d=100 @@5: line 3.2- t=1008918 -> @4 Here, the relevant part is @@2: line 1.3 t=8649 -> @0 ... @ via @2 b=3 p=0 d=1000169 @@4: line 2.2 t=1008818 -> @2 So breakpoint @2 is .3 aka. "tight", while @4 is .2 aka. "decent". This break gives demerits d=100169=1000000+(10+3)^2. So \adjdemerits are inserted which is wrong because tight and decent lines are "visually compatible" as defined in the TeXbook. The problem is in linebreak.c. There the demerits are inserted by if (abs(fit_class - fitness(r)) > 1) d = d + adj_demerits; Both fit_class and fitness(r) have type fitness_value which is an enum with four entries. These enums have an implementation defined type and the bug is triggered if LuaTeX is compiled with a compiler which uses an unsigned type: Then `fit_class - fitness(r)` is a difference of unsigned values and therefore unsigned. If fitness(r) is bigger than fit_class, the attempt to store a negative difference in a unsigned value wraps around to a high positive value which is bigger than 1. So if fitness(r) is bigger than fit_class, the condition is always true. This can be fixed by explicitly casting fit_class and fitness(r) to a signed type. A possible patch is attached. Best regards, Marcel

3 4

Luatex 1.08.0 announcement
by luigi scarso 28 Oct '19

28 Oct '19

============================================================== LuaTeX 1.08 2018-08-28 ============================================================== (1) This release is a prelude to 1.10, the next stable iteration of LuaTeX after version 1.00. (2) Lua 5.3 is now considered to be default and we might use 5.4 in version 1.10. There are no real functional changed expected. You still need to rename the binary for 5.3! (3) Binary mode is no longer available in MPlib but it is still available in stand alone MetaPost. This simplifies compilation and reduces dependencies. (4) The dependency on Poppler for pdf image inclusion has been removed. We now use a small dedicated library written by Pawel Jakowski. We no longer need c++ compilers. We're in the process of making it behave well on all platforms. It has been tested on intel platforms. (5) We know that there can be some (alignment) issues with the arm platform but these are looked into. Therefore, later this year we will release 1.09. Version 1.10 is planned for TeXlive. We hope that ffi works ok on intel and arm platforms at that point. (6) There have been some extensions to the Lua libraries and some callbacks have been added. Also, a few new primitives have been introduced. The documentation mentions the stable extensions. (7) There are the usual bug fixes and cleanups but there have been no real fundamental changes in the API. The LuaTeX team

2 5

mlist_to_hlist restriction
by Linas Stonys 02 Oct '19

02 Oct '19

Hi luatex team, mlist_to_hlist callback requires to return a hlist. And that is a big problem because there is no possibility to use this callback if some style already used it. If two different styles uses this callback luatex always throws error "! This can't happen (mlist1)." because mlist is already converted to hlist for second callback. I want to offer a new callback "mlist_to_mlist" were one could access math list without conversion to hlist. Usually if one uses this callback he needs mlist and not a hlist. And probably would be even better to have one more callback "mhpack_filter" which could access result of node.milst_to_hlist(math list converted hlist). So mlist_to_hlist callback would be splited into to parts like: m = list_of_mlist_to_mlist_callbacks (h) n = node.mlis_to_hlist(m) x = list_of_mhpack_filter_callbacks (n) return x What do you think? Linas

2 4