November 2018 - dev-luatex - NTG Mailing Lists

Luatex 1.08.0 announcement
by luigi scarso 28 Oct '19

28 Oct '19

============================================================== LuaTeX 1.08 2018-08-28 ============================================================== (1) This release is a prelude to 1.10, the next stable iteration of LuaTeX after version 1.00. (2) Lua 5.3 is now considered to be default and we might use 5.4 in version 1.10. There are no real functional changed expected. You still need to rename the binary for 5.3! (3) Binary mode is no longer available in MPlib but it is still available in stand alone MetaPost. This simplifies compilation and reduces dependencies. (4) The dependency on Poppler for pdf image inclusion has been removed. We now use a small dedicated library written by Pawel Jakowski. We no longer need c++ compilers. We're in the process of making it behave well on all platforms. It has been tested on intel platforms. (5) We know that there can be some (alignment) issues with the arm platform but these are looked into. Therefore, later this year we will release 1.09. Version 1.10 is planned for TeXlive. We hope that ffi works ok on intel and arm platforms at that point. (6) There have been some extensions to the Lua libraries and some callbacks have been added. Also, a few new primitives have been introduced. The documentation mentions the stable extensions. (7) There are the usual bug fixes and cleanups but there have been no real fundamental changes in the API. The LuaTeX team

2 5

Not searching the font cache
by Jouke Witteveen 08 Jan '19

08 Jan '19

Hi, I ran into an inconvenience with lualatex and would like some help debugging it. Consider the following .tex file: ---- documentclass{article} \usepackage{bbm} \begin{document} $\mathbbm{1}$ \end{document} ---- When compiling with pdflatex, the font file for the bbm font is created in the local cache (~/.texlive/texmf-var/fonts) only if it does not exist. When the font exists in the cache, it is used without warning. When compiling with lualatex, the font file is created every time. When the font exists, this results in a simple warning ("mktexpk: [...] already exists."). Trying to create the fonts slows down compilation noticeably. How do I go about debugging this? The fact that pdflatex has no problems suggests that this is not an issue with my texmf.cnf, right? Regards, - Jouke

2 3

Error reading fonts with version 0 OS/2 table
by Khaled Hosny 27 Nov '18

27 Nov '18

Hi all, LuaTeX stops with an error when reading version 0 OS/2 table, and this is is because it unconditionally reads ulCodePageRange1 and ulCodePageRange2 fields that were not defined in version 0. Such fonts can be found here [1] (from the question [2]). I’m attaching a patch that makes this conditional, it also fixed reading version 2 specific fields that would be skipped for tables with versions greater than 2. It does not look like these fields are used anywhere, so another fix is to only read the fields LuaTeX actually needs. Regards, Khaled 1. https://mp.uu.se/documents/432512/911262/18350_Typsnitt_PC.zip/4c0954cf-84b… 2. https://tex.stackexchange.com/q/430837/729

1 0

Recalculating tail of the list in ligaturing callback
by Esger Renkema 16 Nov '18

16 Nov '18

Dear list, Running the following one-liner: \directlua{callback.register('ligaturing', node.ligaturing)}ffi\bye results in: ------------------------------------------- This is LuaTeX, Version 1.09.0 (TeX Live 2019/dev/Debian) restricted system commands enabled. **\directlua{callback.register('ligaturing', node.ligaturing)}ffi\bye ! error: (linebreak): invalid list tail, probably missing glue ! ==> Fatal error occurred, no output PDF file produced! ------------------------------------------- The problem seems to be that, when restoring the original ‘tail of the list’ after applying the ligaturing callback (in this case consisting of the parfillskip glue), it is appended to the tail of the /original/ list-to-be-ligatured. The latter, of course, has been lost if the paragraph ends in a ligature. According to the manual, the first node of the list passed to the callback is guaranteed not to be a glyph_node. Should the same not hold for the last one, too? The manual states that it ‘normally’ is a glue, but that is clearly not the case here. Best, Esger

3 5

Reading tounicode from shared fonts
by Khaled Hosny 14 Nov '18

14 Nov '18

Hi, While debugging an issue with PDF /ToUnicode, I found code for reading character-level tounicode from the fonts, always takes them from the first font in fonts in merges in the PDF. Usually this is not an issue since the font loader will have identical tounicode values for all characters in the same font loaded multiple times. However, my code sets character-level tounicode only after processing the nodes (to avoid parsing the cmap and GSUB tables ahead of typesetting), so the same font loaded multiple times can have different tounicodes and characters used only in later instances of the font will not have their tounicodes in the PDF file. It seems to be one character fix to instead check the font the character is used in, and looks to me like a typo since the first revision this code was introduced (r710) that went unnoticed. Patch attached. Regards, Khaled

2 1

255 characters limit on font names
by Khaled Hosny 13 Nov '18

13 Nov '18

Hi all, Currently fonts with names are longer than 255 characters are ignored and the font fails to load without an explanation. See https://github.com/lualatex/luaotfload/issues/424 for background. The code suggests this is a limitation of the DVI format (the relevant check was add back in 2009 according to repository history), but it shouldn’t affect PDF mode. The attached patch restricts the check to DVI mode and adds an error message. Alternatively the font name can be just truncated in DVI mode (which what happens if the check is removed, but the extra characters are removed from the start of the name, no idea why) an just issue a warning, or use the file name for DVI format since the rest of the spec is irrelevant to DVI files (but this can be debatable, I don’t know if there are any DVI drivers that can parse LuaTeX font names and load the fonts). Regards, Khaled

2 3

Re: [Dev-luatex] Callbacks in Lua wrapper of mplib
by Marcel Krüger 08 Nov '18

08 Nov '18

---- On Wed, 07 Nov 2018 13:00:50 +0100 luigi scarso <luigi.scarso(a)gmail.com> wrote ---- > > > On Wed, Nov 7, 2018 at 12:18 PM Marcel Krüger <tex(a)2krueger.de> wrote: > Hi, > > I recently discovered that the Lua wrapper of mplib registers all callbacks globally > such that creating multiple mplib instances with different callbacks breaks. > > > why not this ? > \directlua{ local current_instance local custom_find_file = custom_find_file or {} local mp_find_file_driver = function(name, mode, type) print("I am ",current_instance) if custom_find_file[current_instance] then return custom_find_file[current_instance](name, mode, type) else return "unregistered" end end custom_find_file['mp1'] = function(name, mode, type) return name end custom_find_file['mp2'] = function(name, mode, type) if name == 'cmr10' then print'This should not happen' end return name end > local mp1 = mplib.new{ find_file = mp_find_file_driver } > local mp2 = mplib.new{ find_file = mp_find_file_driver } current_instance = "mp1" mp1:execute[[ input cmr10; ]] > current_instance = "mp2" mp2:execute[[ input cmr10; ]] > > } \bye That's similar to the workaround I'm using right now, but it seems odd: The callbacks are given as in the initializer table of a single instance, so you do not expect them to affect other instances. Especially this means that if anyone uses "short-living" MPLib instances, this person will probably not expect this behaviour and just pass the find_file method directly. This means that every user of "long-living" MPLib instances has to fear that every called library might change this global state. If this happens, there does not even seem to be a way to change it back, so the safest approach would be to call mplib.new and create and destroy a temporary instance every time `:execute` or `:finish` is called. So if this is the intended behaviour, IMO a note should be added to the documentation. Especially because every non-function entry in the initializer only affects this instance. But I still think it should be changed: The current behaviour seems unintuitive and inconsistant. Also manually keeping track of the current instance seems at least odd if mplib can access this information anyway. Additionally the change would make the Lua API of mplib more similar to the C API. Best regards Marcel Krüger

2 1

Callbacks in Lua wrapper of mplib
by Marcel Krüger 07 Nov '18

07 Nov '18

Hi, I recently discovered that the Lua wrapper of mplib registers all callbacks globally such that creating multiple mplib instances with different callbacks breaks. This affects the undocumented `run_script` and `make_text` callbacks, but also the documented `find_file`: For example the plain LuaTeX document \directlua{ local mp = mplib.new{ find_file = function(name, mode, type) return name end } do % Create a temporary second instance with a different run_script handler mplib.new{ find_file = function(name, mode, type) if name == 'cmr10' then print'This should not happen' end return name end }:finish() end % Now trigger find_file using the *first* mp instance which should *not* trigger a message mp:execute[[ input cmr10; ]] } \bye shows the message 'This should not happen', because the find_file implementation of the second call to `mplib.new` is also used for the first `mp` instance. This can be fixed by storing the callbacks in userdata associated with the mp uservalue instead of in the global registry: diff --git a/source/texk/web2c/mplibdir/lmplib.c b/source/texk/web2c/mplibdir/lmplib.c index eb8432bb5..88873ed66 100644 --- a/source/texk/web2c/mplibdir/lmplib.c +++ b/source/texk/web2c/mplibdir/lmplib.c @@ -329,7 +329,9 @@ static char *mplib_find_file(MP mp, const char *fname, const char *fmode, int ft { lua_State *L = (lua_State *)mp_userdata(mp); lua_checkstack(L, 4); - lua_getfield(L, LUA_REGISTRYINDEX, "mplib.file_finder"); + lua_getuservalue(L, 1); + lua_getfield(L, -1, "file_finder"); + lua_replace(L, -2); if (lua_isfunction(L, -1)) { char *s = NULL; const char *x = NULL; @@ -365,9 +367,9 @@ static int mplib_find_file_function(lua_State * L) /*tex An error. */ return 1; } - lua_pushstring(L, "mplib.file_finder"); + lua_pushstring(L, "file_finder"); lua_pushvalue(L, -2); - lua_rawset(L, LUA_REGISTRYINDEX); + lua_rawset(L, -4); return 0; } @@ -379,8 +381,10 @@ static void mplib_warning(const char *str) static void mplib_script_error(MP mp, const char *str) { lua_State *L = (lua_State *)mp_userdata(mp); - lua_checkstack(L, 1); - lua_getfield(L, LUA_REGISTRYINDEX, "mplib.script_error"); + lua_checkstack(L, 2); + lua_getuservalue(L, 1); + lua_getfield(L, -1, "script_error"); + lua_replace(L, -2); if (lua_isfunction(L, -1)) { lua_pushstring(L, str); /*tex We assume that the function is okay. */ @@ -397,17 +401,19 @@ static int mplib_script_error_function(lua_State * L) /*tex An error. */ return 1; } - lua_pushstring(L, "mplib.script_error"); + lua_pushstring(L, "script_error"); lua_pushvalue(L, -2); - lua_rawset(L, LUA_REGISTRYINDEX); + lua_rawset(L, -4); return 0; } static char *mplib_run_script(MP mp, const char *str) { lua_State *L = (lua_State *)mp_userdata(mp); - lua_checkstack(L, 1); - lua_getfield(L, LUA_REGISTRYINDEX, "mplib.run_script"); + lua_checkstack(L, 2); + lua_getuservalue(L, 1); + lua_getfield(L, -1, "run_script"); + lua_replace(L, -2); if (lua_isfunction(L, -1)) { char *s = NULL; const char *x = NULL; @@ -433,17 +439,19 @@ static int mplib_run_script_function(lua_State * L) if (!(lua_isfunction(L, -1) || lua_isnil(L, -1))) { return 1; /* error */ } - lua_pushstring(L, "mplib.run_script"); + lua_pushstring(L, "run_script"); lua_pushvalue(L, -2); - lua_rawset(L, LUA_REGISTRYINDEX); + lua_rawset(L, -4); return 0; } static char *mplib_make_text(MP mp, const char *str, int mode) { lua_State *L = (lua_State *)mp_userdata(mp); - lua_checkstack(L, 1); - lua_getfield(L, LUA_REGISTRYINDEX, "mplib.make_text"); + lua_checkstack(L, 2); + lua_getuservalue(L, 1); + lua_getfield(L, -1, "make_text"); + lua_replace(L, -2); if (lua_isfunction(L, -1)) { char *s = NULL; const char *x = NULL; @@ -471,9 +479,9 @@ static int mplib_make_text_function(lua_State * L) /*tex An error. */ return 1; } - lua_pushstring(L, "mplib.make_text"); + lua_pushstring(L, "make_text"); lua_pushvalue(L, -2); - lua_rawset(L, LUA_REGISTRYINDEX); + lua_rawset(L, -4); return 0; } @@ -544,6 +552,8 @@ static int mplib_new(lua_State * L) /* options->script_error = mplib_script_error; */ options->print_found_names = 1; options->ini_version = 1; + /*tex Here we create the uservalue table for the callbacks: */ + lua_newtable(L); if (lua_type(L, 1) == LUA_TTABLE) { for (i = 0; mplib_parms[i].name != NULL; i++) { lua_getfield(L, 1, mplib_parms[i].name); @@ -606,6 +616,7 @@ static int mplib_new(lua_State * L) lua_pop(L, 1); } } + lua_setuservalue(L, -2); *mp_ptr = mp_initialize(options); xfree(options->command_line); xfree(options->mem_name);

2 1