- dev-luatex - Mailing Lists

Recalculating tail of the list in ligaturing callback
by Esger Renkema 16 Nov '18

16 Nov '18

Dear list, Running the following one-liner: \directlua{callback.register('ligaturing', node.ligaturing)}ffi\bye results in: ------------------------------------------- This is LuaTeX, Version 1.09.0 (TeX Live 2019/dev/Debian) restricted system commands enabled. **\directlua{callback.register('ligaturing', node.ligaturing)}ffi\bye ! error: (linebreak): invalid list tail, probably missing glue ! ==> Fatal error occurred, no output PDF file produced! ------------------------------------------- The problem seems to be that, when restoring the original ‘tail of the list’ after applying the ligaturing callback (in this case consisting of the parfillskip glue), it is appended to the tail of the /original/ list-to-be-ligatured. The latter, of course, has been lost if the paragraph ends in a ligature. According to the manual, the first node of the list passed to the callback is guaranteed not to be a glyph_node. Should the same not hold for the last one, too? The manual states that it ‘normally’ is a glue, but that is clearly not the case here. Best, Esger

3 5

Reading tounicode from shared fonts
by Khaled Hosny 14 Nov '18

14 Nov '18

Hi, While debugging an issue with PDF /ToUnicode, I found code for reading character-level tounicode from the fonts, always takes them from the first font in fonts in merges in the PDF. Usually this is not an issue since the font loader will have identical tounicode values for all characters in the same font loaded multiple times. However, my code sets character-level tounicode only after processing the nodes (to avoid parsing the cmap and GSUB tables ahead of typesetting), so the same font loaded multiple times can have different tounicodes and characters used only in later instances of the font will not have their tounicodes in the PDF file. It seems to be one character fix to instead check the font the character is used in, and looks to me like a typo since the first revision this code was introduced (r710) that went unnoticed. Patch attached. Regards, Khaled

2 1

255 characters limit on font names
by Khaled Hosny 13 Nov '18

13 Nov '18

Hi all, Currently fonts with names are longer than 255 characters are ignored and the font fails to load without an explanation. See https://github.com/lualatex/luaotfload/issues/424 for background. The code suggests this is a limitation of the DVI format (the relevant check was add back in 2009 according to repository history), but it shouldn’t affect PDF mode. The attached patch restricts the check to DVI mode and adds an error message. Alternatively the font name can be just truncated in DVI mode (which what happens if the check is removed, but the extra characters are removed from the start of the name, no idea why) an just issue a warning, or use the file name for DVI format since the rest of the spec is irrelevant to DVI files (but this can be debatable, I don’t know if there are any DVI drivers that can parse LuaTeX font names and load the fonts). Regards, Khaled

2 3

Re: [Dev-luatex] Callbacks in Lua wrapper of mplib
by Marcel Krüger 08 Nov '18

08 Nov '18

---- On Wed, 07 Nov 2018 13:00:50 +0100 luigi scarso <luigi.scarso(a)gmail.com> wrote ---- > > > On Wed, Nov 7, 2018 at 12:18 PM Marcel Krüger <tex(a)2krueger.de> wrote: > Hi, > > I recently discovered that the Lua wrapper of mplib registers all callbacks globally > such that creating multiple mplib instances with different callbacks breaks. > > > why not this ? > \directlua{ local current_instance local custom_find_file = custom_find_file or {} local mp_find_file_driver = function(name, mode, type) print("I am ",current_instance) if custom_find_file[current_instance] then return custom_find_file[current_instance](name, mode, type) else return "unregistered" end end custom_find_file['mp1'] = function(name, mode, type) return name end custom_find_file['mp2'] = function(name, mode, type) if name == 'cmr10' then print'This should not happen' end return name end > local mp1 = mplib.new{ find_file = mp_find_file_driver } > local mp2 = mplib.new{ find_file = mp_find_file_driver } current_instance = "mp1" mp1:execute[[ input cmr10; ]] > current_instance = "mp2" mp2:execute[[ input cmr10; ]] > > } \bye That's similar to the workaround I'm using right now, but it seems odd: The callbacks are given as in the initializer table of a single instance, so you do not expect them to affect other instances. Especially this means that if anyone uses "short-living" MPLib instances, this person will probably not expect this behaviour and just pass the find_file method directly. This means that every user of "long-living" MPLib instances has to fear that every called library might change this global state. If this happens, there does not even seem to be a way to change it back, so the safest approach would be to call mplib.new and create and destroy a temporary instance every time `:execute` or `:finish` is called. So if this is the intended behaviour, IMO a note should be added to the documentation. Especially because every non-function entry in the initializer only affects this instance. But I still think it should be changed: The current behaviour seems unintuitive and inconsistant. Also manually keeping track of the current instance seems at least odd if mplib can access this information anyway. Additionally the change would make the Lua API of mplib more similar to the C API. Best regards Marcel Krüger

2 1

Callbacks in Lua wrapper of mplib
by Marcel Krüger 07 Nov '18

07 Nov '18

Hi, I recently discovered that the Lua wrapper of mplib registers all callbacks globally such that creating multiple mplib instances with different callbacks breaks. This affects the undocumented `run_script` and `make_text` callbacks, but also the documented `find_file`: For example the plain LuaTeX document \directlua{ local mp = mplib.new{ find_file = function(name, mode, type) return name end } do % Create a temporary second instance with a different run_script handler mplib.new{ find_file = function(name, mode, type) if name == 'cmr10' then print'This should not happen' end return name end }:finish() end % Now trigger find_file using the *first* mp instance which should *not* trigger a message mp:execute[[ input cmr10; ]] } \bye shows the message 'This should not happen', because the find_file implementation of the second call to `mplib.new` is also used for the first `mp` instance. This can be fixed by storing the callbacks in userdata associated with the mp uservalue instead of in the global registry: diff --git a/source/texk/web2c/mplibdir/lmplib.c b/source/texk/web2c/mplibdir/lmplib.c index eb8432bb5..88873ed66 100644 --- a/source/texk/web2c/mplibdir/lmplib.c +++ b/source/texk/web2c/mplibdir/lmplib.c @@ -329,7 +329,9 @@ static char *mplib_find_file(MP mp, const char *fname, const char *fmode, int ft { lua_State *L = (lua_State *)mp_userdata(mp); lua_checkstack(L, 4); - lua_getfield(L, LUA_REGISTRYINDEX, "mplib.file_finder"); + lua_getuservalue(L, 1); + lua_getfield(L, -1, "file_finder"); + lua_replace(L, -2); if (lua_isfunction(L, -1)) { char *s = NULL; const char *x = NULL; @@ -365,9 +367,9 @@ static int mplib_find_file_function(lua_State * L) /*tex An error. */ return 1; } - lua_pushstring(L, "mplib.file_finder"); + lua_pushstring(L, "file_finder"); lua_pushvalue(L, -2); - lua_rawset(L, LUA_REGISTRYINDEX); + lua_rawset(L, -4); return 0; } @@ -379,8 +381,10 @@ static void mplib_warning(const char *str) static void mplib_script_error(MP mp, const char *str) { lua_State *L = (lua_State *)mp_userdata(mp); - lua_checkstack(L, 1); - lua_getfield(L, LUA_REGISTRYINDEX, "mplib.script_error"); + lua_checkstack(L, 2); + lua_getuservalue(L, 1); + lua_getfield(L, -1, "script_error"); + lua_replace(L, -2); if (lua_isfunction(L, -1)) { lua_pushstring(L, str); /*tex We assume that the function is okay. */ @@ -397,17 +401,19 @@ static int mplib_script_error_function(lua_State * L) /*tex An error. */ return 1; } - lua_pushstring(L, "mplib.script_error"); + lua_pushstring(L, "script_error"); lua_pushvalue(L, -2); - lua_rawset(L, LUA_REGISTRYINDEX); + lua_rawset(L, -4); return 0; } static char *mplib_run_script(MP mp, const char *str) { lua_State *L = (lua_State *)mp_userdata(mp); - lua_checkstack(L, 1); - lua_getfield(L, LUA_REGISTRYINDEX, "mplib.run_script"); + lua_checkstack(L, 2); + lua_getuservalue(L, 1); + lua_getfield(L, -1, "run_script"); + lua_replace(L, -2); if (lua_isfunction(L, -1)) { char *s = NULL; const char *x = NULL; @@ -433,17 +439,19 @@ static int mplib_run_script_function(lua_State * L) if (!(lua_isfunction(L, -1) || lua_isnil(L, -1))) { return 1; /* error */ } - lua_pushstring(L, "mplib.run_script"); + lua_pushstring(L, "run_script"); lua_pushvalue(L, -2); - lua_rawset(L, LUA_REGISTRYINDEX); + lua_rawset(L, -4); return 0; } static char *mplib_make_text(MP mp, const char *str, int mode) { lua_State *L = (lua_State *)mp_userdata(mp); - lua_checkstack(L, 1); - lua_getfield(L, LUA_REGISTRYINDEX, "mplib.make_text"); + lua_checkstack(L, 2); + lua_getuservalue(L, 1); + lua_getfield(L, -1, "make_text"); + lua_replace(L, -2); if (lua_isfunction(L, -1)) { char *s = NULL; const char *x = NULL; @@ -471,9 +479,9 @@ static int mplib_make_text_function(lua_State * L) /*tex An error. */ return 1; } - lua_pushstring(L, "mplib.make_text"); + lua_pushstring(L, "make_text"); lua_pushvalue(L, -2); - lua_rawset(L, LUA_REGISTRYINDEX); + lua_rawset(L, -4); return 0; } @@ -544,6 +552,8 @@ static int mplib_new(lua_State * L) /* options->script_error = mplib_script_error; */ options->print_found_names = 1; options->ini_version = 1; + /*tex Here we create the uservalue table for the callbacks: */ + lua_newtable(L); if (lua_type(L, 1) == LUA_TTABLE) { for (i = 0; mplib_parms[i].name != NULL; i++) { lua_getfield(L, 1, mplib_parms[i].name); @@ -606,6 +616,7 @@ static int mplib_new(lua_State * L) lua_pop(L, 1); } } + lua_setuservalue(L, -2); *mp_ptr = mp_initialize(options); xfree(options->command_line); xfree(options->mem_name);

2 1

Fwd: Re: values for `mode` field in tex.nest
by Hans Hagen 29 Oct '18

29 Oct '18

On 10/28/2018 2:53 PM, Marcel Krüger wrote: > Hi, > > according to the documentation, the `mode` field in `tex.nest` is explained as > >> a number representing the main mode at this level: 0 = no >> mode (this happens during \write), 1 = vertical, 127 = hor- >> izontal, 253 = display math, -1 = internal vertical, -127 = >> restricted horizontal, -253 = inline math > > This looks as if these values were constant across engine versions, but the real values > depend on `max_command_cmd` which changes if new commands are added. > Especially the current values are `(-)132` for (restricted) horizontal mode and > `(-)263` for (inline) math. > > So I think the documentation should be either updated to delete these outdated values > and instead include a note that you have to query these values yourself or LuaTeX could > normalize these values before passing them to Lua. > > One way to do this could be i'll give them names (hmode vmode innerhmode innervmode inlinemath displaymath none) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

1 0

ligatures before dashes not working with LuaLaTeX 1.07.0
by Toni Dietze 28 Oct '18

28 Oct '18

Hello, when I switched from version 1.0.4 (TeX Live 2017/Arch Linux) to 1.07.0 (TeX Live 2018/Arch Linux) of LuaLaTeX, I noticed that before dashes ligatures are not typeset anymore. Consider the following example: \documentclass{article} \begin{document} \Huge kickoff meeting kickoff-meeting rofl copter rofl-copter profi font profi-font \end{document} With LuaLaTeX 1.07.0, only in the non-dashed versions the last two letters of the first words are typeset as ligatures. Before a dash, the letters are typeset without a ligature. With pdfLaTeX 3.14159265- 2.6-1.40.19 (TeX Live 2018/Arch Linux), ligatures are also typeset before dashes. Is this behaviour intended? -- Regards Toni Dietze

1 0

values for `mode` field in tex.nest
by Marcel Krüger 28 Oct '18

28 Oct '18

Hi, according to the documentation, the `mode` field in `tex.nest` is explained as > a number representing the main mode at this level: 0 = no > mode (this happens during \write), 1 = vertical, 127 = hor- > izontal, 253 = display math, -1 = internal vertical, -127 = > restricted horizontal, -253 = inline math This looks as if these values were constant across engine versions, but the real values depend on `max_command_cmd` which changes if new commands are added. Especially the current values are `(-)132` for (restricted) horizontal mode and `(-)263` for (inline) math. So I think the documentation should be either updated to delete these outdated values and instead include a note that you have to query these values yourself or LuaTeX could normalize these values before passing them to Lua. One way to do this could be diff --git a/manual/luatex-tex.tex b/manual/luatex-tex.tex index 7edd561e6..f69906d4c 100644 --- a/manual/luatex-tex.tex +++ b/manual/luatex-tex.tex @@ -943,13 +943,13 @@ The known fields are: \DB key \BC type \BC modes \BC explanation \NC \NR \TB \NC \type{mode} \NC number \NC all \NC a number representing the main mode at this level: - \type {0} = no mode (this happens during \prm {write}), - \type {1} = vertical, - \type {127} = horizontal, - \type {253} = display math, - \type {-1} = internal vertical, - \type {-127} = restricted horizontal, - \type {-253} = inline math \NC \NR + \type {0} = no mode (this happens during \prm {write}), + \type {1} = vertical, + \type {2} = horizontal, + \type {3} = display math, + \type {-1} = internal vertical, + \type {-2} = restricted horizontal, + \type {-3} = inline math \NC \NR \NC \type{modeline} \NC number \NC all \NC source input line where this mode was entered in, negative inside the output routine \NC \NR \NC \type{head} \NC node \NC all \NC the head of the current list \NC \NR diff --git a/source/texk/web2c/luatexdir/lua/ltexlib.c b/source/texk/web2c/luatexdir/lua/ltexlib.c index c3e459b22..c71492644 100644 --- a/source/texk/web2c/luatexdir/lua/ltexlib.c +++ b/source/texk/web2c/luatexdir/lua/ltexlib.c @@ -2355,7 +2355,10 @@ static int lua_nest_getfield(lua_State * L) const char *field = lua_tostring(L, -1); r = *rv; if (lua_key_eq(field,mode)) { - lua_pushinteger(L, r->mode_field); + int mode = r->mode_field; + lua_pushinteger(L, + mode == 0 ? 0 : (mode + max_command_cmd * ((mode > 0) * 2 - 1)) / + (max_command_cmd + 1)); } else if (lua_key_eq(field,head)) { lua_nodelib_push_fast(L, r->head_field); } else if (lua_key_eq(field,tail)) { @@ -2396,7 +2399,8 @@ static int lua_nest_setfield(lua_State * L) r = *rv; if (lua_key_eq(field,mode)) { i = lua_tointeger(L, -1); - r->mode_field = i; + r->mode_field = i == 0 ? 0 : + (max_command_cmd + 1) * i + max_command_cmd * ((i < 0) * 2 - 1); } else if (lua_key_eq(field,head)) { n = check_isnode(L, -1); r->head_field = *n; Best regards Marcel Krüger

1 0

Opening files in safer mode
by Marcel Krüger 25 Oct '18

25 Oct '18

Hi, I think there is a bug in `io.open` in --safer mode: \directlua{ io.open(tex.jobname .. '.tex'):close() } \bye If this document is run with `luatex --safer` it leads to `bad argument #1 to 'gsub' (string expected, got nil)`. The problem is a missing `not` in luatex-core.lua, function luatex_io_open_readonly: The default argument 'r' is used if an explicit argument is given and when no "how" argument is given it tries to interpret the non existing argument. This can be fixed by (I omit the changes to `luatex-core.c` in the diff, they are auto-generated anyway): diff --git a/source/texk/web2c/luatexdir/lua/luatex-core.lua b/source/texk/web2c/luatexdir/lua/luatex-core.lua index f2d55fd99..de7f12006 100644 --- a/source/texk/web2c/luatexdir/lua/luatex-core.lua +++ b/source/texk/web2c/luatexdir/lua/luatex-core.lua @@ -54,7 +54,7 @@ local function luatex_io_open(name,how) end local function luatex_io_open_readonly(name,how) - if how then + if not how then how = 'r' else how = gsub(how,'[^rb]','') Best regards, Marcel Krüger

2 2

Re: [Dev-luatex] Bug in \letcharcode
by Marcel Krüger 23 Oct '18

23 Oct '18

---- On Mon, 22 Oct 2018 22:06:56 +0200 luigi scarso <luigi.scarso(a)gmail.com> wrote ---- > > > On Mon, Oct 22, 2018 at 9:58 PM Marcel Krüger <tex(a)2krueger.de> wrote: > > Thank you for the report. Do you have a sample document ? > On my system at least this is triggered by the following LuaLaTeX document: \documentclass{article} \usepackage{unicode-math} \show\document \letcharcode"305=\relax \show\document \begin{document} \end{document} Beside failing during \begin{document}, you can see that between the `\show\document` lines a occurence of `\ifdim \topskip <1sp` changes to `\ifdim ̅<1sp`. I could not reproduce it with ConTeXt/plain yet.

2 1