- dev-luatex - NTG Mailing Lists

Strings in LuaTeX's pdfe
by Marcel Fabian Krüger 07 Apr '20

07 Apr '20

Hi, I recently tried to do something with the embedded pdfe library and noticed that accessing strings comes with certain problems. PDF strings are always returned in raw form without the surrounding <> or (), so any script using them will need to know if it is a hex string or a "normal" () delimited string in order to treat it correctly. So pdfe.getstring is a bit weird: It gives a Lua string but no indication which type of string is returned. So if pdfe.getstring e.g. returns "425", it can be either correspond to the actual text "425" or it can be the hexadecimal encoding of "BP". Given that PDF allows beoth upper and lowercase letters and even an odd number of digits in a hexadecimal string, even guessing the right format is hard and error-prone, making pdf.getstring not particularly useful. The same issue appears with the `__index` metafunctions of dictionaries and arrays. This is especially weird because it's inconsistent with PDF names which always get decoded before they are passed to the user. Also even after the Lua script figures out if it is a hex string or a literal string, it has to decode it. (Of course this part only applies if the actual value is needed and not if it only should be passed into another PDF string) That's not complicated, but it feels weird: After all, the underlying pplib already decoded the string, so it seems like it would be easier to make this decoded version accessible to the user. So would it be possible to maybe either change the existing functions or add new ones to 1. return the already decoded value and/or 2. give an indication if a literal or a hex string is returned? Best regards, Marcel

4 7

node.get_properties_table is broken
by Marcel Fabian Krüger 31 Mar '20

31 Mar '20

Hi, the function node.get_properties_table returns a tabe which does not contain the actual node properties and also doesn't allow setting the node properties. See for example \directlua{ node.set_properties_mode(true, true) } \directlua{ local indirectprops = node.get_properties_table() local directprops = node.direct.get_properties_table() local n = node.new'glue' % Type doesn't really matter local p = {} node.setproperty(n, p) assert(node.getproperty(n) == p, "Assert failed: getproperty after setproperty") % works fine local directn = node.direct.todirect(n) assert(node.direct.getproperty(directn) == p, "Assert failed: direct.getproperty after setproperty") % works fine assert(directprops[directn] == p, "Assert failed: direct.get_properties_table after setproperty") % works fine assert(indirectprops[n] == p, "Assert failed: get_properties_table after setproperty") % This breaks node.free(n) } \bye Here a property is set using node.setproperty and then we try to access it through different methods. All of them work, except for node.get_properties_table because this uses a different table which is unrelated to the actual property table returned by e.g. node.direct.get_properties_table. This issue appear because the metatable __index and __newindex functions for node.get_properties_table set the property in the table passed as first parameter, which is the (non-direct) node.get_properties_table table instead of node.direct.get_properties_table(). Here is a possible fix which passes the actual property table as upvalue to the metafunctions: --- source/texk/web2c/luatexdir/lua/lnodelib.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/source/texk/web2c/luatexdir/lua/lnodelib.c b/source/texk/web2c/luatexdir/lua/lnodelib.c index d5ab00a55..4e713aaa2 100644 --- a/source/texk/web2c/luatexdir/lua/lnodelib.c +++ b/source/texk/web2c/luatexdir/lua/lnodelib.c @@ -8406,7 +8406,7 @@ static int lua_nodelib_get_property_t(lua_State * L) if (n == null) { lua_pushnil(L); } else { - lua_rawgeti(L,1,n); + lua_rawgeti(L,lua_upvalueindex(1),n); } return 1; } @@ -8417,7 +8417,7 @@ static int lua_nodelib_set_property_t(lua_State * L) halfword n = *((halfword *) lua_touserdata(L, 2)); if (n != null) { lua_settop(L,3); - lua_rawseti(L,1,n); + lua_rawseti(L,lua_upvalueindex(1),n); } return 0; } @@ -8831,16 +8831,19 @@ static const struct luaL_Reg nodelib_p[] = { static void lua_new_properties_table(lua_State * L) { - lua_pushstring(L,"node.properties"); lua_newtable(L); + lua_pushstring(L,"node.properties"); + lua_pushvalue(L,-2); lua_settable(L,LUA_REGISTRYINDEX); lua_pushstring(L,"node.properties.indirect"); lua_newtable(L); luaL_newmetatable(L,"node.properties.indirect.meta"); - luaL_openlib(L, NULL, nodelib_p, 0); + lua_pushvalue(L,-4); + luaL_openlib(L, NULL, nodelib_p, 1); lua_setmetatable(L,-2); lua_settable(L,LUA_REGISTRYINDEX); + lua_pop(L,1); } /* node.direct.* */ -- 2.26.0 Best regards, Marcel

3 4

Misleading documentation for node.setglue
by Marcel Fabian Krüger 27 Mar '20

27 Mar '20

Hi, according to the LuaTeX manual for node.setglue: > When you pass values, only arguments that are numbers are assigned so > node.setglue(n,655360,false,65536) > will only adapt the width and shrink. This does not correspond to the actual behaviour. In practise, arguments which are not numbers are treated as zero: \directlua{ local n = node.new'glue' node.setglue(n, 100000, 100000, 100000, 2, 2) print('BEFORE', node.getglue(n)) node.setglue(n, 655360, false, 655360) print('AFTER', node.getglue(n)) node.free(n) } \bye prints BEFORE 100000 100000 100000 2 2 AFTER 655360 0 655360 0 0 The documentation should probably be adjusted to reflect the actual behaviour. - Marcel

1 0

depth of glyph nodes with y_displacement<0
by Marcel Fabian Krüger 13 Mar '20

13 Mar '20

Hi, I found a curiosity in LuaTeX's treatment of `y_displacement` in glyph nodes: Take the following ConTeXt example: \usetypescript[pagella] \setupbodyfont[pagella,12pt] \starttext f\char776\char776\char776\char807\char807\char807\vrule \stoptext (See https://github.com/latex3/luaotfload/issues/134 for a more practical case where this leads to issues) Here the \vrule shows that the marks attached at the top get their y_displacement added to their height, while for the marks added at the bottom the additional depth is ignored. (The first \char807 is still covered by the rule because that is the natural height of this glyph) This is explicitly done in texnodes.c: For glyph_height, the displacement is always added. For glyph_depth, the displacement is only added if it is positive: scaled glyph_height(halfword p) { scaled w = char_height(font(p), character(p)) + y_displace(p); if (w < 0) w = 0; return w; } scaled glyph_depth(halfword p) { scaled w = char_depth(font(p), character(p)); if (y_displace(p) > 0) w = w - y_displace(p); if (w < 0) w = 0; return w; } This seems inconsistant. Is there a reason why this would be required or could it be changed to handle height and depth consistently? Best regards, Marcel

2 5

append_to_vlist with single return value broken
by Marcel Fabian Krüger 07 Mar '20

07 Mar '20

Hi, according to the documentation for `append_to_vlist_filter`: > The prevdepth is also optional so it should be allowed to only return a box in which case the prevdepth value shouldn't change. But if we actually try this: \directlua{ callback.register("append_to_vlist_filter", function(b) return b end) } abc \bye LuaTeX answers with warning (pdf backend): no pages of output. because it discards all boxes. This is caused by a bug in luanode.c, lua_appendtovlist_callback: The callback is called with lua_pcall(Luas, 4, 2, 0). Then LuaTeX tries to determine if one or two values have been returned by looking at the type of slot -1: If this is a number, the second return value have been a number and it is evaluated as new prevdepth. Otherwise else if (lua_type(Luas, -1) != LUA_TNIL) { p = check_isnode(Luas, -1); *result = *p; } is evaluated to scan for a single return value which is a node. But because lua_pcall(Luas, 4, 2, 0) explicitly requested two return values, a callback with one return values would result in a stack where the one return value is followed by `nil`. Therefore this `if` block is not entered. The problem can be fixed by always looking at stack slot -2 for the returned node: source/texk/web2c/luatexdir/lua/luanode.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/source/texk/web2c/luatexdir/lua/luanode.c b/source/texk/web2c/luatexdir/lua/luanode.c index a0c97e37d..d4baab96d 100644 --- a/source/texk/web2c/luatexdir/lua/luanode.c +++ b/source/texk/web2c/luatexdir/lua/luanode.c @@ -183,12 +183,9 @@ int lua_appendtovlist_callback(halfword box, int location, halfword prev_depth, if (lua_type(Luas,-1) == LUA_TNUMBER) { *next_depth = lua_roundnumber(Luas,-1); *prev_set = true; - if (lua_type(Luas, -2) != LUA_TNIL) { - p = check_isnode(Luas, -2); - *result = *p; - } - } else if (lua_type(Luas, -1) != LUA_TNIL) { - p = check_isnode(Luas, -1); + } + if (lua_type(Luas, -2) != LUA_TNIL) { + p = check_isnode(Luas, -2); *result = *p; } return 1; -- Best regards, Marcel

3 3

Potential bug with kerning in math mode
by Matthias Nagel 07 Mar '20

07 Mar '20

Dear all, as I am unsure what is the officially correct way to submit a potential bug, I post it here and would like to draw your attention to this post, because I do not now of the tracker is still in use. http://tracker.luatex.org/view.php?id=1019 My apologies, if this is a double posting. Also, I already have posted this problem to the Libertinus project as I initially thought this would be a bug in the font, but the maintainer of Libertinus said it is an issue with LuaTex. Bests, M.N.

3 5

os.setenv()
by Reinhard Kotucha 22 Feb '20

22 Feb '20

Hi, I just tried this with texlua 1.10.0: -------------------------------------------- local function time_of_day (TZ) os.setenv('TZ', TZ) local time = os.gettimeofday() print(os.date('%c', math.floor(time))) end time_of_day ('America/New_York') time_of_day ('Europe/Paris') time_of_day ('Asia/Tokyo') -------------------------------------------- It seems that os.setenv() is only executed once and all results are in Eastern Standard Time: Fri Feb 21 16:37:13 2020 Fri Feb 21 16:37:13 2020 Fri Feb 21 16:37:13 2020 When I call "time_of_day ('Asia/Tokyo')" first I get Sat Feb 22 06:39:46 2020 Sat Feb 22 06:39:46 2020 Sat Feb 22 06:39:46 2020 Is it possible to fix this behavior? Regards, Reinhard -- ------------------------------------------------------------------ Reinhard Kotucha Phone: +49-511-3373112 Marschnerstr. 25 D-30167 Hannover mailto:reinhard.kotucha@web.de ------------------------------------------------------------------

2 3

Some legacy fonts cause "invalid list tail, probably missing glue"
by Daniel Benjamin Miller 20 Feb '20

20 Feb '20

There is an issue with lualatex which comes about when using a document such as this (based on a discussion here https://tex.stackexchange.com/questions/525898/combining-french-and-greek-b…) Try running lualatex on the following document, and an error will be returned. This seems to apply for all fonts whose packages were generated by autoinst (step is just an example). \documentclass[12pt]{article} \usepackage[utf8]{luainputenc}\usepackage[LGR,T1]{fontenc}\input{lgrenc.dfu} \usepackage{step} \usepackage[greek.ancient,main=french]{babel} \title{Ça marche?} %\date{} \author{me} \begin{document} \maketitle un, deux, trois \foreignlanguage{greek}{ἀγαγεῖν} \end{document} According to Marcel Krüger, this is related to an issue which was reported over a year ago: https://mailman.ntg.nl/pipermail/dev-luatex/2018-November/006138.html.

3 7

Disappearing quoteleft in luatex
by Norbert Preining 07 Feb '20

07 Feb '20

Dear all, I see a strange thing happening with the following document all with uptodate TeX Live: \documentclass{article} \usepackage{fontspec} \setmainfont{Gaultier} \begin{document} Hello `World' \end{document} xelatex: ok lualatex: quoteleft character is missing lualatex-dev with [RawFeature={mode=harf;}]: ok So it seems that the lualatex font loader seems to be broken here. The character is present in the font 325 (0x145) U+2018 "quoteleft" LEFT SINGLE QUOTATION MARK and separate from the quoteright. Any suggestions how to fix these kind of errors? Best Norbert -- PREINING Norbert http://www.preining.info Accelia Inc. + IFMGA ProGuide + TU Wien + JAIST + TeX Live + Debian Dev GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

2 2

Updates for (lua)HarfBuzz
by Marcel Fabian Krüger 25 Jan '20

25 Jan '20

Hello, would it be possible to pull a new version of luaharfbuzz in before TeXLive 2020 is frozen? There hasn't been an official release, but there are some new interfaces which would be very useful for luaotfload. There are also at least two bugs in HarfBuzz affecting luaotfload for which the fixes are in the repo but not yet released, but I guess the HarfBuzz version is controlled by the TeXLive team? Best regards, Marcel

3 4