- dev-luatex - NTG Mailing Lists

Dump sharing - redundant copying/allocations
by Michal Vlasák 21 Jul '21

21 Jul '21

Hello, when NO_DUMP_SHARE is not defined and the system is Little endian, then for the possibility of sharing format files between architectures all dumped multi-byte valules are byte swapped by the function "swap_items" in "tex/texfileio.c". The origin of the function is in Web2C's "texmfmp.c" and it seems that it was added to LuaTeX in the version 1.08 when the source was converted from CWeb (or am I reading the history wrong?). However, there are two differences between Web2C's and LuaTeX's "swap_items": 1) LuaTeX also supports 12 byte values. and more importantly: 2) LuaTeX essentially does this: - allocate temporary array - copy input to temporary array - [common code with web2c] - copy temporary array back to input to serve as output - free temporary array This all seems redundant and causes many small allocations (~350K allocations for a ~800K format file), because most allocations are of only 4 bytes. Curiously the gcc -O2 optimizer doesn't catch this even though it is a static function (and changing xmalloc/xfree to the "intrinsic" malloc/free doesn't help it). Maybe the possible unsigned int overflow prevents the optimization? Or am I missing some side effect/purpose of the copying/allocating? See my proposal below. (Note that in the LuaTeX repository --disable-dump-share is the default, while it isn't in TeX Live, I think.) Michal Vlasák --- a/tex/texfileio.c +++ b/tex/texfileio.c @@ -1125,13 +1125,9 @@ static gzFile gz_fmtfile = NULL; */ -static void swap_items(char *pp, int nitems, int size) +static void swap_items(char *p, int nitems, int size) { char temp; - unsigned total = (unsigned) (nitems * size); - char *q = xmalloc(total); - char *p = q; - memcpy(p,pp,total); /*tex Since `size' does not change, we can write a while loop for each case, @@ -1201,8 +1197,6 @@ static void swap_items(char *pp, int nitems, int size) default: FATAL1("Can't swap a %d-byte item for (un)dumping", size); } - memcpy(pp,q,total); - xfree(q); } #endif

5 12

\tracingtokens=1, ( ) are logged curiously
by Petr Olsak 21 May '21

21 May '21

Suppose: ----------------------------------------- \tracingscantokens=1 \scantextokens{first} \scantextokens{second} \bye ----------------------------------------- and run luatex test. Then the ( and ) are logged curiously: ( ( ... )) Why all closing )) are logged after end command? If we change \scantextokens to \scantokens, then ( and ) behaves more logical in log file and in the terminal: ( ) ( ). Petr Olsak

3 3

BUG: Luatex gets high unicode codepoints with an offset
by Sergio Callegari 12 May '21

12 May '21

Hi, please consider this sample code|:| ||||\documentclass{article} \usepackage{fontspec} \newfontface\MDI{Material Design Icons}[] \begin{document} Look at this character {\MDI \char"F1372}! \end{document} It is meant to work with the materialdesignicons-webfont.ttf font downloadable from https://materialdesignicons.com/ <https://materialdesignicons.com/> . The code works just fine with xelatex, but with lualatex it picks a wrong char (there seems to be an offset by 14). See also https://tex.stackexchange.com/questions/596610/how-to-use-luatex-with-large… . Seems a bug to me, certainly an inconsistency with xelatex. Any idea? Best, Sergio ||

2 1

Potential race condition in makefiles
by Dmitry Kalinkin 08 May '21

08 May '21

Hi, We are observing a following build failure: ../../../texk/web2c/luatexdir/luamd5/md5lib.c:197:10: fatal error: 'utilsha.h' file not found #include "utilsha.h" ^~~~~~~~~~~ 1 error generated. For the full log, see https://hydra.nixos.org/build/142527628/nixlog/2 Best regards, Dmitry

2 3

\protrusionboundary
by Robert 06 May '21

06 May '21

Hi, the command \protrusionboundary could potentially be very useful, but I'm not sure I understand how it's supposed to work. The manual gives the example (p.43): | \protrusionboundary1\llap{!\quad}«Who needs protrusion?» However, with the following: \input luaotfload.sty \parindent=0pt \protrudechars=2 \directlua{ fonts.protrusions.setups['test'] = { [0x00AB] = { 1, 0 } % left guillemet } } \font\x=[lmroman10-regular.otf]:protrusion=test; \x Who\par «Who\par \protrusionboundary1\llap{!\quad}«Who \bye the guillemet on the last line is not protruded as I would have expected. Or am I missing something? Secondly, the documentation says that \protrusionboundary will ignore the next/previous node (singular). What would be much more practical in real life, I think, is to have a command that would simply say, "consider the current point as left/right/both margin(s)", because more often than not, you'd find a succession of hboxes, penalties etc. (plural) that you'd want to skip. Would that be feasible? Best, -- Robert

1 0

i686-mingw: texlua.exe FFI module not working
by Naveen M K 28 Apr '21

28 Apr '21

Hi, I had been trying to compile TexLive on MINGW compilers on Windows provided by MSYS2[0] project. See this PR[1]. It was only for i686-mingw. x86-64-mingw worked without any problems. I found that there was Segmentation Fault(or crash) when running the `runscript.exe` the launchers which TexLive provides on Windows and I could see it uses LuaTeX. You can find the source here[2]. What is essentially does is calls the `texlua.exe` with `runscript.tlu` passed it in. I initially thought it was a problem with compiled version of the launcher itself, but it wasn't it, it was a problem with `texlua.exe`. `runscript.exe` just returns the code returned from `texlua.exe` and the return code was a crash and there is a popup which say the program isn't working correctly and crashed at `ntdll.dll`. While just running `texlua.exe` on a normal hello world lua script[3] works without any crash or error. So, I thought something from the code in `runscript.tlu`[4] should be causing this issue. I don't know how to debug a lua script, so my plan was to randomly place print statements (you know that is the easiest and best way to debug something), and whether it printed or not. I first looked over the `runscript.tlu` to see if there are parts of code which could crash the interpreter and I found this block of code[5] which uses ffi module. Now I placed print statements before and after local runscripthandle = ffi.C.GetModuleHandleA("runscript.dll") and found that the one before was printed and `texlua.exe` crashed. So it is something related to ffi module. I tried the same with `texluajit.exe` and it seemed to work. What I did to make the launchers work was to patch this [6] and it worked without any problem. But that isn't the actual fix the Crash in `texlua.exe` on using ffi module in `i686-mingw`. I hope anybody will help me fix this. For the version, I am building TexLive with this `ccb6338507149edf2772bfccb9933b72761f4cc4` commit on GitHub or `svn://tug.org/texlive/trunk/Build/source@58686`. Best, Naveen [0]: https://www.msys2.org [1]: https://github.com/msys2/MINGW-packages/pull/8271 [2]: https://github.com/TeX-Live/texlive-source/blob/9bad8358d2f98a8bed86ecfa238… [3]: https://www.lua.org/pil/1.html [4]: https://github.com/TeX-Live/texlive-source/blob/9bad8358d2f98a8bed86ecfa238… [5]: https://github.com/TeX-Live/texlive-source/blob/9bad8358d2f98a8bed86ecfa238… [6]: https://github.com/msys2/MINGW-packages/blob/3e2204b3fa18cd967b7aa616dfef9e…

2 1

Uunderdelimiter & friends and mathstyles
by Marcel Fabian Krüger 20 Apr '21

20 Apr '21

Hi, I noticed some odd behavior about \U{under,over}delimiter/\Udelimiter{under,over} related to the applied mathstyles for the nucleus: Take the following example \input luaotfload.sty \font\lmmath"Latin Modern Math:script=math,mode=base" \font\scriptlmmath"Latin Modern Math:script=math,mode=base" at 7pt \font\scriptscriptlmmath"Latin Modern Math:script=math,mode=base" at 5pt \textfont42=\lmmath \scriptfont42=\scriptlmmath \scriptscriptfont42=\scriptscriptlmmath $$ \Udelimiterunder42 "2192{\mathstyle x^L} \Udelimiterover42 "2192{\mathstyle x^L} \Uunderdelimiter42 "2192{\mathstyle x^L} \Uoverdelimiter42 "2192{\mathstyle x^L} $$ \bye In all four situations \mathstyle shows 1, but neither of the four x^L are actually set in \mathstyle 1 (crampeddisplay): \Udelimiterunder sets it's argument in the outer mathstyle. IMO this is the expected behavior but should be reflected in \mathstyle. \Udelimiterover sets it's argument in the outer mathstyle. IMO it should be cramped instead. \Uoverdelimiter sets it's argument in subscript style (cramped and one size smaller) Since there's nothing printed on top of it' argument, I think that superscript style would be better. (aka not cramped) In any case it should be reflected in \mathstyle. \Uunderdelimiter sets it's argument in superscript style (one size smaller) Since the delimiter is printed on top of it I think that subscript style would be better (aka cramped). In any case it should be reflected in \mathstyle. The patch at the end adapts both \mathstyle and changes the actually used styles. Best regards, Marcel diff --git a/source/texk/web2c/luatexdir/tex/mlist.c b/source/texk/web2c/luatexdir/tex/mlist.c index 2ac631ea3..71e7c88ea 100644 --- a/source/texk/web2c/luatexdir/tex/mlist.c +++ b/source/texk/web2c/luatexdir/tex/mlist.c @@ -2385,7 +2385,7 @@ static void make_over_delimiter(pointer q, int cur_style) pointer x, y, v; scaled shift_up, shift_down, clr, delta, wd; boolean stack; - x = clean_box(nucleus(q), sub_style(cur_style), cur_style, math_nucleus_list); + x = clean_box(nucleus(q), sup_style(cur_style), cur_style, math_nucleus_list); check_widths(q,x); y = do_delimiter(q, left_delimiter(q), cur_size, wd, true, cur_style, true, &stack, NULL, NULL); left_delimiter(q) = null; @@ -2416,7 +2416,7 @@ static void make_under_delimiter(pointer q, int cur_style) pointer x, y, v; scaled shift_up, shift_down, clr, delta, wd; boolean stack; - y = clean_box(nucleus(q), sup_style(cur_style), cur_style, math_nucleus_list); + y = clean_box(nucleus(q), sub_style(cur_style), cur_style, math_nucleus_list); check_widths(q,y); x = do_delimiter(q, left_delimiter(q), cur_size, wd, true, cur_style, true, &stack, NULL, NULL); left_delimiter(q) = null; @@ -2447,7 +2447,7 @@ static void make_delimiter_over(pointer q, int cur_style) pointer x, y, v; scaled shift_up, shift_down, clr, actual, wd; boolean stack; - y = clean_box(nucleus(q), cur_style, cur_style, math_nucleus_list); + y = clean_box(nucleus(q), cramped_style(cur_style), cur_style, math_nucleus_list); check_widths(q,y); x = do_delimiter(q, left_delimiter(q), cur_size + (cur_size == script_script_size ? 0 : 1), wd, true, cur_style, true, &stack, NULL, NULL); left_delimiter(q) = null; diff --git a/source/texk/web2c/luatexdir/tex/texmath.c b/source/texk/web2c/luatexdir/tex/texmath.c index 461a51578..0c0e8a9cc 100644 --- a/source/texk/web2c/luatexdir/tex/texmath.c +++ b/source/texk/web2c/luatexdir/tex/texmath.c @@ -1580,6 +1580,7 @@ static void scan_delimiter(pointer p, int r) void math_radical(void) { halfword q; + int new_mstyle; int chr_code = cur_chr; halfword options = 0; tail_append(new_node(radical_noad, chr_code)); @@ -1600,28 +1601,35 @@ void math_radical(void) } } radicaloptions(tail) = options; - if (chr_code == 0) + if (chr_code == 0) { /*tex \.{\\radical} */ scan_delimiter(left_delimiter(tail), tex_mathcode); - else if (chr_code == 1) + new_mstyle = cramped_style(m_style); + } else if (chr_code == 1) { /*tex \.{\\Uradical} */ scan_delimiter(left_delimiter(tail), umath_mathcode); - else if (chr_code == 2) + new_mstyle = cramped_style(m_style); + } else if (chr_code == 2) { /*tex \.{\\Uroot} */ scan_delimiter(left_delimiter(tail), umath_mathcode); - else if (chr_code == 3) + new_mstyle = cramped_style(m_style); + } else if (chr_code == 3) { /*tex \.{\\Uunderdelimiter} */ scan_delimiter(left_delimiter(tail), umath_mathcode); - else if (chr_code == 4) + new_mstyle = sub_style(m_style); + } else if (chr_code == 4) { /*tex \.{\\Uoverdelimiter} */ scan_delimiter(left_delimiter(tail), umath_mathcode); - else if (chr_code == 5) + new_mstyle = sup_style(m_style); + } else if (chr_code == 5) { /*tex \.{\\Udelimiterunder} */ scan_delimiter(left_delimiter(tail), umath_mathcode); - else if (chr_code == 6) + new_mstyle = m_style; + } else if (chr_code == 6) { /*tex \.{\\Udelimiterover} */ scan_delimiter(left_delimiter(tail), umath_mathcode); - else if (chr_code == 7) + new_mstyle = cramped_style(m_style); + } else if (chr_code == 7) /*tex \.{\\Uhextensible} */ scan_delimiter(left_delimiter(tail), umath_mathcode); else @@ -1645,12 +1653,12 @@ void math_radical(void) vlink(degree(tail)) = null; q = new_node(math_char_node, 0); nucleus(tail) = q; - (void) scan_math(nucleus(tail), cramped_style(m_style)); + (void) scan_math(nucleus(tail), new_mstyle); } } else { q = new_node(math_char_node, 0); nucleus(tail) = q; - (void) scan_math(nucleus(tail), cramped_style(m_style)); + (void) scan_math(nucleus(tail), new_mstyle); } }

3 4

linkarea in rtl context
by Ulrike Fischer 19 Apr '21

19 Apr '21

In luatex 1.13.0 (texlive 2021) this here gives the wrong linkarea: it stretches to the left from the end of the link instead to the right. In texlive 2020 the behaviour is correct: \pardir TRT \textdir TRT \linedir TRT X \pdfextension startlink attr {/Border[0 0 1]} user{ /Subtype/Link% /A<<% /Type/Action% /S/URI% /URI(blub)% >>% } Linktext \pdfextension endlink y \bye -- Ulrike Fischer https://www.troubleshooting-tex.de/

2 4

What exactly means \suppressmathparerror=1 ?
by Petr Olsak 28 Mar '21

28 Mar '21

The \suppressmathparerror=1 should suppress \par errors in math mode. OK, this works: ----------------- \suppressmathparerror=1 $$ a b $$ \bye ------------------ but the following example gives error: Display math should end with \Ustopdisplaymath ------------------- \suppressmathparerror=1 $$ \halign{#\cr a\cr} $$ \bye ------------------- The LuaTeX manual does not exactly describes the behavior of this case. So, I don't know if this is feature or bug. Note that the second example is primitive meaning of something like this: -------------------- $$ \eqalignno{...} $$ -------------------- If a user know that "empty lines are allowed in math mode" then he/she could leave empty line after \eqalignno too. Best regards Petr Olsak

3 2

segfault in luatex, assert in pdftex
by Martin Budaj 16 Mar '21

16 Mar '21

Hi, there is a segfault produced by LuaTeX, Version 1.12.0 (TeX Live 2020/Debian) when processing the following: \ifdefined\directlua \protected\def\pdfmapfile {\pdfextension mapfile } \protected\def\pdfmapline {\pdfextension mapline } \edef\pdfgentounicode {\pdfvariable gentounicode} \protected\def\pdfglyphtounicode {\pdfextension glyphtounicode } \fi \pdfglyphtounicode{A}{0041} \pdfgentounicode=1 \pdfmapfile{} \pdfmapline{cmr10 CMR10 34} % no font embedding anything \end PdfTex, Version 3.14159265-2.6-1.40.21 (TeX Live 2020/Debian) gives the following assertion: pdftex: ../../../texk/web2c/pdftexdir/writefont.c:555: write_fontdictionary: Assertion `fo->fd->builtin_glyph_names != NULL' failed. Aborted If the combination of using \pdfgentounicode and not embedding any font is invalid, some reasonable error message should be produced. Best regards Martin

2 3