Re: [Dev-luatex] bug#48064: texlive-* packages fail to build non-deterministically

Em quarta-feira, 30 de junho de 2021, às 08:53:41 -03, luigi scarso escreveu:

On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès wrote:

...

Hi,

Ludovic Courtès skribis:

...

While investigating luatex crashes in the TeX Live 2020 package of GNU Guix¹, we identified the following heap corruption reported by

...

Valgrind (this is on GNU/Linux, with glibc 2.33): This time with debug info for luatex: Thank you for the report, I will check asap.

Thanks! I was able to run Valgrind on LuaTeX 1.13.0, which is the latest one in TeX Live 2021. The invalid reads and writes don’t happen on every run. I had to re-run the command 3 or 4 times until I got the result below (which matches our experience with the build failures in Guix packages) -- Thanks, Thiago $ valgrind --extra-debuginfo-path=/gnu/store/rkhx3pj1qi7fx6pi9p2cg2sb9zn59qmg-profile/lib/debug luatex amsclass.ins ==239904== Memcheck, a memory error detector ==239904== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==239904== Using Valgrind-3.17.0 and LibVEX; rerun with -h for copyright info ==239904== Command: luatex amsclass.ins ==239904== This is LuaTeX, Version 1.13.0 (TeX Live 2021) restricted system commands enabled. ==239904== Invalid write of size 8 ==239904== at 0x4860691: lua_pushlstring (lapi.c:483) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== ==239904== Invalid write of size 4 ==239904== at 0x48606A2: lua_pushlstring (lapi.c:483) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== ==239904== Invalid read of size 16 ==239904== at 0x4861269: lua_rawset (lapi.c:809) ==239904== by 0x56A974: load_hyphenation (texlang.c:307) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== ==239904== Invalid read of size 1 ==239904== at 0x486127D: lua_rawset (lapi.c:811) ==239904== by 0x56A974: load_hyphenation (texlang.c:307) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== ==239904== Invalid write of size 8 ==239904== at 0x485F068: auxgetstr (lapi.c:596) ==239904== by 0x463955: check_texconfig_init (luainit.c:1198) ==239904== by 0x4F0507: main_body (mainbody.c:565) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== ==239904== Invalid write of size 4 ==239904== at 0x485F07A: auxgetstr (lapi.c:596) ==239904== by 0x463955: check_texconfig_init (luainit.c:1198) ==239904== by 0x4F0507: main_body (mainbody.c:565) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== ==239904== Invalid write of size 4 ==239904== at 0x4880608: luaV_finishget (lvm.c:176) ==239904== by 0x485F089: auxgetstr (lapi.c:598) ==239904== by 0x463955: check_texconfig_init (luainit.c:1198) ==239904== by 0x4F0507: main_body (mainbody.c:565) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== ==239904== Invalid read of size 4 ==239904== at 0x485F092: auxgetstr (lapi.c:601) ==239904== by 0x463955: check_texconfig_init (luainit.c:1198) ==239904== by 0x4F0507: main_body (mainbody.c:565) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== ==239904== Invalid read of size 4 ==239904== at 0x485F6D9: lua_type (lapi.c:253) ==239904== by 0x463966: check_texconfig_init (luainit.c:1199) ==239904== by 0x4F0507: main_body (mainbody.c:565) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== (./amsclass.ins (/gnu/store/rkhx3pj1qi7fx6pi9p2cg2sb9zn59qmg-profile/share/texmf-dist/tex/latex/base/docstrip.tex==239904== Conditional jump or move depends on uninitialised value(s) ==239904== at 0x4647BD: tprint (printing.c:484) ==239904== by 0x4E5E6B: write_out (extensions.c:583) ==239904== by 0x4E62EA: wrapup_leader (extensions.c:1324) ==239904== by 0x4E62EA: do_extension (extensions.c:423) ==239904== by 0x4F4860: main_control (maincontrol.c:1030) ==239904== by 0x4F0537: main_body (mainbody.c:577) ==239904== by 0x45118D: main (luatex.c:609) ==239904== ==239904== Conditional jump or move depends on uninitialised value(s) ==239904== at 0x4647BD: tprint (printing.c:484) ==239904== by 0x4E5DED: write_out (extensions.c:585) ==239904== by 0x4E62EA: wrapup_leader (extensions.c:1324) ==239904== by 0x4E62EA: do_extension (extensions.c:423) ==239904== by 0x4F4860: main_control (maincontrol.c:1030) ==239904== by 0x4F0537: main_body (mainbody.c:577) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Utility: `docstrip' v2.6a <2020-07-07> English documentation <2020-07-11> ==239904== Conditional jump or move depends on uninitialised value(s) ==239904== at 0x464523: tprint (printing.c:512) ==239904== by 0x4E5DED: write_out (extensions.c:585) ==239904== by 0x4E62EA: wrapup_leader (extensions.c:1324) ==239904== by 0x4E62EA: do_extension (extensions.c:423) ==239904== by 0x4F4860: main_control (maincontrol.c:1030) ==239904== by 0x4F0537: main_body (mainbody.c:577) ==239904== by 0x45118D: main (luatex.c:609) ==239904== ********************************************************** * This program converts documented macro-files into fast * * loadable files by stripping off (nearly) all comments! * ********************************************************** ******************************************************** * No Configuration file found, using default settings. * ******************************************************** ) Generating file(s) amsthm.sty amsart.cls amsbook.cls amsproc.cls Processing file amsclass.dtx (amsthm) -> amsthm.sty (amsart,classes) -> amsart.cls (amsbook,classes) -> amsbook.cls (amsproc,classes) -> amsproc.cls File amsclass.dtx ended by \endinput. Lines processed: 5197 Comments removed: 2926 Comments passed: 21 Codelines passed: 2062 ) warning (pdf backend): no pages of output. Transcript written on amsclass.log. ==239904== Invalid write of size 8 ==239904== at 0x486C013: GCTM (lgc.c:819) ==239904== by 0x486D779: callallpendingfinalizers (lgc.c:862) ==239904== by 0x486D779: luaC_freeallobjects (lgc.c:971) ==239904== by 0x4877A0B: close_state (lstate.c:245) ==239904== by 0x4E33A5: do_final_end (errors.c:257) ==239904== by 0x45118D: main (luatex.c:609) ==239904== Address 0x894aa40 is 16 bytes after a block of size 1,184 alloc'd ==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==239904== by 0x466BCD: my_luaalloc (luastuff.c:115) ==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86) ==239904== by 0x486A122: luaD_reallocstack (ldo.c:182) ==239904== by 0x486CC17: traversethread (lgc.c:549) ==239904== by 0x486CC17: propagatemark (lgc.c:588) ==239904== by 0x486CFFF: singlestep (lgc.c:1057) ==239904== by 0x486D8BB: luaC_step (lgc.c:1137) ==239904== by 0x48606BB: lua_pushlstring (lapi.c:485) ==239904== by 0x56A963: load_hyphenation (texlang.c:306) ==239904== by 0x56D0CC: undump_one_language (texlang.c:1277) ==239904== by 0x56D0CC: undump_language_data (texlang.c:1290) ==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520) ==239904== by 0x4F03DD: main_body (mainbody.c:540) ==239904== valgrind: m_mallocfree.c:303 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed. valgrind: Heap block lo/hi size mismatch: lo = 1248, hi = 102. This is probably caused by your program erroneously writing past the end of a heap block and corrupting heap metadata. If you fix any invalid writes reported by Memcheck, this assertion failure will probably go away. Please try that before reporting this as a bug. host stacktrace: ==239904== at 0x5803F050: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux) ==239904== by 0x5803F157: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux) ==239904== by 0x5803F2DE: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux) ==239904== by 0x58048742: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux) ==239904== by 0x58037DCB: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux) ==239904== by 0x58036637: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux) ==239904== by 0x5803AAB2: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux) ==239904== by 0x58035988: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux) ==239904== by 0x100417A3ED: ??? ==239904== by 0x1002CB9F2F: ??? ==239904== by 0xBF0E: ??? sched status: running_tid=1 Thread 1: status = VgTs_Runnable (lwpid 239904) ==239904== at 0x486C01B: GCTM (lgc.c:820) ==239904== by 0x486D779: callallpendingfinalizers (lgc.c:862) ==239904== by 0x486D779: luaC_freeallobjects (lgc.c:971) ==239904== by 0x4877A0B: close_state (lstate.c:245) ==239904== by 0x4E33A5: do_final_end (errors.c:257) ==239904== by 0x45118D: main (luatex.c:609) client stack range: [0x1FFEFB0000 0x1FFF000FFF] client SP: 0x1FFF000130 valgrind stack range: [0x1002BBA000 0x1002CB9FFF] top usage: 9624 of 1048576 Note: see also the FAQ in the source distribution. It contains workarounds to several common problems. In particular, if Valgrind aborted or crashed after identifying problems in your program, there's a good chance that fixing those problems will prevent Valgrind aborting or crashing, especially if it happened in m_mallocfree.c. If that doesn't help, please report this bug to: www.valgrind.org In the bug report, send all the above text, the valgrind version, and what OS and version you are using. Thanks. $ echo $? 1