On 5/17/2021 11:43 PM, Thomas A. Schmitz wrote:
Hi all,
I have a rather large (almost 120,000 lines) xml file, processing pulls in another large xml file. The project compiles under mkiv; with lmtx, I get
tex error > tex error on line 8 in file : TeX capacity exceeded, sorry [reference count=32767]
This is not the normal memory consumption that bothers you \enabletrackers[system.usage] will show you more detail. Normally the default context configuration will provide enough. In lmts memory is abit more dynamic so we start otu smaller and then stepwise grow. And one can configure even more (actually i just updated some configuration options for that). Anyway: this more smells like some nesting going wrong and therefore excessive token sharing
<macro> \reallanguagetag
just twenty pages shy of the end of the document.
There is an error log with this content:
return { ["errortype"]=3, ["filename"]="", ["lastcontext"]="\n\n\n<macro> \\reallanguagetag \n \n<macro> \\reallanguagetag #1->\\ifcsname \\??languagelinked #1\n \\endcsname \\lastnamedcs \\else #1\\fi \n<macro> \\setusedlanguage #1->\\edef \\currentusedlanguage {\\reallanguagetag {#1}\n }\\ifempty \\currentusedlanguage \\let \\currentusedlanguage \\currentlanguage \\always\\let \\usedlanguageparameter \\languageparameter \\orelse \\ifx \\currentusedlanguage \\v!global \\let \\currentusedlanguage\n\n<macro> \\typo_delimited_push ...\\currentdelimitedtext {#1}\\edef \\m_delimited_argument {#2}\\ifempty \\m_delimited_argument \\typo_delimited_set_language_nop \n \\else \\typo_delimited_set_language_yes \\fi \\let \\currentparentdelimitedtext \\currentdelimitedtext \\global \\advance \\c_typo...\n\n<macro> \\delimitedtext [#1]#*[#2]->\\dontleavehmode \\typo_delimited_push {#1}{#2}\n \\edef \\p_delimited_method {\\delimitedtextparameter \\c!method }\\ifx \\p_delimited_method \\v!font \\expandafter \\typo_delimited_fontdriven \\else \\expandafter \\typo_delimited_other \\fi \n\n<to be read again> \n {\n...\n\n<initial> \n ", ["lastluaerror"]="", ["lasttexerror"]="TeX capacity exceeded, sorry [reference count=32767]", ["lasttexhelp"]="If you really absolutely need more capacity, you can ask a wizard to enlarge me.", ["linenumber"]=8, ["offset"]=10, ["skiplinenumber"]=8, }
In mkiv, these are the stats at the end of the run:
mwe ...
mkiv lua stats > used config file: selfautoparent:/texmf/web2c/texmfcnf.lua mkiv lua stats > used cache path: /mnt/shared/lmtx/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e
mkiv lua stats > resource resolver: loadtime 0.009 seconds, 1 scans with scantime 0.005 seconds, 0 shared scans, 27 found files, scanned paths: /home/tas/texmf mkiv lua stats > stored bytecode data: 455 modules (0.122 sec), 96 tables (0.007 sec), 551 chunks (0.130 sec) mkiv lua stats > traced context: maxstack: 1422, freed: 2, unreachable: 1420 mkiv lua stats > cleaned up reserved nodes: 63 nodes, 9 lists of 442 mkiv lua stats > node memory usage: 17 attribute, 4 attribute_list, 1 glue, 51 glue_spec, 3 kern, 2 penalty, 2 temp, 2 user_defined mkiv lua stats > node list callback tasks: 8 unique task lists, 9336 instances (re)created, 177413 calls mkiv lua stats > h-node processing time: 4.800 seconds including kernel mkiv lua stats > attribute processing time: 0.617 seconds front- and backend mkiv lua stats > driver time: 0.886 seconds mkiv lua stats > used backend: pdf (backend for directly generating pdf output) mkiv lua stats > jobdata time: 0.161 seconds saving, 0.052 seconds loading mkiv lua stats > callbacks: internal: 261417, file: 319, direct: 3, late: 31831, function 200128, total: 493698 (464 per page) mkiv lua stats > randomizer: resumed with value 0.47143577262518 mkiv lua stats > positions: 9535 collected, 9535 regular, 0 special mkiv lua stats > v-node processing time: 0.401 seconds mkiv lua stats > result saved in file: plutarchusvitae.pdf, compresslevel 3, objectcompresslevel 1 mkiv lua stats > loaded patterns: de::3 en::1 la::65, load time: 0.000 mkiv lua stats > xml load time: 0.648 seconds, 4 files, 4 converted mkiv lua stats > lxml preparation time: 0.626 seconds, 233814 nodes, 578788 lpath calls, 578743 cached calls mkiv lua stats > interactive elements: 0 references, 37 destinations mkiv lua stats > margin data: 10798 entries, 0 pending mkiv lua stats > loaded fonts: 9 files: garamondpremrpro.otf, sabonltstd-bold.otf, sabonltstd-italic.otf, sabonltstd-roman.otf, kadmosu.otf, oldstandard-bold.otf, oldstandard-italic.otf, xitsmath-regular.otf, dejavusansmono.ttf mkiv lua stats > font engine: otf 3.113, afm 1.513, tfm 1.000, 17 instances, 6 shared in backend, 0 common vectors, 6 common hashes, load time 0.146 seconds mkiv lua stats > used platform: linux-64, type: unix, binary subtree: texmf-linux-64 mkiv lua stats > used engine: luatex version: 1.13, functionality level: 7420, banner: this is luatex, version 1.13.0 (tex live 2021) mkiv lua stats > used hash slots: 47131 of 65536 + 250000 mkiv lua stats > lua properties: engine: lua 5.3, used memory: 601 MB, ctx: 579 MB, max: unknown MB, hash chars: min(64,40), symbol mask: utf (τεχ) mkiv lua stats > runtime: 22.310 seconds, 1068 processed pages, 1068 shipped pages, 47.871 pages/second system | total runtime: 213.928 seconds
hardly demanding in terms of memory Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------