[NTG-context] bottlenecks

[Hans Hagen]

On 12/18/2022 4:21 PM, mf via ntg-context wrote:
> Il 18/12/22 14:49, Hans Hagen via ntg-context ha scritto:
>> On 12/18/2022 2:14 PM, mf via ntg-context wrote:
>>
>> > ...
>>
>> So we gained a lot in the meantime.
>>
>>> mkiv lua stats  > loaded fonts: 27 files: lmmono10-regular.otf, 
>>> texgyrepagella-math.otf, dejavusansmono.ttf, gfsneohellenic.otf, 
>>> firasans-bolditalic.otf, firasans-regular.otf, 
>>> firasans-regularitalic.otf, firasanscondensed-bold.otf, 
>>> firasanscondensed-regular.otf, garamondpremrpro-bdit.otf, 
>>> garamondpremrpro-med.otf, garamondpremrpro-medcapt.otf, 
>>> garamondpremrpro-meddisp.otf, garamondpremrpro-medit.otf, 
>>> garamondpremrpro-meditcapt.otf, garamondpremrpro-medsubh.otf, 
>>> garamondpremrpro-smbdit.otf, myriadpro-bold.otf, 
>>> myriadpro-boldcond.otf, myriadpro-boldcondit.otf, myriadpro-cond.otf, 
>>> myriadpro-condit.otf, myriadpro-it.otf, myriadpro-regular.otf, 
>>> myriadpro-semiboldit.otf, quivira.otf, sileot.ttf
>>> mkiv lua stats  > font engine: otf 3.131, afm 1.513, tfm 1.000, 533 
>>> instances, 501 shared in backend, 6 common vectors, 495 common 
>>> hashes, load time 0.196 seconds
>> How does that report with compact font mode?
>>
>>
> mkiv lua stats  > resource resolver: loadtime 0.005 seconds, 0 scans 
> with scantime 0.000 seconds, 0 shared scans, 107 found files, scanned 
> paths: <none>
> mkiv lua stats  > stored bytecode data: 507 modules (0.285 sec), 106 
> tables (0.017 sec), 613 chunks (0.302 sec)
> mkiv lua stats  > traced context: maxstack: 1923, freed: 246, 
> unreachable: 1677
> mkiv lua stats  > cleaned up reserved nodes: 76 nodes, 9 lists of 504
> mkiv lua stats  > node memory usage: 1684 attribute, 110 fontspec, 419 
> glue, 87 gluespec, 3 kern, 647 mathspec, 4 penalty, 2 temp, 591 whatsit
> mkiv lua stats  > node list callback tasks: 13 unique task lists, 17 
> instances (re)created, 325895 calls
> mkiv lua stats  > h-node processing time: 10.654 seconds including kernel
> mkiv lua stats  > attribute processing time: 2.810 seconds front- and 
> backend
> mkiv lua stats  > driver time: 11.714 seconds
> mkiv lua stats  > used backend: pdf
> mkiv lua stats  > jobdata time: 0.810 seconds saving, 0.194 seconds loading
> mkiv lua stats  > callbacks: file: 5439, saved: 644924, direct: 1386, 
> function: 534140, value: 1033, message: 0, bytecode: 613, late 0, total: 
> 1187535 (1151 per page)
> mkiv lua stats  > randomizer: resumed with value 0.58701179146283
> mkiv lua stats  > v-node processing time: 1.480 seconds
> mkiv lua stats  > loaded patterns: en::1 it::66, load time: 0.000
> mkiv lua stats  > xml load time: 0.444 seconds, 1 files, 1 converted
> mkiv lua stats  > lxml preparation time: 26.359 seconds, 47936 nodes, 
> 24702 lpath calls, 24331 cached calls
> mkiv lua stats  > interactive elements: 9327 references, 12951 destinations
> mkiv lua stats  > margin data: 6 entries, 0 pending
> mkiv lua stats  > loaded fonts: 27 files: lmmono10-regular.otf, 
> texgyrepagella-math.otf, dejavusansmono.ttf, gfsneohellenic.otf, 
> firasans-bolditalic.otf, firasans-regular.otf, 
> firasans-regularitalic.otf, firasanscondensed-bold.otf, 
> firasanscondensed-regular.otf, garamondpremrpro-bdit.otf, 
> garamondpremrpro-med.otf, garamondpremrpro-medcapt.otf, 
> garamondpremrpro-meddisp.otf, garamondpremrpro-medit.otf, 
> garamondpremrpro-meditcapt.otf, garamondpremrpro-medsubh.otf, 
> garamondpremrpro-smbdit.otf, myriadpro-bold.otf, myriadpro-boldcond.otf, 
> myriadpro-boldcondit.otf, myriadpro-cond.otf, myriadpro-condit.otf, 
> myriadpro-it.otf, myriadpro-regular.otf, myriadpro-semiboldit.otf, 
> quivira.otf, sileot.ttf
> mkiv lua stats  > font engine: otf 3.131, afm 1.513, tfm 1.000, 46 
> instances, 16 shared in backend, 4 common vectors, 12 common hashes, 
> load time 0.148 seconds
> mkiv lua stats  > text directions: 5.911 seconds
> mkiv lua stats  > metapost: 1.180 seconds, loading: 0.083, execution: 
> 1.109, n: 1039, average: 0.002, instances: 2, luacalls: 46885 (file: 
> 34713, text: 0, script: 5932, log: 6240), memory: 8.995 M
> mkiv lua stats  > math tweaking time: 0.462 seconds, 8 math goodie tables
> mkiv lua stats  > math processing time: 0.211 seconds
> mkiv lua stats  > pdf annotations: 9327 links (9316 unique), 0 special
> mkiv lua stats  > font embedding time: 0.104 seconds, 23 fonts
> mkiv lua stats  > result saved in file: v32.pdf, compresslevel 0, 
> objectcompresslevel 0
> mkiv lua stats  > positions: 8572 collected, 4774 deltas, 8565 shared 
> partials, 305 partial entries
> mkiv lua stats  > used platform: linux-64, type: unix, binary subtree: 
> texmf-linux-64
> mkiv lua stats  > used engine: luametatex version: 2.1004, functionality 
> level: 20221214, format id: 679, compiler: gcc
> mkiv lua stats  > tex properties: 740852 hash slots used of 2097152, 
> 93821 control sequences, approximate memory usage: 65 MB
> mkiv lua stats  > lua properties: engine: lua 5.4, used memory: 338 MB, 
> ctx: 266 MB, max: 415 MB, symbol mask: utf (τεχ)
> mkiv lua stats  > runtime: 114.917 seconds, 1032 processed pages, 1032 
> shipped pages, 8.980 pages/second
> 
> It's better, but the pages are 1032 against 1026.
> 
> It means that some lines at the end of a page slip in the following 
> pages. Since there are forced page breaks and chapters starting at odd 
> pages, they are minimal differences that result in pages with only 1 or 
> 2 lines of text, or white even pages.
> 
> I don't know whether this is expected as the effect of 
> \enableexperiments[fonts.compact] only.

i has to do with sightly different accuracy as we scale on the fly ... 
(we are somewhat stuck to this tex scaling model where 1000 = 1.0 but 
maybe it's time to revision that

> I expected a difference between version 2020-04-26 and the latest one, 
> because it always happened when I tested the newest versions, though 
> there has been a convergence of the resulting pdf and the differences 
> are minimal now (between version 2020-04-26 and 2022-12-15). But it did 
> not happen, the resulting pages are still 1026.

most of this has to do with fonts .. we bumped the accuracy a bit 
recently (to cope better with radical line alignment in some radicals)

> Here's the results in short:
> 
> ConTeXt 2020-04-26:
> 
> runtime: 172.721 seconds, 1026 processed pages, 1026 shipped pages, 
> 5.940 pages/second
> 
> ConTeXt 2022-12-15:
> 
> runtime: 130.080 seconds, 1026 processed pages, 1026 shipped pages, 
> 7.887 pages/second
> 
> ConTeXt 2022-12-15 with \enableexperiments[fonts.compact]:
> 
> runtime: 114.917 seconds, 1032 processed pages, 1032 shipped pages, 
> 8.980 pages/second
ok, so in your case compact font mode helps ... you could enable it in 
cont-loc.mkxl (we might switch to that mode by default in the future) )

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------