Hi Andrei, Hans, Hraban, On Sun, 2025-07-27 at 17:59 +0300, andrei@borisov.dev wrote:
We’ve been working on adding LuaMetaTeX and ConTeXt standalone support for the Markdown package (https://github.com/Witiko/markdown/pull/557) and encountered an interesting issue.
The package loads quite a huge library — `expl3-code.tex` from `l3kernel` package which eats a lot of time during compilation.
What would be a good way to mitigate this?
Some options: - Manually install v2.13.0 of the Markdown package, which doesn't require expl3. - Use pandoc: https://github.com/gucci-on-fleek/lua-widow-control/blob/b08ddbcd/docs/manua... - Convince Vit to rewrite the ConTeXt interface in Lua, similar to https://github.com/Witiko/markdown/issues/215#issuecomment-1359250887 - Use the builtin Markdown module, which was just updated in yesterday's release.
Is there a way to pre-load the library before compilation? Perhaps by making a custom ConTeXt format and making it with `context --make`?
Unfortunately, I haven’t found a documentation on how to make your own custom formats, hence this question.
This is a terrible idea, and you definitely should not recommend any of your users to do this, but if you're just looking for a hack to speed up your CI/testing, it is possible. The trick is to override one of ConTeXt's core files (which is why this is a terrible idea). "libs-ini.mkxl" is a good choice for this since it's loaded near the end and isn't used by most documents. So, make a file called "$TEXMFHOME/tex/context/third/libs-ini.mkxl" with the following contents: \directlua{function pdf.getcreationdate() end} % expl3 bug \usemodule[expl3-generic] and then you can run $ TEXMFHOME='{/PATH/TO/TEXLIVE/texmf-dist/tex/latex-dev/,/PATH/TO/TEXMFHOME/}' context --make You need to put "latex-dev" into the search path since only the prerelease expl3 versions currently work with ConTeXt. Again, I don't recommend this since it is very likely that a future ConTeXt update will break this, and building expl3 into the format might break core parts of ConTeXt, but it's certainly possible.
What is the process of adding thrid-party modules in the default list of ConTeXt distribution (https://modules.contextgarden.net/)?
It's fairly simple, you essentially just need to sign up for an account and then upload a zip file. However, the ConTeXt Standalone Distribution doesn't include expl3, so just adding the Markdown module alone probably wouldn't be very useful. On Sun, 2025-07-27 at 17:53 +0200, Hans Hagen via ntg-context wrote:
Actually, one can wonder what expl3 bring to context at all; some intermediate layer like that just doesn't fit in.
No one would ever write a ConTeXt-native module using expl3, but if you're using it for a LaTeX package anyways, then the ConTeXt support comes for "free" (from the developers side; you'll still pay for it in a much slower runtime).
So, can't you avoid loading the bottleneck expl code? Skip unicode stuff as a start? You probably then end up below a second. Boosting that code (by looking at it i might spot some) is not on my agenda.
Most of the Markdown module is written in Lua; only the TeX interface parts use expl3. I suspect that the easiest option would be to simply rewrite the TeX interface for ConTeXt using Lua. I actually tried doing this in March 2024 (and then got distracted and just switched to using Pandoc); the Markdown module has changed its interface since then, but it still works with TL23 and should be easy to port to the latest version. I've attached the file that I used, so feel free to use that as inspiration. On Sun, 2025-07-27 at 18:07 +0200, Henning Hraban Ramm wrote:
Am 27.07.25 um 16:59 schrieb andrei@borisov.dev:
The package loads quite a huge library — `expl3-code.tex` from `l3kernel` package which eats a lot of time during compilation.
What has expl3 to do with ConTeXt?
It's a generic programming layer for TeX, much like pgfkeys/pgfmath (the non-graphics parts of TikZ). It was originally designed for LaTeX, but its authors have put quite a bit of effort into making sure that it's usable in all formats, including ConTeXt. Thanks, -- Max local markdown = require "markdown" local headings = { "chapter", "section", "subsection", "subsubsection", "subsubsubsection", } local fmt = string.formatters local _writer = { ellipsis = [[[\dots]], code = [[\type{%s}]], space = [[\space]], hard_line_break = [[\crlf]], nbsp = [[\nobreakspace]], strong = [[\bold{%s}]], emphasis = [[\emph{%s}]], inline_html_tag = [[\type{%s}]], block_html_element = [[\type{%s}]], verbatim = [[\type{%s}]], thematic_break = [[\blackrule]], interblocksep = [[\par]], string = false, paragraph = false, plain = false, inline_html_comment = function() end, block_html_comment = function() end, document = [[\starttext %s \stoptext]], blockquote = [[\startquotation %s \stopquotation ]], link = function(label, url, title, attributes) return fmt[ [[\goto{%s}{url(%s)}]] ](label, url) end, image = function(label, url, title, attributes) return fmt[ [=[\externalfigure[%s]]=] ](url) end, bulletlist = fmt[[\startitemize %s \stopitemize ]], heading = function(content, level) return fmt[ [[\%s{%s}]] ](headings[level], content) end, } local writer = table.setmetatableindex({}, function(t, k) local func = _writer[k] if func == false then func = fmt["%s"] elseif type(func) == "string" then func = fmt[func .. "\n"] end return function(...) local args = { ... } for i, v in ipairs(args) do if type(v) == "table" then args[i] = table.concat(v) end end local out = func(table.unpack(args)) print(out) return out end end) local convert = markdown.reader.new(writer, { html = true, shiftHeadings = 1, eagerCache = false }).finalize_grammar({}) local out = convert[[ # Title Hello *World* ]] context.startext() context(out) context.stoptext()