[Dev-luatex] Extension language integration of LuaTeX and LilyPond

David Kastrup dak at gnu.org
Tue May 7 17:57:03 CEST 2013

Taco Hoekwater <taco at elvenkind.com> writes:

> Hi David,
> On May 7, 2013, at 1:13 PM, David Kastrup <dak at gnu.org> wrote:
>> I've added some rough sketches at the end of the article that should
>> make clear why this can't be done in formats alone but will require
>> primitive support as well if things are supposed to turn out nicely.
> My brain is wired such that code in the scheme language never
> clarifies anything, I fear.

Well, one can at least look at its input, output, and the syntax and
workings of individual syntactic elements not yet wrapped into any

The implementation detail is a walk through LilyPond internals to give
an impression how that apparent magic is wired.  Of course, the wiring
for Lua/LuaTeX would be completely different, but that it's possible to
peek at how GUILE/LilyPond manages to do the raw work before coating it
in parens and at least take away the impression that it is possible to

> Nevertheless, I picked up something interesting from your postscript,
> and that is the tex:get_dimen() construct. Ignoring the catcode issues
> for the moment, I feel that the main reason why it is so hard
> currently to interface TeX -> Lua at the moment is the
> near-uselessness of the 'token' library. Extending that library (and
> its embedding) in such a way that it can reliably fetch various
> 'units' like dimensions and skips and integers from the input (either
> file or tokenlist), with or without expansion, would help a lot, IMO.

One feature of good programming language and interface design in my
opinion is that you are finished after writing down a naïve sketch.
Without then having an additional implementation phase where you are
just fighting technicalities.

So if a naïve sketch would include something like "get a dimension",
that's what a programming interface should provide.  A programmer wants
to get _his_ work done, not that of the platform.

In LilyPond, the point of functions like "define-music-function" is that
you have mechanisms to write stuff in Scheme that can get at arguments
and values just in the same way as LilyPond itself does.  For one thing,
it means that quite a bit of LilyPond functionality can actually be
implemented in Scheme, and thus can be redefined and/or rewritten
without needing to recompile.  For another, Scheme is a user-level
programming language.  It does its own garbage collection and memory
management, checks its types, and is reasonably expressive.

I've encountered people without programming background who started
working with LilyPond or Emacs and reached impressive mastery of them
without ever getting the hang of low-level languages like C.

Actually, one of such people is a major financial backer of my ongoing
work on LilyPond since working on beating consistency into a programming
model that was a loose connection of independent hacks at one point of
time was what empowered increasingly effective work of his.  That's what
the "It is whole now as if it had never been broken." quote at the start
of the article was about.

> The Lua -> TeX route is better covered at the moment, take
> e.g. ConTeXt's CLD files and Patrick's typesetting engine that is
> almost completely lua-based.

But Context's CLD files are not part of the LuaTeX mindframe/universe to
any serious degree.  You never learn what they are by reading the LuaTeX
manual.  So their ways of doing work are Context's ways of doing work,
not LuaTeX's ways of doing work.

The LuaTeX manual at the current point of time is as relevant for
resolving the question "why would I _want_ to use LuaTeX?" as a proof
that you can use it for implementing some particular class of Turing
machine.  It shows more abstract capabilities rather than actual
application programming strategies.

Now it is nice that somewhere there are also answers to the question
"why would I _want_ to use Context?".  But treating them as separately
important from the question "why would I want to use LuaTeX?" might
expand the number of people interested in LuaTeX.

David Kastrup

More information about the dev-luatex mailing list