I've finished testing my lua script, and it does exactly what I need. I
think I'll write a small article about it for one of the next context
group proceedings, but wanted to just give a very brief summary that
might be of interest to some:
Pure Lua is wonderful, but as the language is deliberately kept very
small, one sometimes has to find rather cumbersome workarounds. If you
run Lua under ConTeXt (with mtxrun --script), you get a version with
batteries included, which is perfect for manipulating and analyzing
text. Some areas that I found particularly impressive:
1. I've tried several times to make use of lpegs. Hans' code is full of
it, and I know that it's fast and extremely versatile, but I found it
difficult to wrap my head around its functioning and write useful code
with lpegs. The ConTeXt wrapper makes this extremely easy; this is lpeg
for the rest of us. It's really a delight to build patterns and see them
work immediately!
2. The utf library and string manipulation with characters.(...) is
absolutely necessary if you want to handle non-ASCII text because pure
Lua gives very unexpected results in this area. These operations work
wonderfully with the context libraries.
3. Lua's handling of tables is very efficient and fast. For analyzing my
Greek texts, I have to use huge tables for morphological parsing, with
more than 900,000 entries. Looking up words in these tables is around 3x
faster in Lua than in python!
One final thought: one limitation that I still find cumbersome to work
around is the fact that associative arrays ("pairs" in Lua speak) do not
have an order. When I analyze my texts, I want book numbers, chapters,
paragraphs preserved in the order in which they are read (entered into
the table). In many cases, it is not possible (or extremely awkward) to
sort these numbers, since chapters may be numbered something like 2, 2a,
3, 3α, 3β etc. python has the OrderedDict() in its collections module.
In Lua, the best I could find was entering the chapter numbers into an
array (ipair) and then retrieve it from there. Maybe there is a better way?
All of this just to say how grateful I am for the way Hans, Taco,
Wolfgang, Luigi, and the other developers have enhanced Lua. Thanks
guys, you make my work much more pleasant and efficient!
Thomas