[NTG-context] suggestions for context documentation

Hans Hagen pragma at wxs.nl
Mon Mar 8 13:31:28 CET 2010







Hi all,

As I was on the Dante 2010 meeting and as I could not access my
mail last week, today I ran into the long thread about
documentation. I will not reply to each mail but stick to this
summary. Now, I understand that there is a lack of documentation
but before one complains too loud about it, consider the


\startitem We started with \CONTEXT\ in the early 90's and it went
public around 1995. So, we're 15 years down the road. Whatever
comment one has on the system, including its documentation, has to
be seen into this light: we're looking back at over 15 years of
development and ahead at quite some more. \stopitem

\startitem In all those years we've been writing a lot of code,
not only for our own use, but also for users. Just to mention a
few areas: specific language and font support is non trivial in
traditional \TEX\ and took quite some time and backends change and
being involved in the development also brings a price (in many
aspects). \stopitem

\startitem What started a system for our own use, is now used by
others as well, and this brings not only the responsibility to fix
bugs fast but also to monitor lists etc.\ Add to that quite some
involvement in \TEX\ user groups, conferences, writing articles
etc.\ In the process we also happen to come up with some manuals.
Just wonder for a while where I find the time to do my regular
work (the work that pays the bills). \stopitem

\startitem Even an old manual can quite well describe
functionality as much didn't change. It's only with \MKIV\ that
some compatibility is dropped and only for obscure features. Of
course I could trick users by regenerating a manual with a newer
date. I often use the excellent book \quote {\TEX\ by Topic} which
is already quite old and does not cover \ETEX, \PDFTEX, \LUATEX\
or whatever but what is told in it is still true. I never look at
it thinking it being old. \stopitem

\startitem As one can visually get all kind of output and as
typographical elements can interfere the ultimate manual would
show $n!$ variants and become quite unreadable. There is no easy
way out of this. For other languages there's a lot of code
googlable but I find myself always writing from scratch as each
case seems to be different. Of course printed manuals can be of
help (the \LUA\ book being a very good example of a manual) but
writing one takes time. \stopitem

\startitem More documentation would not help all users. Some are
better of with a simple manual and some occasional help on the
list. I've been using all kind of programming languages and the
fact that some have huge (auto generated) documentation systems is
no guarantee that they can be used. I find myself quite often just
look in the source to see what is (not) happening and then
probably feel as confused as users looking into \CONTEXT\ sources.

\startitem There are quite some options that were never meant for
usage beyond our own, but as we ship the full product, they become
visible. No, they are not documented apart from the source. Yes,
if useful they should be documented but why by me? \stopitem

\startitem Any comparison with \LATEX\ documentation is useless.
One reason for starting to write \CONTEXT\ is that I didn't
understand the \LATEX\ book that well as well as that for proper
non English usage one had to patch unreadable code. When \CONTEXT\
came around the internet and mail were already replacing articles
and books. Just look at how the content in user group journals
changed from beginners explanations to more expert and niche
topics. Also, the fact that new books about \LATEX\ are still
written means that there is no perfect one yet. If you ever run
into one of the authors of the companions, just ask them how much
time it took \unknown\ close to a lifetime I bet. \stopitem

\startitem There was some comment on me being the only developer.
This might be true to a large extend but Taco, Wolfgang, Aditya,
Mojca, Luigi (I mention just a few currently active developers and
feel sorry for those I forget so feel free to amend me) know their
way around the source quite well and contribute patches too. We
don't have a formal team (as that would introduce the problem of
adding|/|removing active members and I like to be more informal
and users on the list know pretty well who are contributors.

\startitem There is no real cutting|-|out going on, it's just the
way it evolved. On the other hand, as I use the system myself I
would probably quit using it if anyone could push in code. It's
hard to keep \TEX\ doing what you want it to do and breaking it is
too easy due to interference. So, developers need some feeling
about what side effects can occur. Even then we see bugs creep in.

\startitem Believe me: when some folks send me a patch it goes in
without too much thinking. For instance, code that Wolfgang
submits fits perfectly in the \quote {design} of the system and I
trust any correction to math that Aditya sends me. At some point I
want to make it easier for those to work on the code directly but
it should not cripple my own workflows. \stopitem

\startitem At the last Dante meeting there were a some talks about
\LATEX. One was about how to let all those packages work together
(hard and error prone and unsolvable). Others were about what
package to use for what (similar) feature (without running into
conflicts). Arbitrary contributions are fine, but can become a
nightmare. Of course, for \CONTEXT\ one can write modules that
work quite well, as long as one uses the hooks provided and not
messes around with core commands. \stopitem

\startitem Just look at the excellent tutorials by Willi, Thomas,
Aditya and others (at \CONTEXT\ conferences for instance) and you
will notice that there is more info than on our website. \stopitem

\startitem Only the fact that we have a rather tight control over
development makes it possible for us to keep doing this. I work at
a quite small company and 75\% of my time goes into development
and support of \CONTEXT\ (and \LUATEX). To that you can add rather
specialized sub projects like the Oriental \TEX\ project where
we're talking of years of work just for free. It's might be true
that Taco and I are candidates for writing (and updating) manuals,
but we both have single|-|core minds. \stopitem

\startitem The user interface is indeed described in an \XML\
file. That info used to be spread over the source and actually in
the 90's it was used in a help system in an editor that only we
used. Eventually I might redistribute the \XML\ snippets over the
code again, but I'm not sure about it. Anyway, it's manual work to
keep that one up to date as we don't want experimental features to
exposed too soon, and because of inheritance issues. \stopitem

\startitem I don't use revision control in my source tree (after
all, when we started revision control was not that wide spread,
and I also don't have the patience for it) but all \CONTEXT\
releases are checked in eventually. I have been considering using
svn and might eventually use git on my machine but I'll only do it
when I do it for all \CONTEXT\ related code and that would mean
that I have to sort out 15 years of manuals, presentations,
articles, etc.\ I simply don't have time for that now (and it does
not gain me that much in my case). \stopitem

\startitem Concerning embedded documentation: this only makes
sense when \TEX\ is processing it as we need typeset examples.
Also, for me personally, I only work on a manual (or article or
whatever) if it's fun to do. The \METAFUN\ manual took me close to
a year of evenings (excluding the \MKIV update). Interesting is
that with \MKIV\ writing manuals is more convenient as embedded
\METAPOST\ runs are real fast now. \stopitem

\startitem The only generic manual markup I'm willing to use is
either \TEX\ or \XML\ (for multiple output). Of course others can
use whatever they want. When writing manuals the biggest issue is
not a bit of extensive tagging (so minimal markup is a no|-|go for
me personally) but the fact that we need to be able to process
code as examples which means that only \TEX\ (as \PDF) renderer)
can be used. I'm occasionally thinking of an \HTML\ backend but as
I don't need it I cannot spend time on it anyway. I might give
those pads a change when they get some quality, speed and battery
live. \stopitem

\startitem We're in the transition from \MKII\ to \MKIV. Most of
what applies to \MKII\ also applies to \MKIV\ but everything
encoding related is gone. Quite some font related stuff is new but
that's simply a side effect of going \OPENTYPE. Apart from that
most in the manuals is still quite valid. There are two history
documents (\type {mk.pdf} and \type {hybrid.pdf}) and they contain
info useful for developers but they are not advocated as user
manuals: they're test cases, historic overviews, development
notes, explorations or whatever description suits them. \stopitem

\startitem My experience is that \CONTEXT\ users are willing to
move on, and that comes with a price: if you use experimental
features you're mostly ok (as I use them myself interfaces are
unlikely to change anyway), but there is not much documentation
then. The documentation landscape would look less confusing if
we'd kick out \PDFTEX\ and \XETEX\ support (read: \MKII). We
should look forward. \stopitem

\startitem Several attempts have been made to come up with a test
suite (Sanjoy even implemented a test framework) but not that much
ended up in there: why should users do it if what gets posted on
the lists works okay? I am currently building a tree with test
files (based on snippets that get posted on the list) but I need
to sort it out before it gets into the source. So, we might revive
that project. \stopitem

\startitem Also, Luigi does a pretty good (and time consuming) job
on keeping the sources processable into module documentation. The
test suite will probably be part of Luigi's module documentation
system too (but he doesn't know that yet). \stopitem

\startitem Idris is writing a \CONTEXT\ book but even that cannot
cover everything. It's up to him to elaborate on that. \stopitem

\startitem You cannot compare a system with 20 years of history
with one of a few years. I'm not going to rewrite history and for
sure do things differently. Actually, the whole \LUATEX/\MKIV\
effort is sort of unique for the \TEX\ community as normally one
tends to stick to the past. As Taco and I are also part of
mainstream \TEX\ developments the users pay the price. \stopitem

\startitem Off topic: there is an \HTML\ variant of the \type
{cont-*.pdf} files, just run \typ {mtxrun --script server --auto}.

\startitem The \WIKI\ is a nice example of users contributions and
there are some quite good explanations on it. As with all
documentation, the more there is, the more confused users can get.
Users normally respond quite well when asked to wikify a solution.
I'm really grateful for all those user contributions and I hope
that Patrick will keep gardening and harvesting forever. \stopitem

\startitem The minimals are a nice example of an effort that moved
from being our work to users (mostly Mojca but as she's
multi|-|core, she counts for 10 users). I fear the moment when she
will say \quotation {enough is enough}. \stopitem

\startitem It has always been possible to add structured
documentation to the commands in the \WIKI\ interface but somehow
it never worked out well. Maybe a merge in the \WIKI\ will work
out ok. But \unknown\ it's users who have to fill it, as I simply
have no more time left. Also: write articles for the user group
journals. \stopitem

\startitem I have plans to put more manual code online. This
starts making sense now that internet access gets comfortable but
I will only do that with clean and properly coded documents so it
needs checking and therefore time. I only put stuff online that
makes me feel comfortable. Concerning the licences: these are
either the one that covers \CONTEXT\ or some creative common's one
(and then only a reference as I dislike wasting bytes (and time)
on those licences). \stopitem

\startitem Contrary to what users might think, a lot of code in
any \TEX\ macro package is written for the sake of documentation:
verbatim. Without verbatim much would look simpler (less
exceptions due to catcodes for instance). So, writing
documentation is no big deal: one only needs time. \stopitem



                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl

More information about the ntg-context mailing list