Hi Oliver, I was a bit more ticked off yesterday than I really had right to, in retrospect. The big problem is that tex installations become more unpredictable the further they are away from Hans' private machine. Even my own machine(s) sometimes behaves differently from his, and mine in fact are extremely closeby. That means we already spend a lot of effort in packaging, and it is annoying when whatever we do is still not good enough. Anyway, here's some thoughts and explanations Oliver Buerschaper wrote:
Still I remember that from one MKII release to the next bugs kept creeping into the code that suddenly broke everything.
Yes, and even if we try really hard to not insert bugs, it simply cannot be helped. There is no regression test suite at the moment, and nobody seems motivated to create one. Hans and I certainly do not have time to set such a thing up. But even if there was a test suite: it would likely run for days on end, so we could probably not afford to run it before each release in any case. We've been working with separate "production", "beta" and "alpha" releases for a while, but that did not work either. The "alpha" release usually went untested outside of the Hans-me circle, so there was no point in having it. In fact "beta" releases had the same problem until Hans may ctxtools --update default to fetching the beta (if there was one newer than production). To make matters worse, then "beta" was (maybe still is) picked up automatically by Gerben Wierda's i-installer. So even our "test" releases suddenly appeared in users' production settings. The whole situation is why the museum was created: at least you will always be able to revert back to an older context release if something suddenly becomes broken in a production project.
Additionally, from the point of view of a package developer I keep having the feeling that more often than not the reasons for certain changes are left in the dark rather than explained. For example, my understanding is that many Perl scripts have been superseded by Ruby scripts over time, still they're shipped with current MKII minimals.
The perl scripts you mean? I don't know about that. I know there are a few that are not (yet) converted to ruby, if there really duplicates still in minimals, then the perl ones should be removed. Mojca, can you give some comments on this?
And then they seem to access dozens of environment variables and configuration files so you have no clue about whether those are needed or not. You just have to know. Which I don't.
Over the past two days, we have been working on getting rid of all of those environment variables. What we have so far only applies to mkiv, but doing a similar thing to texmfstart (and thereby all of mkii) should be relatively simple. First. let's talk about mkiv. Later today (maybe already by the time you read this) there will be a new beta release in which mkiv (to be exact: luatools.lua, mtxrun.lua and the format-including luat-inp.lua) should be able to do without any environment variables, assuming the following condition is met: luatex and texlua(c) have to be installed in a TDS compliant directory that is inside the search PATH. The lua code uses an internal form of autoparent discovery to find the directory where the texmf.cnf resides. (Btw, This is another one of those things where it worked for Hans in an hour, then it took two full days to get it to work for me. It is likely that another day or two are needed to make it work on texlive-based systems, and Miktex may take even more time. Well, it works for me now, so now it is time for other web2c-style people to have a try. We can worry about Miktex and /usr/bin/ installations later, ok?) Now for the details: For luatools --generate, the key task is to find the configuration file directory (or directories), and this is handled by input.identify_cnf in luat-inp.lua. First, if TEXMFCNF is set, It is traversed and all found texmf.cnf are read in. This is normal behavior for web2c based systems, and is not changed from last week's logic. But if TEXMFCNF is not set, MkIV will attempt to find a single default configuration file, based on the location of the luatex or texlua binary (and that should be something like /opt/tex/texmf-linux/bin). It actually traverses PATH to find the executable with the name that was in argv[0]. I suspect that absolute path names will also work, but that is untested (and nonstandard). So, assuming /opt/tex/texmf-linux/bin, it attempts the following directories, in this order: [/opt/tex/texmf]/web2c % two parents up [/opt/tex/texmf-linux]/texmf/web2c % one parent up [/opt/tex]/texmf-local/web2c [/opt/tex/texmf-linux]/texmf-local/web2c Searching stops as soon as a texmf.cnf is found: you can not have multiple texmf.cnfs if TEXMFCNF is not set, and the directory tests are not all that elaborate. You cannot setup a chained TEXMFCNF inside of texmf.cnf, either. Please note this implementation logic is not set in stone yet. We preferred to start as simple as possible, but extensions may be needed. (currently we have a "works for me" system). Keep in mind that if your installation is really outlandish, you can just set TEXMFCNF in the environment explicitly. And that is nearly all there is to explain. I have MKIV running now with nothing changed in my system except an addition to PATH. ------ However, this all applies to luatools.lua in particular, because luatools parses the texmf.cnf and stores the preprocessed version inside the luatex-cache tree. But there is a bit of extra functionality we have had to add: you may not like the luatex-cache to appear in your $HOME. It is not possible to set TEXMFCACHE in texmf.cnf because of a chicken-and-egg situation, so we came up with something else. Alongside each texmf.cnf (more exact: in each directory where a texmf.cnf can exist), there can be a lua file with name texmfcnf.lua. All such files will be read before any of the texmf.cnfs, and so they can be used to overrule texmf.cnf values, TEXMFCACHE in particular. The lua script has to return a table. Mine (I have only one) looks like this: config = {} config.TEXMFCACHE = "/tmp" return config ------- Comments and testing are welcome, of course. Best wishes, Taco