On 5/14/2018 9:36 PM, Henning Hraban Ramm wrote:

Hi Hans, thank you so much!

Am 2018-05-14 um 17:17 schrieb Hans Hagen :

...

- Get rid of inconsistencies in the user interface e.g. by introducing new commands with settings.

+1

...

- Check what additional features users want (miss) and decide to what extent and with what priority we will put effort in this. We've reached a point where interference prevents more complex extensions.

* Recently I had some difficulties with (foot)notes, you’ll remember the margin notes thread. There are still some things that don’t work/look as I’d like them to, and I hope also others would welcome more flexible/simpler placement options.

Notes and margins will always be somewhat tricky esp used with other features because in the end tex has to make a choice .. in that respect i think that fully automated typesetting will never be ok.

* User/list/structure variables: Probably it’s just me, but there are a few difficulties.

hm, they're just data carried with whatever items that have accessors ... and thre tightly bound so real problems cannot happen there

* References: see Massimiliano’s question, apparently there are options missing for code before/after the list of pages

These hooks were added; adding a few more hooks is normally no big deal ... the main problem with such additions is that they seldom get documented (in fact that imo is also a it up to the one requesting such features).

* ePub: I got requests for ePubs again and will look into what’s still wrong/difficult. Last time I checked, footnote markers were accumulating in strange places. Maybe some more default constructs could get a representation in export XML, e.g. \bf, \it

It's hard to comment on that one without examples ... keep in mind that the xport is *not* meant as visual companion to the pdf but as a kind of representation of structure as seen by context ... any reordering due to typesetting can reflect that unless one does the obvious: for the expor make a special run with dedicated settings (like making notes end notes) ... notes (and other inserts) are a separate thing in the tex engine and have an asynchronous character. The better the input structure the better the export, and fro real robust multiple usage just code in xml. Visual properties (e.g. fonts) are not part of a structure, but you can just use highlights and such as these get tagged (with classes) so one then can relate them to e.g. fonts or colors. Keep in mind that \bf is not a construct, but a highlight named 'important' is. We can't make structure from non-structure. (FWIW: as long as we don't have projects that need this the priority for such features - the export basically started as a proof of concept - they get a low priority.)

* There are a few things missing in image handling - when I wrote my placement macros I needed image size calculations that already exist in ConTeXt but were not easily accessible.

hm, there's a lot available (actually also already in mkii) but i fear that there are too many variantions in demands .. there is a rather extensive plugin mechanism too (but not that documented) ... if some info is missing it can be added (although i don't think that there is more available in the engine that we expose)

* Would it be possible (or is it already?) to place stuff on layers and let "everything else" run around it? E.g. for half-page images I’m having a hard time using the float placement, while I could simply place images on a layer.

I'm not sure what you mean here but for the main flow we only hav eside floats. However, normally everything can be put on / moved to layers (iir the details manual shows some of that) but tex will never be loks dtp. Again, when i'd need that (given that I know what that means) for a project thaen i'd probably look into it. I've learned that much can be done in tex but not all without interference with other mechanisms.

...

- Are there reasonable challenges left.

* ePub

the export + a (dedicated) css ... exports have to be generic ... i must admit that i gave away my ebook reader as i never use(d) it and bying a new one is not on the agenda/budget (now)

* PDF/* (X, A, UA - probably only reasonable with external tools, but it seems like there’s not a lot that’s usable)

we support various formats (to some extend the backend even adapts to it) ... tagged pdf ... already there for quite a while but i never had any demand for it so i never really check the current state (also because imo it's a it weird feature ... only there because publishers don't want to distribute sources that are suitable for rendering variations ... so we're stuck with some pseudo structure related to visuals) (i might upgrade tagging and exports as they closely relate but it's not easy to get motivated for something that one has no real use for so at most it will cold winter evening stuff)

* even better error messages - often you get some obscure errors if you just miss a brace or bracket somewhere. I guess that’s a big hurdle for beginners.

not trivial ... related ot the fact that expansion is involved and macro languages are like that

* documentation... (sigh) While I’m trying to enhance our wiki and my book on ConTeXt, I often need to search the code base for undocumented options or usage examples, sometimes on stuff that looks simple or would be simple to do in InDesign...

well, there's already a lot (also in the distribution) and i try to keep up with new things but i don't see it as my duty to sit down and write tons more (luatex/context comes for free and it takes a lot of time and effort ... one can hardly complain so i don't feel too guilty about is .. documentation is also up to the users)

...

LuaTeX 1.09: - We expect the ffi interface to external libraries to become more stable over time. ConTeXt will not introduce dependencies (what can be done in Lua will happen in Lua) but on the other hand we might put some libraries in the distribution e.g. for database support.

Sounds good. I could use JSON, SQLite and/or MySQL support and some user interface stuff – at the moment my invoicing solution is a mixture of Bash, Python, Lua und ConTeXt, I’m already using a minimal OO library (classy.lua) and was looking into tekui, but got stuck... Maybe I should better look into the ConTeXt web framework as GUI. Or finally finish my (Django based) web shop and write a ConTeXt backend for PDF generation... Anyway.

sure, variation is welcome .. docker is also on some users' radar json: there's already a parser in context for many years (i needed one in the past), there's also a good performing sql framework present with support for mysql and sqlite (i also needed that in the past .. in fact some of our rendering on demand runs on luatex (lua interoreter) that way). (Many years ago i demo-ed sql support at a ctx meeting.)

Some default imaging solution would be nice (GraphicsMagick library).

i've played with that but in the end doing extensive manipulations runtime makes no sense .. i might pick up that thread when i can find a reason (but it's a huge set if libs and as such fragile wrr versions) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

On 5/15/2018 12:52 AM, Henri Menke wrote:

Math typesetting is really crappy in ConTeXt, but I get that this is beyond your priorities.  I plan to develop a module which resembles the features and syntax of the amsmath LaTeX package for my PhD thesis.  I'm not sure how well this will integrate with the existing mechanisms.

hm. i have no clue what you refer to ... afaik most is configureable

- columnsets, the new ones have considerably fewer features than the old ones.

like ... but adding some is no problem (only predictable stuff) .. no column handler suits all (we now also have page columns btw)

- rowwise setups in xtables and maybe columnwise, but that is computationally expensive.

indeed so that's why we have categories instead

...

- We can add more trickery for fonts and scripts. There are some pending extensions.

- Maybe we should provide a few more general styles.

What does that mean?  Things like the TUGboat style?

no, e.g. some basic educational stuff

More callbacks.  I'm missing callbacks into error handling (i.e. intercept errors) not just into error reporting like show_error_hook.

if you want to intercept errors then that has to happen at the macro level, because once tex starts expanding the error can be anywhere (so, in a macro package one can set at the tex level flags that one can act upon in the error callback) (the eror messages themselves might become a layer but that's for later)

Throw out all non-Lua-related primitives and ntg-context@ntg.nlreplace them with Lua functions.  People can then define those primitives themselves, e.g.

way too slow ... in that case i'd drop tex completely (i.e. do all in lua) also, you can right now (re)define primitives if you like (depending on the definition of primitive)

\suppressoutererror

becomes

\protected\def\suppressoutererror{%     \directlua{errors.suppressoutererror()}}

This makes it much easier to access that stuff from Lua.  Also interface all the \pdfvariable and \pdfextension stuff to Lua.

all pdf stuff is already doable in lua (context doesn't even use \pdf* for quite a while)

This should have maybe been done before 1.0 but I guess with 2.0 you can introduce “breaking” features.

well, a 2.0 (if ever) will probably only be useable for context ...

LuaJIT will always be 5.1 compatible.  That is one of the declared goals of the project.  However there exist compatibility layers for Lua which implement recent features for older interpreters. https://github.com/keplerproject/lua-compat-5.3

in that case in the end it will be dropped ...

I would rather not see LuaJIT support being dropped.  The VM by itself (without JIT) is already a lot faster than regular Lua and I feel that the ConTeXt runs benefit from that quite a lot.  I use contextjit as my daily driver.

hm, at most 20% which is also what i get when i buy a new laptop keep in mind that luajit has some limitations (stack and such) (and the last few years i managed to squeeze out a lot from lua, and with lua 5.3 the gaps became narrower) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

...

* References: see Massimiliano’s question, apparently there are options missing for code before/after the list of pages

These hooks were added; adding a few more hooks is normally no big deal ... the main problem with such additions is that they seldom get documented (in fact that imo is also a it up to the one requesting such features).

Touché! I've added them to the wiki. I was already updating that page of the wiki (\setupregister) on Friday, but also realized that it was way behind the syntax you can check with setup-en.pdf (the Commands manual). It missed the brand new "pageleft" and "pageright" parameters, but even much more than them. So i asked myself about the opportunity to document a brand new feature without the missing ones that have been already in place for a much longer time. Anyway, that's the spirit of a wiki: everybody can add his 2 cents of knowledge. The table of the command parameters was also difficult to read and modify in the wiki source, so i gave up for that moment and wondered about a more general solution for command documentation. Your messages today motivated me to update that page anyway. The setup-en.pdf manual is a great source of documentation and it's synchronized with every new version, because it's automatically generated from sources. It lets you know the commands, their parameters and their types. I've looked a bit into the "i-" files in the "interface" directory of the ConTeXt distribution, and i've seen that the commands are also classified into categories. What you miss in that manual is the parameters' meaning and some example of usage. I'm thinking about some companion files to the "i-" ones that carry the information about the parameters' meaning and usage. Some of those texts could be "recycled" from the current wiki pages. Then we could generate some automatic, up to date wiki pages directly from the combination of those files. I'm not going to work on that before the meeting and i've already promised more than i can achieve. But we could discuss that at the meeting. Greetings, Massi

On 5/15/2018 9:17 PM, Pablo Rodriguez wrote:

On 05/14/2018 05:17 PM, Hans Hagen wrote:

...

Hi,

The ConTeXt meeting is - as usual - the right place and moment to discuss the roadmap. We never had real binding roadmaps, more informal ones. Anyway, here are some thoughts on the two main components: MkIV and LuaTeX.

ConTeXt MkIV: [...] - Check what additional features users want (miss) and decide to what extent and with what priority we will put effort in this. We've reached a point where interference prevents more complex extensions.

Hans,

I wonder whether it would be possible to implement a feature that you mentioned in the past.

In order to avoid widow and orphan lines, you mentioned that it would be possible to automatically adapt the interline space for the page, so that it may have one more line to avoid orphans (when interline space is decreased) or it may move a line to the next page to avoid widows (increasing the interline space in the present page).

Would it be possible to add this feature to ConTeXt?

actually it was something that came up when talking with Hermann Zapf years ago: he suggested to just vertically scale the text area (i actully implemented that an hour later) ... his opinion was that only a very small percentage of readers will notice (at that time we did experiments with hz, the expansion in pdftex: in fact nobody noticed that too, real interesting was that texies commented on all kind of things related to how tex is supposed to work: a clear demonstration that the average user knows what the virtues of tex are but not really sees it)

I have another feature request for notes, but I would like to know whether the feature to avoid widow and orphan lines can be implemented in ConTeXt. you can always ask ... giving it priority is another matter

Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

If I am not mistaken, you can write : node.new(‘noad’, nodes.noadcodes.bin) De : Henri Menke Envoyé le :mercredi 16 mai 2018 07:39 À : ntg-context@ntg.nl >> mailing list for ConTeXt users Objet :Re: [NTG-context] roadmap I just thought of another thing. Could you expose _all_ the subtypes of _all_ the different node types similar to node.id? (currently this is only enabled for whatsits) As of now I always have to go to texnodes.w, find the array and count to find out which number a subtype has. It would be much easier if we could node.new("noad", "bin") instead of node.new("noad", 4) and I would also like to see node.subtype("noad", "bin") -- return 4 On 15/05/18 11:34, Hans Hagen wrote:

On 5/15/2018 12:52 AM, Henri Menke wrote:

...

Math typesetting is really crappy in ConTeXt, but I get that this is beyond your priorities.  I plan to develop a module which resembles the features and syntax of the amsmath LaTeX package for my PhD thesis.  I'm not sure how well this will integrate with the existing mechanisms.

hm. i have no clue what you refer to ... afaik most is configureable

...

- columnsets, the new ones have considerably fewer features than the old ones.

like ... but adding some is no problem (only predictable stuff) .. no column handler suits all (we now also have page columns btw)

...

- rowwise setups in xtables and maybe columnwise, but that is computationally expensive.

indeed so that's why we have categories instead

...

...

- We can add more trickery for fonts and scripts. There are some pending extensions.

- Maybe we should provide a few more general styles.

What does that mean?  Things like the TUGboat style?

no, e.g. some basic educational stuff

...

More callbacks.  I'm missing callbacks into error handling (i.e. intercept errors) not just into error reporting like show_error_hook.

if you want to intercept errors then that has to happen at the macro level, because once tex starts expanding the error can be anywhere

(so, in a macro package one can set at the tex level flags that one can act upon in the error callback)

(the eror messages themselves might become a layer but that's for later)

...

Throw out all non-Lua-related primitives and ntg-context@ntg.nlreplace them with Lua functions.  People can then define those primitives themselves, e.g.

way too slow ... in that case i'd drop tex completely (i.e. do all in lua)

also, you can right now (re)define primitives if you like (depending on the definition of primitive)

...

\suppressoutererror

becomes

\protected\def\suppressoutererror{%      \directlua{errors.suppressoutererror()}}

This makes it much easier to access that stuff from Lua.  Also interface all the \pdfvariable and \pdfextension stuff to Lua.

all pdf stuff is already doable in lua (context doesn't even use \pdf* for quite a while)

...

This should have maybe been done before 1.0 but I guess with 2.0 you can introduce “breaking” features.

well, a 2.0 (if ever) will probably only be useable for context ...

...

LuaJIT will always be 5.1 compatible.  That is one of the declared goals of the project.  However there exist compatibility layers for Lua which implement recent features for older interpreters. https://github.com/keplerproject/lua-compat-5.3

in that case in the end it will be dropped ...

...

I would rather not see LuaJIT support being dropped.  The VM by itself (without JIT) is already a lot faster than regular Lua and I feel that the ConTeXt runs benefit from that quite a lot.  I use contextjit as my daily driver.

hm, at most 20% which is also what i get when i buy a new laptop

keep in mind that luajit has some limitations (stack and such)

(and the last few years i managed to squeeze out a lot from lua, and with lua 5.3 the gaps became narrower)  Hans

-----------------------------------------------------------------                                           Hans Hagen | PRAGMA ADE               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___________________________________________________________________________________

On 05/15/2018 11:26 PM, Hans Hagen wrote:

On 5/15/2018 9:17 PM, Pablo Rodriguez wrote:

...

[...] Hans,

I wonder whether it would be possible to implement a feature that you mentioned in the past.

In order to avoid widow and orphan lines, you mentioned that it would be possible to automatically adapt the interline space for the page, so that it may have one more line to avoid orphans (when interline space is decreased) or it may move a line to the next page to avoid widows (increasing the interline space in the present page).

Would it be possible to add this feature to ConTeXt?

actually it was something that came up when talking with Hermann Zapf years ago: he suggested to just vertically scale the text area (i actully implemented that an hour later) ... his opinion was that only a very small percentage of readers will notice (at that time we did experiments with hz, the expansion in pdftex: in fact nobody noticed that too, real interesting was that texies commented on all kind of things related to how tex is supposed to work: a clear demonstration that the average user knows what the virtues of tex are but not really sees it)

I must confess that I can recognize hanging characters, but quality expansion is impossible to see (at least, for me). I would even say that if expansion can be seen by a human reader, the results will be crappy. How can I use the vertical scale feature to prevent widows and orphans? Many thanks for your help, Pablo

...

I have another feature request for notes, but I would like to know whether the feature to avoid widow and orphan lines can be implemented in ConTeXt. you can always ask ... giving it priority is another matter

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___________________________________________________________________________________

-- http://www.ousia.tk

On 5/16/2018 9:14 PM, Fabrice L wrote:

Dear all,

...

- Check what additional features users want (miss) and decide to what extent and with what priority we will put effort in this.

As asked, I add a wish to the list...

One feature which I depend a lot on is to be able to do animations: I actually use the Raw Steps module, by David Munger (dated from 2006), which still worked, but in MKII only. This is essential for me for my teaching and talks. During teaching, I have designed courses notes in ConText using a lot modes of context: the notes are different for the teacher, are available in two formats for the students (paper and for completion on tablets), and there is a last version for use in the class by me. In class, this is essential for me that the material is presented by steps, otherwise students have a tendency to not listen, or to not try to solve a problem is the solution is already on the screen. I can not use the animation tools available in ConTexT using javascript, since I use to show notes in class an iPad, and I use the stylus to fill blanks present in the notes (which work great by the way !). The facility to use modes in the courses notes like this have convinced several of my colleagues to use ConTeXt for their teaching needs.

Following threads on this list on animations, my understanding is that Hans is not a great supporter of the method of animations in the Raw Steps module, for technical reasons.

I have switch to MKIV for others documents, but I'm still on MKII for this reason for my courses notes, which are 90% one my needs in TeX. Fabrice. A few remarks:

(1) The mkiv approach uses not that much javascript actually, it uses layers that get turned on and off and it has a simplicity that i like, also because there is no interference with repetition (e.g. of references) cq. are no side effects (like bad spacing). Unfortunately layers are hardly supported and the trivial javascript to control some aspects of viewing neither. (Luigi and I have been thinking of adding lua to a mupdf based viewer ... maybe we should pick up that thread.) (2) Steps i.e. repetitive content has the problem that one then needs to make sure references get flushes only ones which also demands care by the user, and spacing not be influenced too badly by redoing chunks. This is why I made some explicit steppers (there are some in the mkiv part as s-present* files). I can give it some thought but it's not that trivial to do it right (according to my specs). Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

Hi Mohammad Hossein, Thanks for your work regarding the RawSteps! However I tried just now the examples you have on github, but it seems that it does not work out of the box: ConTeXt reports an error at line 114 of p-rsteps.tex, complaining about an « Undefined control sequence » which points to \catcode`\^^M=\active Maybe you have fixed the issue now, but regarding the feature request put forward by Fabrice, I agree with him that such a stepping feature would be nice to have in mkiv, since the steps commands which exist now cannot be used in all circumstances. Best regards: OK

On 17 May 2018, at 16:24, Mohammad Hossein Bateni wrote:

Fabrice,

Accepting the caveats Hans pointed out for problematic spacing and some issues with references, you might find [https://github.com/bateni/rawsteps-mkiv] useful. I ported RawSteps to MkIV and have used it in a few presentations. The version on github might be buggy, but a good start if you really insist on using RawSteps.