On 5/14/2017 6:43 PM, Idris Samawi Hamid ادريس سماوي حامد wrote:

On Sun, 14 May 2017 09:33:04 -0600, Hans Hagen wrote:

...

...

2. Modification: It appears that all command names that appear in slanted text are used to mention user-defined commands of a particular class, e.g.,

/\BACKGROUND/ - slanted text

which obviously names each of a class of commands defined by

\definebackground - regular text

Can a version of the auto-generation script be made that will generate a version setup-mkiv-strict-en.pdf that is restricted to mkiv commands explicitly defined in mkiv, excluding class commands or anything else represented in slanted text?

it's xml so one can benerate anything from the file (i was in the process of generating instances in the lua files for scintilla but (1) had to prepare for bachotex, (2) got back and ill, (3) need some motivation to pick up that thread)

Let's call commands like \BACKGROUND etc abstract commands; the rest are concrete. An abstract command constitutes the form of a concrete instance that is, e.g., defined by the user via a corresponding \define<instance>.

that's what i meant with "i'm working on getting these instances in the lua lists too" but it takes time (and motivation) so it will happen stepwise (normally i prioritize such things by need in projects but in the projects that we have the authors code in xml)

[Are there other kinds of abstract command not involving some \define<instance>?]

dunno

For text editors a generic list of all concrete mkiv commands is needed.

...

...

3. Although it is intuitive, a legend precisely explaining the semantics of the command list in setup.pdf would be useful. If there is already one available, could you point it out to me?

dunno, maybe somewhere on the wiki

The closest I could find is here:

http://wiki.contextgarden.net/Command_Syntax

but it does not contain a complete or precise legend. Such a thing would obviously be useful for users..

the command reference is quite complete (and user defined instances will never be part of syntax highlight anyway)

...

...

4. Would it be possible for the auto-generation to export the command list to a semi-formatted text file? Copying the text of setup-en.pdf to a text file gives usable results but a text file with some regular structure might be better.

one can (1) use some xml transformation or (2) use the lua tables to get strings

The lua tables in the scite distribution are incomplete. For example, in scite-context-data-interfaces.lua there is no mention of the commands for natural tables - \bTABLE etc. This is the reason for taking the auto-generation approach, to get a comprehensive and complete list.

could be but wolfgang did a huge effort in making them pretty complete (even low level commands) so ic something is not in there (the i-*.xml files), it's with good reason keep in mind that all this happens in spare time Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

On Sun, 14 May 2017 11:38:19 -0600, Wolfgang Schuster wrote:

...

...

The lua tables in the scite distribution are incomplete. For example, in scite-context-data-interfaces.lua there is no mention of the commands for natural tables - \bTABLE etc. This is the reason for taking the auto-generation approach, to get a comprehensive and complete list.

could be but wolfgang did a huge effort in making them pretty complete (even low level commands) so ic something is not in there (the i-*.xml files), it's with good reason

Sure, I was referring to scite-context-data-interfaces.lua, not the i-*.xml files.. Fortunately setup-en.pdf makes all of these concrete commands explicit: setup-en.pdf is as complete as one could ask for. But for editors the abstract commands are superfluous, as you pointed out:

...

the command reference is quite complete (and user defined instances will never be part of syntax highlight anyway)

OTOH user-defined commands can be added to the ConTeXt lexer via the Style Configurator (Notepad++) and get their own highlight color. I have found this very useful in writing long documents. See attached (User-defined Keywords dialog).

Environments with custom begin/end-strings (e.g. \bTR)

cd:arguments

get the default start/stop string in the scite files.

Ah, "setupTABLE" is listed in scite-context-data-interfaces.lua. Wolfgang: In that case, is there a way to generate an explicit list of all concrete commands that derive from the ["en"] class in scite-context-data-interfaces? If the results are sufficiently complete, we could distinguish high-level mkiv commands from the low-level ones. Such a list might be more beneficial for most users. Put another way, we could have mkiv-list-high - one syntax highlighting (say, bold) mkiv-list-low - second syntax highlighting (say, regular) OTOH, much of this is a matter of taste: I would argue that \unprotect and \protect are high-level (as part of the meta-language used to mark off low-level code) and should therefore go into scite-context-data-interfaces (not there at present). Idris -- Idris Samawi Hamid, Professor Department of Philosophy Colorado State University Fort Collins, CO 80512

Idris Samawi Hamid ادريس سماوي حامد mailto:Idris.Hamid@colostate.edu 14. Mai 2017 um 20:06 On Sun, 14 May 2017 11:38:19 -0600, Wolfgang Schuster wrote:

...

...

...

The lua tables in the scite distribution are incomplete. For example, in scite-context-data-interfaces.lua there is no mention of the commands for natural tables - \bTABLE etc. This is the reason for taking the auto-generation approach, to get a comprehensive and complete list.

could be but wolfgang did a huge effort in making them pretty complete (even low level commands) so ic something is not in there (the i-*.xml files), it's with good reason

Sure, I was referring to scite-context-data-interfaces.lua, not the i-*.xml files..

Fortunately setup-en.pdf makes all of these concrete commands explicit: setup-en.pdf is as complete as one could ask for. But for editors the abstract commands are superfluous, as you pointed out:

...

...

the command reference is quite complete (and user defined instances will never be part of syntax highlight anyway)

OTOH user-defined commands can be added to the ConTeXt lexer via the Style Configurator (Notepad++) and get their own highlight color. I have found this very useful in writing long documents. See attached (User-defined Keywords dialog).

You can limit the number of custom commands when you use \startnamedsection[topic][title=...] instead of \starttopic[title=...]

...

Environments with custom begin/end-strings (e.g. \bTR)

cd:arguments

get the default start/stop string in the scite files.

Ah, "setupTABLE" is listed in scite-context-data-interfaces.lua. \bTABLE is also listed but Hans script includes it as \startTABLE (because it ignores the begin/end attributes). Wolfgang: In that case, is there a way to generate an explicit list of all concrete commands that derive from the ["en"] class in scite-context-data-interfaces? If the results are sufficiently complete, we could distinguish high-level mkiv commands from the low-level ones. Such a list might be more beneficial for most users. Put another way, we could have

mkiv-list-high - one syntax highlighting (say, bold) mkiv-list-low - second syntax highlighting (say, regular)

OTOH, much of this is a matter of taste: I would argue that \unprotect and \protect are high-level (as part of the meta-language used to mark off low-level code) and should therefore go into scite-context-data-interfaces (not there at present). Adding tags to the commands is planned but the question is how to categorize them (internal, api, user level, low level, primitives, defininitions (\define...), setups etc.).

Wolfgang

Idris Samawi Hamid ادريس سماوي حامد mailto:Idris.Hamid@colostate.edu 14. Mai 2017 um 20:55 On Sun, 14 May 2017 12:20:43 -0600, Wolfgang Schuster wrote:

...

...

OTOH user-defined commands can be added to the ConTeXt lexer via the Style Configurator (Notepad++) and get their own highlight color. I have found this very useful in writing long documents. See attached (User-defined Keywords dialog).

You can limit the number of custom commands when you use

\startnamedsection[topic][title=...]

instead of

\starttopic[title=...]

First time encountering these two commands; they're not on the wiki, need to learn more about them..

Can you explain or give a complete sample test file illustrating how they apply in the matter under discussion? Thanks. This is only a example to demonstrate that you can create new commands without using new command names (and the need to add them to your lexer) in your document.

...

...

...

Environments with custom begin/end-strings (e.g. \bTR)

cd:arguments

get the default start/stop string in the scite files.

Ah, "setupTABLE" is listed in scite-context-data-interfaces.lua. \bTABLE is also listed but Hans script includes it as \startTABLE (because it ignores the begin/end attributes). Wolfgang: In that case, is there a way to generate an explicit list of all concrete commands that derive from the ["en"] class in scite-context-data-interfaces? If the results are sufficiently complete, we could distinguish high-level mkiv commands from the low-level ones. Such a list might be more beneficial for most users. Put another way, we could have

mkiv-list-high - one syntax highlighting (say, bold) mkiv-list-low - second syntax highlighting (say, regular)

OTOH, much of this is a matter of taste: I would argue that \unprotect and \protect are high-level (as part of the meta-language used to mark off low-level code) and should therefore go into scite-context-data-interfaces (not there at present). Adding tags to the commands is planned but the question is how to categorize them (internal, api, user level, low level, primitives, defininitions (\define...), setups etc.).

This is good to know. At the moment we're working on a full-featured ConTeXt lexer and are experimenting with different ideas of organizing commands for user-friendly syntax highlighting. Currently considering something like the following:

TeX primitives - Knuth luaTeX primitives - excluding Knuth The syntax files contain at the moment only ConTeXt specific command,

In the example below I create a new section but I don’t have to use a new user generated command in the document itself. \definehead[topic][subject] \starttext \startnamedsection[topic][title=This is a topic] ... \stopnamedsection \stoptext primitives aren’t included.

macro structure - e.g., sectioning etc. micro structure - e.g., itemizations, tables etc. mode structure - e.g., metapost, xml, markdown The XML source has a category attribute for this information. MkiV - remaining high level MkIV - low-level [but how to separate them out?] user-defined Internal commands can be grouped in official helpers (which should be documented) and internal helpers (which can be hidden). Perhaps collapse macro and micro structure into one set... Again, just ideas for now, will work this week to work them into a concrete organization for the lexer - once we generate a complete list of concrete mkiv commands.

If we can generate a complete version of the list in scite-context-data-interfaces (including \bTR etc.) then we can subtract that from the complete list of concrete mkiv commands, giving us mutually exclusive and jointly exhaustive high/low-level lists. The source for the command list are the XML files (search for i-*.xml) and specific lists can be generated by adding metadata to the definitions.

Wolfgang

On 5/14/2017 8:55 PM, Idris Samawi Hamid ادريس سماوي حامد wrote:

On Sun, 14 May 2017 12:20:43 -0600, Wolfgang Schuster wrote:

...

...

OTOH user-defined commands can be added to the ConTeXt lexer via the Style Configurator (Notepad++) and get their own highlight color. I have found this very useful in writing long documents. See attached (User-defined Keywords dialog).

You can limit the number of custom commands when you use

\startnamedsection[topic][title=...]

instead of

\starttopic[title=...]

First time encountering these two commands; they're not on the wiki, need to learn more about them..

in mkiv most commands that have generated names also provide generic instance handling, like here

Can you explain or give a complete sample test file illustrating how they apply in the matter under discussion? Thanks.

afaiks there is not much to add to the above example unless i misunderstood; basically one can always use "topic" as instance name to do something instead of \somecommandwith<topic>init

something like the following:

TeX primitives - Knuth luaTeX primitives - excluding Knuth macro structure - e.g., sectioning etc. micro structure - e.g., itemizations, tables etc.

here i have document level (\startdocument etc) vs the rest

mode structure - e.g., metapost, xml, markdown MkiV - remaining high level MkIV - low-level [but how to separate them out?]

is there a need to separate them? after all, most users will not use low level code and when they do it doesn't matter much to them if they're more or less low level low level is fuzzy anyway: \xmlfirst (more low level?) vs \xmlfilter (more generic)

user-defined

Perhaps collapse macro and micro structure into one set... Again, just ideas for now, will work this week to work them into a concrete organization for the lexer - once we generate a complete list of concrete mkiv commands.

If we can generate a complete version of the list in scite-context-data-interfaces (including \bTR etc.) then we can subtract that from the complete list of concrete mkiv commands, giving us mutually exclusive and jointly exhaustive high/low-level lists.

Any help and other ideas are greatly appreciated!

Idris

-- ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

On 5/15/2017 9:00 PM, Wolfgang Schuster wrote:

...

is there a need to separate them? after all, most users will not use low level code and when they do it doesn't matter much to them if they're more or less low level

low level is fuzzy anyway: \xmlfirst (more low level?) vs \xmlfilter (more generic) I can add class="system|primitive" (or level="...") to the XML files, commands without attribute are user-level.

i assume that with primitive you mean primitive context Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

On 5/15/2017 9:57 PM, Wolfgang Schuster wrote:

...

Hans Hagen mailto:pragma@wxs.nl 15. Mai 2017 um 21:45 On 5/15/2017 9:00 PM, Wolfgang Schuster wrote:

i assume that with primitive you mean primitive context No, TeX primitives.

cd:arguments

aha, so all these have to be entered then ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

On 5/14/2017 8:06 PM, Idris Samawi Hamid ادريس سماوي حامد wrote:

On Sun, 14 May 2017 11:38:19 -0600, Wolfgang Schuster wrote:

...

...

...

The lua tables in the scite distribution are incomplete. For example, in scite-context-data-interfaces.lua there is no mention of the commands for natural tables - \bTABLE etc. This is the reason for taking the auto-generation approach, to get a comprehensive and complete list.

could be but wolfgang did a huge effort in making them pretty complete (even low level commands) so ic something is not in there (the i-*.xml files), it's with good reason

Sure, I was referring to scite-context-data-interfaces.lua, not the i-*.xml files..

what i mean is: all is in the i-* files but not all in those files is (yet) in the scite files (i do such things stepwise as it has a low priority for me; rendering in the pdf is more important) ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------