- dev-context - NTG Mailing Lists

Re: [dev-context] setup-en.pdf and mkiv command list
by Hans Hagen 17 May '17

17 May '17

On 5/17/2017 8:27 PM, Yu,Jason wrote: > I suppose not all abstract commands has <cd:instances> tag. For not yet ... it's work in progress, so you can assume tyhat over time the xml file gets more complete ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

1 0

Re: [dev-context] setup-en.pdf and mkiv command list
by Wolfgang Schuster 17 May '17

17 May '17

> Idris Samawi Hamid ادريس سماوي حامد <mailto:Idris.Hamid@colostate.edu> > 17. Mai 2017 um 16:04 > On Wed, 17 May 2017 07:25:41 -0600, Hans Hagen <pragma(a)wxs.nl> wrote: > >>>> cd:instances >>> >>> While we wait for support cd:instances: How about this heuristic to >>> distinguish abstract commands from pre-supported instances: >>> >>> ======= >>> If slanted and containing and containing *no* capital letters, treat as >>> an instance. >>> ======= >> >> (slanted is not part of the coding, instances is) > > Sure, but we have to distinguish abstract commands from pre-supported > instances of abstract commands, both of which are typeset as slanted. > > So we need a simple name for the genus that encompasses both abstract > commands and pre-supported instances of abstract commands. > > Perhaps 'generic'. All commands typeset as slanted in setup-en etc. > are generic commands. So to rephrase > > ======= > If a command is generic and contains *no* capital letters, treat as a > concrete instance. > ======= > > So whatever hook marks off a command to be treated as slanted is what > marks it as a generic command. Below this is the entry for \section in my own version of the setups files. All commands with instances have the attribute variant="instance" and the predefined instances are listed under cd:instances. Commands which are slanted in the manual are produced by a \defineXXX command which is documented with the generated="yes" attribute. For your list you can ignore the values under cd:sequence and use only the listed instances. <cd:command name="section" generated="yes" variant="instance" file="strc-sec.mkiv"> <cd:sequence> <cd:instance value="section"/> </cd:sequence> <cd:arguments> <cd:resolve name="keyword-reference-list-optional"/> <cd:resolve name="argument-text"/> </cd:arguments> <cd:instances> <cd:constant value="part"/> <cd:constant value="chapter"/> <cd:constant value="section"/> <cd:constant value="subsection"/> <cd:constant value="subsubsection"/> <cd:constant value="subsubsubsection"/> <cd:constant value="subsubsubsubsection"/> <cd:constant value="title"/> <cd:constant value="subject"/> <cd:constant value="subsubject"/> <cd:constant value="subsubsubject"/> <cd:constant value="subsubsubsubject"/> <cd:constant value="subsubsubsubsubject"/> </cd:instances> </cd:command> Wolfgang

1 0

Re: [dev-context] setup-en.pdf and mkiv command list
by Hans Hagen 17 May '17

17 May '17

On 5/17/2017 2:23 PM, Idris Samawi Hamid ادريس سماوي حامد wrote: > On Wed, 17 May 2017 01:43:38 -0600, Hans Hagen <pragma(a)wxs.nl> wrote: > >>>>> I wrote a python script to auto-generate keyword lists from the >>>>> full xml >>>>> file. All the "abstract" keywords (those with a <cd:sequence> >>>>> </cd:sequence> tag) are simply ignored. The full list is in the >>>>> KEYWORDS.txt file. >>>> >>>> idris probably also wants the instances, so \startsubject and so; in >>>> due time we'll add cd:instances to the relevant commands (kind of >>>> experimental now) >>> >>> Hmm, so we have an ambiguity. See setup-en.pdf, p. 302: >>> >>> \startSECTION is typeset in slanted, but is an abstract command that can >>> have an indefinite number of instances >>> >>> \startchapter is typeset in slanted, but is a concrete command -- in >>> this case a pre-supported instance of an abstract command. >>> >>> Question 1: Why is \startchapter typeset in slanted? >> >> to show that it's an instance >> >>> Question 2: Is there a hook in the i-*.xml files so that Jason's script >>> can distinguish between abstract commands and pre-supported instances? >>> We want a way to capture the latter while ignoring the former? Or are we >>> stuck with the ambiguity until cd:instances is supported? In the latter >>> case we'll have to do some manual work... >> >> cd:instances > > While we wait for support cd:instances: How about this heuristic to > distinguish abstract commands from pre-supported instances: > > ======= > If slanted and containing and containing *no* capital letters, treat as > an instance. > ======= (slanted is not part of the coding, instances is) > Wolfgang, Hans: Will this heuristic leave any residue, i.e., leave out > any pre-supported instances? well, you can easilly check that .. probably it's ok but not a complete list ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

1 0

Re: [dev-context] setup-en.pdf and mkiv command list
by Hans Hagen 17 May '17

17 May '17

On 5/17/2017 12:35 AM, Idris Samawi Hamid ادريس سماوي حامد wrote: > On Tue, 16 May 2017 16:17:46 -0600, Hans Hagen <pragma(a)wxs.nl> wrote: > >> On 5/16/2017 9:57 PM, Yu,Jason wrote: >>> Hi all, >>> >>> >>> I wrote a python script to auto-generate keyword lists from the full xml >>> file. All the "abstract" keywords (those with a <cd:sequence> >>> </cd:sequence> tag) are simply ignored. The full list is in the >>> KEYWORDS.txt file. >> >> idris probably also wants the instances, so \startsubject and so; in >> due time we'll add cd:instances to the relevant commands (kind of >> experimental now) > > Hmm, so we have an ambiguity. See setup-en.pdf, p. 302: > > \startSECTION is typeset in slanted, but is an abstract command that can > have an indefinite number of instances > > \startchapter is typeset in slanted, but is a concrete command -- in > this case a pre-supported instance of an abstract command. > > Question 1: Why is \startchapter typeset in slanted? to show that it's an instance > Question 2: Is there a hook in the i-*.xml files so that Jason's script > can distinguish between abstract commands and pre-supported instances? > We want a way to capture the latter while ignoring the former? Or are we > stuck with the ambiguity until cd:instances is supported? In the latter > case we'll have to do some manual work... cd:instances >> anyway, not that you can make lists you can always provide idris with >> updated onces >> >> categories can come later > > Thanks Hans > > Idris -- ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

1 0

Re: [dev-context] setup-en.pdf and mkiv command list
by Yu,Jason 16 May '17

16 May '17

Hi all, I wrote a python script to auto-generate keyword lists from the full xml file. All the "abstract" keywords (those with a <cd:sequence> </cd:sequence> tag) are simply ignored. The full list is in the KEYWORDS.txt file. Thanks, Jason ________________________________ From: Hamid,Idris Sent: Tuesday, May 16, 2017 8:09:28 AM To: Wolfgang Schuster Cc: Hans Hagen; dev-context(a)ntg.nl; Yu,Jason Subject: Re: setup-en.pdf and mkiv command list On Tue, 16 May 2017 07:38:33 -0600, Wolfgang Schuster <schuster.wolfgang(a)gmail.com> wrote: >>>> TeX primitives - Knuth >>>> luaTeX primitives - excluding Knuth >>> The syntax files contain at the moment only ConTeXt specific command, >>> primitives aren’t included. >> >> Sure, we get the TeX and luaTeX commands from the scite*.lua files >> >>>> macro structure - e.g., sectioning etc. >>>> micro structure - e.g., itemizations, tables etc. >>>> mode structure - e.g., metapost, xml, markdown >>> The XML source has a category attribute for this information. >> >> Could you give a couple of examples? > Below is a example for \m{…}. > > <cd:command name="m" category="mathematics" file="math-ini.mkiv"> > <cd:arguments> > <cd:resolve name="keyword-name-optional"/> > <cd:content/> > </cd:arguments> > </cd:command> Wonderful. So the task is two-fold: i) categorization of context commands via the category attribute ii) adding tex and luatex primitive classes For editors, the abstract commands (\ATTACHMENT, \CHARACTERKERNING etc.) do not need to be generated. Idris -- Idris Samawi Hamid, Professor Department of Philosophy Colorado State University Fort Collins, CO 80512

2 1

Re: [dev-context] setup-en.pdf and mkiv command list
by Wolfgang Schuster 16 May '17

16 May '17

> Idris Samawi Hamid ادريس سماوي حامد <mailto:Idris.Hamid@colostate.edu> > 16. Mai 2017 um 15:17 > On Sun, 14 May 2017 14:04:41 -0600, Wolfgang Schuster > <schuster.wolfgang(a)gmail.com> wrote: > >>> Idris Samawi Hamid ادريس سماوي حامد <mailto:Idris.Hamid@colostate.edu> >>> 14. Mai 2017 um 20:55 >>> On Sun, 14 May 2017 12:20:43 -0600, Wolfgang Schuster >>> <schuster.wolfgang(a)gmail.com> wrote: >>> >>>>> OTOH user-defined commands can be added to the ConTeXt lexer via the >>>>> Style Configurator (Notepad++) and get their own highlight color. I >>>>> have found this very useful in writing long documents. See attached >>>>> (User-defined Keywords dialog). >>>> >>>> You can limit the number of custom commands when you use >>>> >>>> \startnamedsection[topic][title=...] >>>> >>>> instead of >>>> >>>> \starttopic[title=...] >>> >>> First time encountering these two commands; they're not on the wiki, >>> need to learn more about them.. >>> >>> Can you explain or give a complete sample test file illustrating how >>> they apply in the matter under discussion? Thanks. >> This is only a example to demonstrate that you can create new commands >> without using new command names (and the need to add them to your lexer) >> in your document. >> >> In the example below I create a new section but I don’t have to use >> a new user generated command in the document itself. >> >> \definehead[topic][subject] >> >> \starttext >> >> \startnamedsection[topic][title=This is a topic] >> ... >> \stopnamedsection >> >> \stoptext > > I see your point. Will make a note of it for the lexer documentation. > >>>> Adding tags to the commands is planned but the question is how to >>>> categorize them (internal, api, user level, low level, primitives, >>>> defininitions (\define...), setups etc.). >>> >>> This is good to know. At the moment we're working on a full-featured >>> ConTeXt lexer and are experimenting with different ideas of organizing >>> commands for user-friendly syntax highlighting. Currently considering >>> something like the following: >>> >>> TeX primitives - Knuth >>> luaTeX primitives - excluding Knuth >> The syntax files contain at the moment only ConTeXt specific command, >> primitives aren’t included. > > Sure, we get the TeX and luaTeX commands from the scite*.lua files > >>> macro structure - e.g., sectioning etc. >>> micro structure - e.g., itemizations, tables etc. >>> mode structure - e.g., metapost, xml, markdown >> The XML source has a category attribute for this information. > > Could you give a couple of examples? Below is a example for \m{…}. <cd:command name="m" category="mathematics" file="math-ini.mkiv"> <cd:arguments> <cd:resolve name="keyword-name-optional"/> <cd:content/> </cd:arguments> </cd:command> Wolfgang

1 0

Re: [dev-context] setup-en.pdf and mkiv command list
by Wolfgang Schuster 16 May '17

16 May '17

> Idris Samawi Hamid ادريس سماوي حامد <mailto:Idris.Hamid@colostate.edu> > 16. Mai 2017 um 14:27 > Hi Wolfgang, > > On Mon, 15 May 2017 13:00:37 -0600, Wolfgang Schuster > <schuster.wolfgang(a)gmail.com> wrote: > >> >>> Hans Hagen <mailto:pragma@wxs.nl> > >>> low level is fuzzy anyway: \xmlfirst (more low level?) vs \xmlfilter >>> (more generic) >> I can add class="system|primitive" (or level="...") to the XML files, >> commands without attribute are user-level. > > It would be great to include this system-primitive class in the lexer, > but if that's too much of a burden short-term then perhaps we should > not wait... Do you have an approximate time-frame for when this can be > done? I presume it will take a long time to put that class together... Marking internal/system command is simple but it will take a few days (boring stuff you won’t do for a few hours). Wolfgang

1 0

Re: [dev-context] setup-en.pdf and mkiv command list
by Hans Hagen 16 May '17

16 May '17

On 5/13/2017 9:41 PM, Idris Samawi Hamid ادريس سماوي حامد wrote: > Dear gang, > > We need to generate a complete list of mkiv commands. setup-en.pdf has > everything but some clarification and modification is needed: > > 1. Clarification: Does setup-en.pdf includes both mkii and mkiv > commands? If so, then can a version of the auto-generation script be > made that will generate a version setup-mkiv-en.pdf that is complete but > restricted to mkiv commands? this is mkiv (there is an cont-en.xml file describing mkii mostly) ... remmeber: mkii is frozen and phased out > [Is the auto-generation script publicly available?] all scripts are in the distribution > 2. Modification: It appears that all command names that appear in > slanted text are used to mention user-defined commands of a particular > class, e.g., > > /\BACKGROUND/ - slanted text > > which obviously names each of a class of commands defined by > > \definebackground - regular text > > Can a version of the auto-generation script be made that will generate a > version setup-mkiv-strict-en.pdf that is restricted to mkiv commands > explicitly defined in mkiv, excluding class commands or anything else > represented in slanted text? it's xml so one can benerate anything from the file (i was in the process of generating instances in the lua files for scintilla but (1) had to prepare for bachotex, (2) got back and ill, (3) need some motivation to pick up that thread) > 3. Although it is intuitive, a legend precisely explaining the semantics > of the command list in setup.pdf would be useful. If there is already > one available, could you point it out to me? dunno, maybe somewhere on the wiki > 4. Would it be possible for the auto-generation to export the command > list to a semi-formatted text file? Copying the text of setup-en.pdf to > a text file gives usable results but a text file with some regular > structure might be better. one can (1) use some xml transformation or (2) use the lua tables to get strings > In sum: we need - urgently - a comprehensive ConTeXt command list > restricted to mkiv that excludes class names for user definitions and > the like. Any help is greatly appreciated, and thanks in advance! > > Best wishes > Idris > -- ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

3 19

Re: [dev-context] setup-en.pdf and mkiv command list
by Hans Hagen 16 May '17

16 May '17

On 5/16/2017 2:35 PM, Idris Samawi Hamid ادريس سماوي حامد wrote: > On Tue, 16 May 2017 06:27:49 -0600, Idris Samawi Hamid ادريس سماوي حامد > <Idris.Hamid(a)colostate.edu> wrote: > >> Hi Wolfgang, >> >> On Mon, 15 May 2017 13:00:37 -0600, Wolfgang Schuster >> <schuster.wolfgang(a)gmail.com> wrote: >> >>> >>>> Hans Hagen <mailto:pragma@wxs.nl> >> >>>> low level is fuzzy anyway: \xmlfirst (more low level?) vs \xmlfilter >>>> (more generic) >>> I can add class="system|primitive" (or level="...") to the XML files, >>> commands without attribute are user-level. >> >> It would be great to include this system-primitive class in the lexer, >> but if that's too much of a burden short-term then perhaps we should >> not wait... Do you have an approximate time-frame for when this can be >> done? I presume it will take a long time to put that class together... > > Apologies, got confused, assumed we were talking about low-level context > commands, not TeX/luaTeX primitives. sure, but a categorization has to cover all aspects of tex Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

1 0

Re: [dev-context] setup-en.pdf and mkiv command list
by Hans Hagen 16 May '17

16 May '17

On 5/16/2017 2:27 PM, Idris Samawi Hamid ادريس سماوي حامد wrote: > Hi Wolfgang, > > On Mon, 15 May 2017 13:00:37 -0600, Wolfgang Schuster > <schuster.wolfgang(a)gmail.com> wrote: > >> >>> Hans Hagen <mailto:pragma@wxs.nl> > >>> low level is fuzzy anyway: \xmlfirst (more low level?) vs \xmlfilter >>> (more generic) >> I can add class="system|primitive" (or level="...") to the XML files, >> commands without attribute are user-level. > > It would be great to include this system-primitive class in the lexer, > but if that's too much of a burden short-term then perhaps we should not > wait... Do you have an approximate time-frame for when this can be done? > I presume it will take a long time to put that class together... everything setups is long term and normally ctx group meetings are moments when something is/gets done (if you'd seen Wolfgangs awesome ledgers at the last meeting you'd probably understand better what i mean) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

1 0