Heiko Oberdiek wrote:

Thus if we add \match, then the newsgroups are full of \match isn't working if I use chess and pdflatex ... And the people would remember, pdfTeX causes problems.

eh .. i can imagine that they are surprised that the primitive \match does not work, but any package that defines its own will since it overloads the primitive btw, i suppose that the switch to etex also caused problems for latex then? anyhow, i have no problem with a prefix, how about: \ext (extensions) -> \extmatch they are not pdftex specific (hence no p prefix) and they can as well end up in jonathan's xetex (i'll discuss it with him when the patches are ready) Hans Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Hi, i've asked several everyday-TeX-users (but quite new to pdfTeX) what they think about prefixes. \pdf* prefix for commands not related to PDF are *more* confusing than non-prefixed that may occasionally conflict with macros.

Hans: \ext (extensions) -> \extmatch

IMHO much better than \pdfmatch. However, in 2010 nobody will remember that regex support was an extension in 2005. All prefixes encounter this problem. Thus, I agree with Taco, that if some command can't be grep-ed in representative TeX archives, it can be used as a new primitive. Regards, -- Pawe/l Jackowski P.Jackowski@gust.org.pl

Hi,

Heiko: And what about packages, macros that are not released? Documents are allowed to contain macro definitions. Thus existing documents can break because of added new primitives[...]

Nope, since macros defined in the document will always override the primitive meaning. The document will NOT crash and won't need a primitive meaning of the clashed command, since was prepared before the command was implemented. Documents could break if some syntax would change but I belive it will never happen. I can't imagine a situation, where some old document crashes because of a new primitive redefined in the document. Regards, -- Pawe/l Jackowski P.Jackowski@gust.org.pl

Heiko Oberdiek wrote:

On Fri, Jul 01, 2005 at 09:11:13AM +0200, Pawel Jackowski wrote:

...

...

Heiko: And what about packages, macros that are not released? Documents are allowed to contain macro definitions. Thus existing documents can break because of added new primitives[...]

Nope, since macros defined in the document will always override the primitive meaning. The document will NOT crash and won't need a primitive meaning of the clashed command, since was prepared before the command was implemented.

Examples with a new primitive: \match

Szenario A: User defines macro: \newcommand{\match}{Hello World} ==> Error message, \match already defined.

Szenario B: \def\match!#1!#2!#3{...}

Some package (hyperref, ...) uses the new primitive: \match{...}{...} ==> Document fails with: Use of \match doesn't match its definition.

so, for latex the solution would be (in the latex specific initialization file): \let\pdfmatch\match \let\match\undefined That way you get (1) a prefix (one that will not clash with other prefixes) and (2) we keep tex prefix-clean Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Heiko Oberdiek wrote:

On Fri, Jul 01, 2005 at 04:13:59PM +0200, Hans Hagen wrote:

...

...

Some package (hyperref, ...) uses the new primitive: \match{...}{...} ==> Document fails with: Use of \match doesn't match its definition.

so, for latex the solution would be (in the latex specific initialization file):

\let\pdfmatch\match \let\match\undefined

That way you get (1) a prefix (one that will not clash with other prefixes) and (2) we keep tex prefix-clean

But then a package doesn't know, whether there is a special format that remaps the primitives. First it must guess the name of the primitive, before using it. And there are packages that work with different formats (eg. plain and latex). Thus this way does not provide a reliable solution.

it depends, if you have a (latex) convention that the prefix is \pdf then it can test for \ifx\pdfmatch\undefined ... etc. Since latex packages work for latex only it will not hurt anything outside that scope. Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On Fri, 1 Jul 2005, Martin Schröder wrote:

On 2005-07-01 16:13:59 +0200, Hans Hagen wrote:

...

so, for latex the solution would be (in the latex specific initialization file):

\let\pdfmatch\match \let\match\undefined

That way you get (1) a prefix (one that will not clash with other prefixes) and (2) we keep tex prefix-clean

Yes, that would be possible (e.g. in pdftexconfig.tex in tetex). But it needs a special format and it's non-portable. The reverse would also be possible: \let\match\ptexmatch

maybe one could find a way to enable new primitives, so that they are hidden if not enabled. Maybe removing the primitive name strings from the hash and keeping them somewhere safe until they are required? Like \use\match? A little like in Perl... Regards, Hartmut

On Fri, 1 Jul 2005, Taco Hoekwater wrote:

Hartmut Henkel wrote:

...

maybe one could find a way to enable new primitives, so that they are hidden if not enabled. Maybe removing the primitive name strings from the hash and keeping them somewhere safe until they are required? Like \use\match? A little like in Perl...

I'm pondering on Hans' \primitive\par notion, which is almost the same idea. After \let\par\undefined, \primitive\par should still execute a \par internal command, so Hans' and your idea seem two facets of the same jewel.

yes, difference might be that i meant it in the sense that \use\match wouldn't execute \match, it would only install it at its destined position within the hash (if that's possible at all), so that you from then on can call it by a simple \match. (which gets tricky if in the meantime a macro with the same name has been defined) Regards, Hartmut

Hi there, As promised, some proposals for the symbol table. No code yet, because careful thinking is needed first, to avoid unexplainable side-effects. Let me propose two rather straightforward new primitives first (please imagine the \ptex prefix yourself): \ifprimitive <csname> True if the <csname> has the same meaning as it would in initex, (but gives an error if <csname> is currently undefined). \executeprimitive <csname> Executes the primitive that would be known as <csname> in initex, even if the <csname> is currently undefined or defined to mean something else. Raises an error if there is no matching internal command (of course). These are quite straightforward, and I do not assume this needs much discussion. But the next ones are related to "if feature ...", and those are bit harder to define. I propose to make it possible to 'predefine' primitives in the executable. All program code is completely ready, except that the hash table entry is never created, making it impossible to use that primitive in initex without extra work. \ifinternal <csname> True if there is code in the pdtex executable to implement the internal command <csname>, regardless of whether or not it is currently reachable from macros. \executeinternal <csname> Executes the internal that is registered as <csname>, even if that internal is currently obscured (disable) or the <csname> is defined to mean something else. Raises an error if there is no matching internal command. \exposeinternal <csname> This does the final step of the primitive() function: it creates a hash entry for <csname> that maps it to its primitive meaning. An error is raised if the <csname> is currently defined. (perhaps this primitive should accept the \global prefix?) \obscureinternal <csname> The inverse of \exposeinternal, this 'undefines' the primitive related to <csname>. If <csname> is currently a macro, nothing needs to be done. Control sequences that are \let to the meaning of this csname are left as is. (perhaps this primitive should accept the \global prefix?) I chose \expose... and \obscure... instead of \disable... \enable because this is closer to the truth: in this proposal it is not possible to unconditionally disable a primitive (because that would require much more work than this). Side note: The pascal implementation can be rather straightforward: a pair of shadow_hash and shadow_eqtb has to be created that contains only the equivalents of the internal control sequences, nothing else. WDYT? Greetings, Taco

Martin Schröder wrote:

On 2005-07-04 12:30:47 +0200, Taco Hoekwater wrote:

...

As promised, some proposals for the symbol table. No code yet, because careful thinking is needed first, to avoid unexplainable side-effects.

Hrm. The problem I see is that Karl is breathing down my neck because he wants a code freeze for TL2005 this week. :-{

I was not really aiming for 1.30. Messing with TeX's internals is better not done with a deadline looming over your head, so let's go slow on this one. Taco

Taco Hoekwater wrote:

As promised, some proposals for the symbol table. No code yet, because careful thinking is needed first, to avoid unexplainable side-effects.

interesting -)

Let me propose two rather straightforward new primitives first (please imagine the \ptex prefix yourself):

\ifprimitive <csname>

True if the <csname> has the same meaning as it would in initex, (but gives an error if <csname> is currently undefined).

why not false instead of an error?

\executeprimitive <csname>

Executes the primitive that would be known as <csname> in initex, even if the <csname> is currently undefined or defined to mean something else. Raises an error if there is no matching internal command (of course).

These are quite straightforward, and I do not assume this needs much discussion. But the next ones are related to "if feature ...", and those are bit harder to define.

I propose to make it possible to 'predefine' primitives in the executable. All program code is completely ready, except that the hash table entry is never created, making it impossible to use that primitive in initex without extra work.

\ifinternal <csname>

True if there is code in the pdtex executable to implement the internal command <csname>, regardless of whether or not it is currently reachable from macros.

\executeinternal <csname>

Executes the internal that is registered as <csname>, even if that internal is currently obscured (disable) or the <csname> is defined to mean something else. Raises an error if there is no matching internal command.

\exposeinternal <csname>

This does the final step of the primitive() function: it creates a hash entry for <csname> that maps it to its primitive meaning. An error is raised if the <csname> is currently defined. (perhaps this primitive should accept the \global prefix?)

\obscureinternal <csname>

The inverse of \exposeinternal, this 'undefines' the primitive related to <csname>. If <csname> is currently a macro, nothing needs to be done. Control sequences that are \let to the meaning of this csname are left as is. (perhaps this primitive should accept the \global prefix?)

I chose \expose... and \obscure... instead of \disable... \enable because this is closer to the truth: in this proposal it is not possible to unconditionally disable a primitive (because that would require much more work than this).

sounds ok to me; concerning the global, i think this is indeed a needed feature (unless it messes up the code too much)

Side note: The pascal implementation can be rather straightforward: a pair of shadow_hash and shadow_eqtb has to be created that contains only the equivalents of the internal control sequences, nothing else.

in the same area, how about a primitive that marks such a primitive (or maybe any macro but that may be harder to do) as being an 'if' \def\ifMyTest#1#2% {..... \iftrue} \exposeif\ifMyTest \ifnum\a>\b \ifMyTest\c\d ... \fi \fi (no missing fi messages and such) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Taco Hoekwater wrote:

Hans Hagen wrote:

...

...

Let me propose two rather straightforward new primitives first (please imagine the \ptex prefix yourself):

\ifprimitive <csname>

True if the <csname> has the same meaning as it would in initex, (but gives an error if <csname> is currently undefined).

why not false instead of an error?

I proposed that because it would be slightly easier to implement, but you need to know what you are testing anyway, so it would only apply after e.g. \let\par\undefined. And does it really make sense to do that to a primitive?

ok, we'll see, one problem that i have with errors is that oen always has to mess around woth the moded in order to catch it

...

in the same area, how about a primitive that marks such a primitive (or maybe any macro but that may be harder to do) as being an 'if'

That is not the same problem. I will think about something like this, but it is much harder than it appears to be at first sight.

Understood (we've discussed such a feature before, so i already guessed that it was complex -) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Hi Heiko, Heiko Oberdiek wrote:

What is the result of \ifprimitive, it is \iftrue/\iffalse or an expansion text "true"/"some error message"?

as in \iftrue/\iffalse: it is a 'real' \if test .

...

\executeprimitive <csname>

What about \executeprimitive itself?

\def\executeprimitive{Hello World} \executeprimitive\executeprimitive\relax

Is the result two "Hello World" or "\relax"?

This result is Hello WorldHello World\relax Without the \def line, you would get only: \relax (the first \executeprimitive executes the second which executes the internal version of \relax)

...

\ifinternal <csname> \executeinternal <csname> \exposeinternal <csname> \obscureinternal <csname>

It needs some thinking. First thoughts: * It looks like a first step to a real namespace implementation, a (perhaps read-only) internal namespace, namespaces for extensions, and user/package macros. Why not going further then?

TeX's current internal structure does not allow a clean solution, so if we want true namespaces in TeX, we probably have to start thinking about a successor to pdftex. Otherwise, we would need to redesign the whole symbol table / token handling while trying to remain 100% compatible. That sounds like a nightmare to me. These last 4 primitives I proposed are an attempt to come up with something that works next month (as opposed to next year), but you are right that it probably needs more thinking, anyway. Taco

Heiko Oberdiek wrote:

It needs some thinking. First thoughts: * It looks like a first step to a real namespace implementation, a (perhaps read-only) internal namespace, namespaces for extensions, and user/package macros. Why not going further then? * Perhaps PostScript ideas of dictionary stacks can help here: systemdict for internal iniTeX primitives, userdict, ...

this is probably more complex (and will take longer to get stable) due to the way tex is coded we can consider adding additional hashes/stacks after this is done Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

Reinhard: I think that it is quite unlikely that someone wrote a macro yet which name begins with `PTEX'[...]

The last chance the user has to avoid *any* clashes with *any* primitives in own home made macros is to use \CAPITALS. IMHO we should never steal this chance... -- Pawe/l Jackowski P.Jackowski@gust.org.pl

Heiko Oberdiek wrote:

existing documents can break because of added new primitives.

a small chance; such a package does not know about the primitive; the only danger can be in: - core latex uses a new primitive - old package overloads overloads new primitive by macro - core latex functionality fails but, for new primitives

With prefixes and reserved namespaces, document and package writers have the chance to avoid name clashes with future versions of pdfTeX. Without is asking for unnecessary trouble.

partially true, what if users have \pdf or whatever as prefix for their own macros? as said ... etex introduced primitives that clashed with context (and probably with other stuff around as well; \protected, \expanded, \interactionmode, those \if's etc are all candidates; and yet, we now default to etex as engine) so ... there is always the danger of a clash; the best we can do is to provide a list of new primitives in a separate file so that users can use that list to parse all their local stuff for clashes. Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------

On 2005-07-01 15:08:16 +0200, Heiko Oberdiek wrote:

On Fri, Jul 01, 2005 at 10:44:13AM +0200, Hans Hagen wrote:

...

Heiko Oberdiek wrote:

...

existing documents can break because of added new primitives.

a small chance; such a package does not know about the primitive; the only danger can be in:

- core latex uses a new primitive - old package overloads overloads new primitive by macro - core latex functionality fails

For example, package foo defines: \newcommand{\match}... and fails with error message. Then we have frequent news groups postings: "After pdfTeX update package foo is not working, ..." or "Error message after pdfTeX update, ..." In the eyes of the readers, pdfTeX is guilty, not the best advertisment for pdfTeX. And in the worst(normal) case author of package foo doesn't have time for an update or cannot be found.

...

but, for new primitives

...

With prefixes and reserved namespaces, document and package writers have the chance to avoid name clashes with future versions of pdfTeX. Without is asking for unnecessary trouble.

partially true, what if users have \pdf or whatever as prefix for their own macros?

But users have the chance to know, that there are "reserved" prefixes and can avoid future name clashes, when new primitives are introduced.

...

as said ... etex introduced primitives that clashed with context (and probably with other stuff around as well; \protected, \expanded, \interactionmode, those \if's etc are all candidates; and yet, we now default to etex as engine)

Perhaps we can ask the etex people, their experience and what they would do.

...

so ... there is always the danger of a clash; the best we can do is to provide a list of new primitives in a separate file so that users can use that list to parse all their local stuff for clashes.

And they would have to do it again and again for each new version of pdfTeX that introduces new primitives.

This issue has come up again in internal discussions because of the imminent release and it seems we have no consensus yet (neither about the use of namespaces nor about the naming of the namespace for non-pdf-extensions). I suggest a compromise: For 1.30.0 all new primitives stay in the \pdf namespace (without renaming recently introduced primitives like \quitvomde), and we will implement real namespaces in 1.40.0 (Tacos proposal). Best Martin -- http://www.tm.oneiros.de

On 2005-07-20 19:43:24 +0200, Pawel Jackowski wrote:

Nothing better comes to my mind. To clarify, you mean in pdfTeX 1.40.0 (containing Taco's jewel), \pdf* prefix will be simply removed from where its not suitable and future primitives will be prefixed with \pdf or none, right?

Yes. Best Martin PS: Unless anybody complains loudly, I'll do rc4 with this tomorrow and a release early next week. -- http://www.tm.oneiros.de