[NTG-pdftex] \expanded

Joseph Wright joseph.wright at morningstar2.co.uk
Sun May 13 22:30:03 CEST 2018


Hello all,

As recently raised on the TeX Live mailing list, the LaTeX team feel 
that the \expanded primitive, suggested for pdfTeX 1.50 around 10 years 
ago, would be a useful addition to TeX engines. (It is present in LuaTeX.)

Attached is a patch which provides the primitive itself (based on the 
LuaTeX version), an entry for the pdfTeX manual, and some tests. (This 
is based on the version in 
https://github.com/josephwright/texlive-source/tree/expanded, slightly 
adjusted as the tests there also cover other primitives to support XeTeX 
additions.) I have chosen the pdfTeX manual as the 'home' for 
documenting behaviour as the idea comes from pdfTeX and as this is 
usually where 'utility' primitives have been documented.

A similar request has been made for XeTeX (for the code part), whilst 
the (u)pTeX developers already have a Git branch which would add this 
functionality. We (the team) do hope that general agreement can be 
reached to make this available in the main engines (pdfTeX, XeTeX, 
LuaTeX, e-pTeX, e-upTeX) for TL'19.

Regards,

Joseph
-------------- next part --------------
Index: doc/manual/pdftex-t.tex
===================================================================
--- doc/manual/pdftex-t.tex	(revision 796)
+++ doc/manual/pdftex-t.tex	(working copy)
@@ -4278,6 +4278,16 @@
 
 \subsection{Miscellaneous}
 
+\pdftexprimitive{\Syntax{\tex {expanded} \Something{tokens}
+  \Whatever{expandable}}}
+\bookmark{\tex{expanded}}
+
+Expands to the result of that expansion of the \Something{tokens}
+in exactly the same way as \type{\message} and leaves the result of
+expansion in the input stream.  (Note that contrarily to \type{\edef}, macro
+parameter characters do not need to be doubled.  Note that \type{\protected}
+macros are not expanded.) \introduced{1.40.20}
+
 \pdftexprimitive{\Syntax{\tex {ifincsname} \Whatever{expandable}}}
 \bookmark{\tex{ifincsname}}
 
Index: source/src/texk/web2c/pdftexdir/ChangeLog
===================================================================
--- source/src/texk/web2c/pdftexdir/ChangeLog	(revision 796)
+++ source/src/texk/web2c/pdftexdir/ChangeLog	(working copy)
@@ -1,3 +1,10 @@
+2018-05-13 Joseph Wright <joseph.wright at morningstar2.co.uk>
+
+	* pdftex.web: New primitive \expanded based on original
+	  suggestion by Heiko Oberdiek, and with work by
+	  Bruno Le Floch and David Carlisle (on behalf of The LaTeX
+	  Project)
+
 2018-04-14  Karl Berry  <karl at tug.org>
 
 	* TeX Live 2018 release, pdftex 1.40.19.
Index: source/src/texk/web2c/pdftexdir/am/pdftex.am
===================================================================
--- source/src/texk/web2c/pdftexdir/am/pdftex.am	(revision 796)
+++ source/src/texk/web2c/pdftexdir/am/pdftex.am	(working copy)
@@ -94,8 +94,9 @@
 
 # pdfTeX Tests
 #
-pdftex_tests = pdftexdir/wprob.test pdftexdir/pdftex.test pdftexdir/pdfimage.test
-pdftexdir/wprob.log pdftexdir/pdftex.log pdftexdir/pdfimage.log: pdftex$(EXEEXT)
+pdftex_tests = pdftexdir/wprob.test pdftexdir/pdftex.test pdftexdir/pdfimage.test \
+  pdftexdir/expanded.test
+pdftexdir/wprob.log pdftexdir/pdftex.log pdftexdir/pdfimage.log pdftexdir/expanded.log: pdftex$(EXEEXT)
 
 EXTRA_DIST += $(pdftex_tests)
 
Index: source/src/texk/web2c/pdftexdir/expanded.test
===================================================================
--- source/src/texk/web2c/pdftexdir/expanded.test	(nonexistent)
+++ source/src/texk/web2c/pdftexdir/expanded.test	(working copy)
@@ -0,0 +1,20 @@
+#! /bin/sh -vx
+# $Id$
+# Public domain
+# You may freely use, modify and/or distribute this file.
+# David Carlisle: version for \expanded based on existing test scripts by Peter Breitenlohner and Karl Berry
+
+TEXMFCNF="$srcdir/../kpathsea"
+TEXINPUTS="$srcdir/pdftexdir/tests:."
+
+
+export TEXMFCNF TEXINPUTS 
+
+
+./pdftex -ini -etex --interaction batchmode expanded.tex 
+sed -n -e 's/[\\]pdf/\\/g' -e '/START/,/END/p' expanded.log > expanded_pdftex.log || exit 1
+
+diff "$srcdir/pdftexdir/tests/expanded.txt"  expanded_pdftex.log || exit 1
+
+exit 0
+
Index: source/src/texk/web2c/pdftexdir/pdftex.web
===================================================================
--- source/src/texk/web2c/pdftexdir/pdftex.web	(revision 796)
+++ source/src/texk/web2c/pdftexdir/pdftex.web	(working copy)
@@ -10748,7 +10748,8 @@
 @d etex_convert_base=5 {base for \eTeX's command codes}
 @d eTeX_revision_code=etex_convert_base {command code for \.{\\eTeXrevision}}
 @d etex_convert_codes=etex_convert_base+1 {end of \eTeX's command codes}
- at d pdftex_first_expand_code = etex_convert_codes {base for \pdfTeX's command codes}
+ at d expanded_code = etex_convert_codes {command code for \.{\\expanded}}
+ at d pdftex_first_expand_code = expanded_code + 1 {base for \pdfTeX's command codes}
 @d pdftex_revision_code     = pdftex_first_expand_code + 0 {command code for \.{\\pdftexrevision}}
 @d pdftex_banner_code       = pdftex_first_expand_code + 1 {command code for \.{\\pdftexbanner}}
 @d pdf_font_name_code       = pdftex_first_expand_code + 2 {command code for \.{\\pdffontname}}
@@ -10790,6 +10791,9 @@
 primitive("fontname",convert,font_name_code);@/
 @!@:font_name_}{\.{\\fontname} primitive@>
 @#
+primitive("expanded",convert,expanded_code);@/
+@!@:expanded_}{\.{\\expanded} primitive@>
+@#
 primitive("pdftexrevision",convert,pdftex_revision_code);@/
 @!@:pdftex_revision_}{\.{\\pdftexrevision} primitive@>
 primitive("pdftexbanner",convert,pdftex_banner_code);@/
@@ -10854,6 +10858,7 @@
   meaning_code: print_esc("meaning");
   font_name_code: print_esc("fontname");
   eTeX_revision_code: print_esc("eTeXrevision");
+  expanded_code: print_esc("expanded");
   pdftex_revision_code: print_esc("pdftexrevision");
   pdftex_banner_code:   print_esc("pdftexbanner");
   pdf_font_name_code:   print_esc("pdffontname");
@@ -10926,6 +10931,20 @@
   end;
 font_name_code: scan_font_ident;
 eTeX_revision_code: do_nothing;
+expanded_code:
+  begin
+    save_scanner_status := scanner_status;
+    save_warning_index := warning_index;
+    save_def_ref := def_ref;
+    save_cur_string;
+    scan_pdf_ext_toks;
+    warning_index := save_warning_index;
+    scanner_status := save_scanner_status;
+    ins_list(link(def_ref));
+    def_ref := save_def_ref;
+    restore_cur_string;
+    return;
+  end;
 pdftex_revision_code: do_nothing;
 pdftex_banner_code: do_nothing;
 pdf_font_name_code, pdf_font_objnum_code, pdf_font_size_code: begin
Index: source/src/texk/web2c/pdftexdir/tests/expanded.tex
===================================================================
--- source/src/texk/web2c/pdftexdir/tests/expanded.tex	(nonexistent)
+++ source/src/texk/web2c/pdftexdir/tests/expanded.tex	(working copy)
@@ -0,0 +1,46 @@
+
+\catcode`\{=1
+\catcode`\}=2
+\catcode`\#=6
+\def\typ#1{\immediate\write-1 {#1}}
+\def\space{ }
+\let\bgroup{
+\let\egroup}
+
+\typ{START}
+\typ{EXPANDED TEST 1}
+% Check the primitive exists
+\show\expanded
+
+\typ{EXPANDED TEST 2}
+% Simple expansion test
+\def\aaa{x}
+\def\bbb{\aaa\aaa}
+\expandafter\def\expandafter\ccc\expandafter{\expanded{\bbb,\aaa}}
+\show\ccc
+
+\typ{EXPANDED TEST 3: the}
+% Expanding \the
+\typ{\expanded{\bbb,\the\numexpr100+20+3\relax}}
+
+\typ{EXPANDED TEST 4: macro param}
+% Constructed #1 is still #1
+\expandafter\def\expandafter\ddd\expandafter#\expanded{1{#\number--1}}
+\show\ddd
+
+\typ{EXPANDED TEST 5: torture}
+% Torture test from Bruno Le Floch testing various tricky interactions
+\expanded\relax\space\ifincsname \BOOM\fi{\ifincsname \BOOM\fi}
+\showtokens\expandafter{\expanded{#,\noexpand\aaa,\unexpanded{\aaa}}}
+\showtokens\expandafter{\expanded{#,\unexpanded{#}}}
+\showtokens\expandafter{\expanded\expandafter{\noexpand\aaa}}
+\showtokens\expandafter{\expanded\expandafter{\unexpanded{\aaa}}}
+\showtokens\expanded{{\expanded{\aaa\noexpand\aaa\noexpand\noexpand\noexpand\aaa}}}
+\toks0{\aaa}
+\showtokens\expanded{{\the\toks0}}
+\expanded\bgroup\show\egroup}
+\edef\foo{\expanded{##}}
+\show\foo
+
+\typ{END}
+\end
Index: source/src/texk/web2c/pdftexdir/tests/expanded.txt
===================================================================
--- source/src/texk/web2c/pdftexdir/tests/expanded.txt	(nonexistent)
+++ source/src/texk/web2c/pdftexdir/tests/expanded.txt	(working copy)
@@ -0,0 +1,61 @@
+START
+EXPANDED TEST 1
+> \expanded=\expanded.
+l.13 \show\expanded
+                   
+
+EXPANDED TEST 2
+> \ccc=macro:
+->xx,x.
+l.20 \show\ccc
+              
+
+EXPANDED TEST 3: the
+xx,123
+EXPANDED TEST 4: macro param
+> \ddd=macro:
+#1->#1.
+l.29 \show\ddd
+              
+
+EXPANDED TEST 5: torture
+> ##,\aaa ,\aaa .
+l.34 ...panded{#,\noexpand\aaa,\unexpanded{\aaa}}}
+                                                  
+
+> ##,##.
+l.35 ...s\expandafter{\expanded{#,\unexpanded{#}}}
+                                                  
+
+> \aaa .
+l.36 ...fter{\expanded\expandafter{\noexpand\aaa}}
+                                                  
+
+> x.
+l.37 ...{\expanded\expandafter{\unexpanded{\aaa}}}
+                                                  
+
+> xx\aaa .
+<inserted text> {xx\aaa }
+                         
+l.38 ...pand\aaa\noexpand\noexpand\noexpand\aaa}}}
+                                                  
+
+> \aaa .
+<inserted text> {\aaa }
+                       
+l.40 \showtokens\expanded{{\the\toks0}}
+                                       
+
+> \egroup=end-group character }.
+<inserted text> \show \egroup 
+                              
+l.41 \expanded\bgroup\show\egroup}
+                                  
+
+> \foo=macro:
+->##.
+l.43 \show\foo
+              
+
+END


More information about the ntg-pdftex mailing list