Re: [NTG-context] Yet another cornercase of ^^ handling in LuaMetaTeX
On 6/17/2020 12:10 PM, Marcel Fabian Krüger wrote:
On Mon, Jun 15, 2020 at 04:16:06PM +0200, Hans Hagen wrote:
ah, i removed some doee that i though was never seen; i'll add that bit again
Beside that, I noticed a smaller weird thing when using LuaMetaTeX in ini mode without Context: The default mathcodes for digits and letters have their class and family values switched. So e.g. "A" has a default mathcode of 119537729=0x7200041, indicating class 1 and family 7, instead of the expected 31457345=0x1E00041, indicating class 7 and family 1. Of course this isn't really a problem because the format will normally assign new mathcodes anyway, but especially class 1 seems like a *very* unnatural default for normal letters. i'll check it (digits get class 0) ... indeed these defaults nowadays never kick in this way
Thanks. Is there any way to convert either a token number (as given e.g. by t.tok) or a numeric cs identifier (as given e.g. by t.tok-cs_token_flag) to a token uservalue? In LuaTeX I used to use `token.new(tok, 0)` for this, but LuaMetaTeX seems to do more checking here.
Indeed there is more checking. So, when you create a token you need to use a valid cmd code and a valid chr code. \ctxlua{ local t = token.new(string.byte("X"),token.command_id("letter")) tex.print(t) } The interface might evolve a bit more. Valid commands (at this moment) are: \ctxlua{ for k, v in ipairs(token.values("command")) do print(k,v) end } So, things like this should work \dimen3456=1234pt \dimen3467= \ctxlua{ local t = token.new(3456,token.command_id("register_dimen")) % local t = token.new("register_dimen",3456) % in upcoming version tex.sprint(t) } \the\dimen3467 As does: \ctxlua{ local t1 = token.new(string.byte("{"),token.command_id("letter")) local t2 = token.new(string.byte("}"),token.command_id("letter")) tex.print(t1) tex.print(t2) } Properties like cs_token_flag are not user level and have no meaning in the interface. In fact, the 'token-as-number' is just some representation with no intended meaning. Also, tokens that are problematic or can interfere badly might become inaccessible in future versions. (I don't expect many users to use the low level token interface anyway but it's there as part of the package.) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On Wed, Jun 17, 2020 at 03:43:49PM +0200, Hans Hagen wrote:
On 6/17/2020 12:10 PM, Marcel Fabian Krüger wrote:
On Mon, Jun 15, 2020 at 04:16:06PM +0200, Hans Hagen wrote:
ah, i removed some doee that i though was never seen; i'll add that bit again
Beside that, I noticed a smaller weird thing when using LuaMetaTeX in ini mode without Context: The default mathcodes for digits and letters have their class and family values switched. So e.g. "A" has a default mathcode of 119537729=0x7200041, indicating class 1 and family 7, instead of the expected 31457345=0x1E00041, indicating class 7 and family 1. Of course this isn't really a problem because the format will normally assign new mathcodes anyway, but especially class 1 seems like a *very* unnatural default for normal letters. i'll check it (digits get class 0) ... indeed these defaults nowadays never kick in this way
Thanks. Is there any way to convert either a token number (as given e.g. by t.tok) or a numeric cs identifier (as given e.g. by t.tok-cs_token_flag) to a token uservalue? In LuaTeX I used to use `token.new(tok, 0)` for this, but LuaMetaTeX seems to do more checking here.
Indeed there is more checking. So, when you create a token you need to use a valid cmd code and a valid chr code.
\ctxlua{ local t = token.new(string.byte("X"),token.command_id("letter")) tex.print(t) }
The interface might evolve a bit more. Valid commands (at this moment) are:
\ctxlua{ for k, v in ipairs(token.values("command")) do print(k,v) end }
So, things like this should work
\dimen3456=1234pt
\dimen3467= \ctxlua{ local t = token.new(3456,token.command_id("register_dimen")) % local t = token.new("register_dimen",3456) % in upcoming version tex.sprint(t) }
\the\dimen3467
As does:
\ctxlua{ local t1 = token.new(string.byte("{"),token.command_id("letter")) local t2 = token.new(string.byte("}"),token.command_id("letter")) tex.print(t1) tex.print(t2) }
Properties like cs_token_flag are not user level and have no meaning in the interface. In fact, the 'token-as-number' is just some representation with no intended meaning. Also, tokens that are problematic or can interfere badly might become inaccessible in future versions.
(I don't expect many users to use the low level token interface anyway but it's there as part of the package.)
Hans
How does this work with mark nodes? Their mark field has a table containing triples of command, character and "cs" value. How can the actual csname of a controlsequence be extracted from this in LuaMetaTeX? Marcel
How does this work with mark nodes? Their mark field has a table containing triples of command, character and "cs" value. How can the actual csname of a controlsequence be extracted from this in LuaMetaTeX? at the token ends marks are stored as token lists but not directly accessible by a token itself as they get resolved during a split; a topmarks token is a cmd/chr combination that indicates that token (\topmarks) which then picks up a number (i noticed that i somehow lost
On 6/17/2020 4:24 PM, Marcel Fabian Krüger wrote: picking up the number at the tex end but i'll fix that) i'll add a tex.getmarks("top|bottom|...",<class number>) for whatveer use it might have and probably also backport that to luatex then Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
participants (2)
-
Hans Hagen -
Marcel Fabian Krüger