After one night, let me rephrase my feature request. I am aware that context has big problems with redirection but latex is also facing similar problems. I do not know the internals of synctex support in luatex but let me just infer that there is a storage for the tag and a storage for the line which are used as is when feeding the synctex attributes of the nodes. Let me call them <current tag> and <current line>. I assume also that there is a tag→ file name key value storage and a function to create a new entry in that table, let me call them <tag map> and <new entry>. I need a lua table named "synctex" with following methods synctex.set_tag(<n>): change <current tag> to <n> synctex.get_tag(<file>): if <file> == nil: return <current tag> If <file> is not a value of <tag map>: call <new entry> return the key for <file> in <tag map> synctex.set_line(<n>): change <current line> to <n> synctex.get_line(): get <current line> These cannot be replaced by the actual force methods which are too aggressive. Finally synctex.save_state, synctex.restore_state a public API to the already existing stack mechanism used by \input for example This will certainly solve the \openin and \read requests due to the callbacks. Being an addition, this low level implementation does not break anything existing. If you do not feel the need for such API, then I would be happy to propose a contribution when I know how to start.

Le 30 mars 2022 à 22:16, Hans Hagen a écrit :

...

There are situations where synctex info is not available: consider ``` \immediate\openin 0 {subfile.tex} \immediate\read 0 to \My \immediate\closein 0 \My \bye ``` Synctex does not see `subfile.tex` at all, which forbids any kind of synchronization between subfile.tex and main.pdf. It would be very cool if the \openin could create a new tag and the \read could use that tag while parsing subfile.tex. A synctex information about the line number would be very nice as well, a default of 1 is far better than nothing. A tex.get_synctex_tag_for_name(‘foo') would be extremely handy. Finally, raw setters and getters should be provided for synctex tag and line, to control exactly what tag and line is written to the .synctex file. No save stack for them, no mode. Simply raw. Some global synctex state stack would help: save_synctex_state/restore_synctex_state… BTW that would be great to deactivate set_synctex_tag for mode 0 just like set_synctex_line is.

On 3/30/2022 9:18 PM, Jérôme LAURENS wrote: the 'forced' tag and line always win and opening files is under callback control so basically one can (only) use those in combination with synctex mode set from the lua end; it has never been the intention to provide interfaces to synctex beyond forcing tag and line numbers that end up in nodes;

the only reason that there is that little bit of (lua) alternative interfacing is that in context we had to get around some limitations (afaiwn synctex works ok with plain latex but not that well with context so we had to hack around a bit) .. basically synctex support is just a black box as in the other tex engines and not really part of 'opening up the internals' (also because in the end the output should be useable by the synctex file interpretation code)

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl ----------------------------------------------------------------- _______________________________________________ dev-luatex mailing list dev-luatex@ntg.nl https://mailman.ntg.nl/mailman/listinfo/dev-luatex

I think a separate lua module at the top level would be more appropriate than a dedicated table in a table. It would be a lua frontend to raw synctex related C features with absolutely no extra logic. That would preserve code separation principles. What is actually in the tex table being more related to how TeX/Context uses Synctex. This library would be available in texlua as well such that eventually in the future, it would allow to create a complete ‘.synctex’ file from scratch, for testing purposes for example, or to amend existing .synctex files. Looking backwards, SyncTeX was written in POC on both ends with absolutely no extra C library such that some basic features were not implemented, Now that LuaTeX can access more libraries on the C side, there is eventually room for future enhancements. One missing feature is the tag←→file name mapping which prevents to use different tags for the same file, when input multiple times. Another one is to feed the .synctex file with paths relative to the directory of the main file, when appropriate. It serves 3 purposes: 1) do not expose info on the content of one’s HD (on the internet) 2) allow to duplicate/share whole tex source folders without breaking synchronization (w/o a new compilation, in the pdf copy, when you click somewhere, you are redirected to the original source instead of the copy). 3) allow to use the same source folder on different OS (see dropbox/git/overleaf) For that, path management may be easier with LuaTeX. While playing with the various synctex related functions on the lua side, I came across some lines in the .synctex file that were not covered by the actual specs (according to `man 5 synctex`). I’ll be happy to add any extension. Also, some unexpected big tag numbers appeared during the process, rather weird...

Le 31 mars 2022 à 10:32, Hans Hagen a écrit :

On 3/31/2022 9:33 AM, Jérôme LAURENS wrote:

...

After one night, let me rephrase my feature request. I am aware that context has big problems with redirection but latex is also facing similar problems.

i remember discussions long ago (early luatex times) where there wasn't much room for adding control over filenames in synctex etc as needed in context otherwise we could have taken it into account then; so in the end context implemented its own synctex file generator that works ok (complications more arise from the switch to a new syntex file format and editors/viewers that get confused but that also get sorted out eventually);

so the 'good' news is that the library doesn't have to keep context in mind as it is not used

...

I do not know the internals of synctex support in luatex but let me just infer that there is a storage for the tag and a storage for the line which are used as is when feeding the synctex attributes of the nodes. Let me call them <current tag> and <current line>.

not that much code; just what other engines do, but one can enforce tag/line when in a different synctex mode (controlled by parameter) ... as with everything luatex that then assumes that one handles specific wishes / features in lua (one can also run over nodelists and tweak synctex fields)

...

I assume also that there is a tag→ file name key value storage and a function to create a new entry in that table, let me call them <tag map> and <new entry>.

there is no such list (there is the input stack that has filenames but that's a regular tex thing, adapted to luatex); if one sets tags from lua one can also maintain additional info there (there are methods to fetch the current filename but not wrt synctex)

...

I need a lua table named "synctex" with following methods synctex.set_tag(<n>): change <current tag> to <n>

that's not how it works in luatex, we don't really manage lua tables (there are some interfaces that look like that but they either generate the table on demand or they are virtual tables); so in this case, say

tex.synctextable

would have to call out to the library and build that table (not that hard assuming there are methods in the library to loop)

so, e.g. latex can implement a synctex (namespace) table with whatever methods it needs using the few tex.*synctex* calls that interface to the library; it is the library itself that has to do the work; a few extra interface function from lua to the library are cheap; the luatex file related machinery is already complex enough

...

synctex.get_tag(<file>): if <file> == nil: return <current tag> If <file> is not a value of <tag map>: call <new entry> return the key for <file> in <tag map> synctex.set_line(<n>): change <current line> to <n> synctex.get_line(): get <current line>

whatever gets added (maybe accessors to info available in the library) will not add logic at the lua end; because of frequent callout to lua an dprint from and that also being input / output it would add complications and overhead; that's why we have callbacks, because no solution si good for all macro packages (now or in the future)

...

These cannot be replaced by the actual force methods which are too aggressive.

one can maintain a stack in lua and push pop the forced settings but if the library itself does it that's also an option

...

Finally synctex.save_state, synctex.restore_state a public API to the already existing stack mechanism used by \input for example

it it's just calling a synctex library function that can be added, something:

tex.synctex_save_state() tex.synctex_restore_state() tagnumber = tex.synctex_save_tag(filename) filename = tex.synctex_restore_tag(n) tex.synctex_save_line(n) linennumber = tex.synctex_restore_line()

where the library implements those .. for lua these are just calls to the library ... it would also limite the changes to the default synxtex mode (read: no need to use the force setters because those operate in the non standard mode)

...

This will certainly solve the \openin and \read requests due to the callbacks.

ok

Hans

----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------