Hi, thanks for your assessment. Indeed, that looks very promising. I've now had a chance to look at this a bit more too. Not that I understand that stuff in detail, but anyway... I also have the impression that extending citeproc-formats.lua should be the easiest part. As an aside, I think the HTML is in there because that is needed for the test suite, but for ConTeXt we should be able to copy the LaTeX mapping and tweak a few things here and there. The LaTeX package is hard to understand, at least if you're not used to the "new" latex3 programming syntax and style. I also can only guess what that stuff does. But as you say this is rather isolated from the citeproc processing, so I guess the main task would be to come up with an equivalent ConTeXt module that writes the citations to the tuc file, calls the citeproc engine, and uses the results for the output. Right? (I.e., AFAICS, everything that lives under https://github.com/zepinglee/citeproc-lua/tree/main/citeproc should pretty much just work, while https://github.com/zepinglee/citeproc-lua/tree/main/latex must be adapted.) For the record: the relevant stuff in the aux file looks like this: \bibstyle {apa} \bibdata {examples.bib} \csloptions {} \citation {{texbook@1}{{id={texbook},label={page},locator={12}}}{1}} \citation {{latex2e@1}{{id={latex2e},prefix={see }}}{1}} \citation {{texbook@2}{{id={texbook},prefix={see },label={page},locator={12}}}{1}} \citation {{texbook@3}{{id={texbook},prefix={see },suffix={, everywhere}}}{1}} \citation {{texbook@4}{{id={texbook},prefix={see },label={page},locator={12, and passim}}}{1}} But I guess that's the way these things work in latex, and adding that stuff to a lua table will probably save us much trouble anyway. (E.g., no need for https://github.com/zepinglee/citeproc-lua/blob/main/latex/citeproc#L81, or https://github.com/zepinglee/citeproc-lua/blob/main/latex/csl-core.lua#L168) Regarding the citation commands, I've noticed that while biblatex's syntax model (\cite[prenote][postnote]{key}, e.g. \cite[see][34]{doe}) is supported, the preferred method is actually \cite[prefix={See }, page={34}]{doe}. But that's of course not so important since the important question is how that stuff is passed to the citeproc. Best, Denis

-----Ursprüngliche Nachricht----- Von: Taco Hoekwater Gesendet: Freitag, 6. Mai 2022 14:57 An: mailing list for ConTeXt users Cc: Maier, Denis Christian (UB) Betreff: Re: [NTG-context] Citeproc-lua

Hi,

I had a quick look at this and it seems very extensible if-not-quite ConTeXt- ready right away.

On the input side, it reads a LaTeX .aux file to parse things like options, databases, and requested citations. For this, it uses a dedicated “citeproc” lua script that should be easy to modify/copy for someone who knows what is actually in the ConTeXt .tuc file.

On the output side, it uses a lookup table to generate the desired markup (in citeproc-formats.lua). Right now, there are definitions provided for HTML and LaTeX, but that is very simple to extend.

There is a LaTeX package for the integration (making sure the right commands appear in the aux file, option processing, et cetera), and that file I do not understand *at all*. It seems to be completely isolated from the core citeproc processing, though.

Very promising stuff, I think.

But: I personally do not understand how to do the integration with ConTeXt’s new biblio system. I only understand half of what ConTeXt does these days, and much less of what the LaTeX package is trying to achieve i.r.t. options.

Best wishes, Taco

...

On 3 May 2022, at 22:00, Denis Maier via ntg-context wrote:

Hi,

I just came across this: https://github.com/zepinglee/citeproc-lua Thats’s an lua implemenation of the CSL language targeting LaTeX. I don’t know if the processor itself contains any LaTeX specific code, but it looks like the integration basically happens here :https://github.com/zepinglee/citeproc-lua/blob/main/latex/citation-st yle-language.sty So, at least in theory, shouldn’t it be possible to make this workable also with context?

Best, Denis

______________________________________________________________ ________

...

_____________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net

______________________________________________________________ ________

...

_____________

Taco Hoekwater Elvenkind BV

On 5/6/2022 11:09 PM, Aditya Mahajan via ntg-context wrote:

On Fri, 6 May 2022, Denis Maier via ntg-context wrote:

...

Hi, thanks for your assessment. Indeed, that looks very promising.

I've now had a chance to look at this a bit more too. Not that I understand that stuff in detail, but anyway...

I also have the impression that extending citeproc-formats.lua should be the easiest part. As an aside, I think the HTML is in there because that is needed for the test suite, but for ConTeXt we should be able to copy the LaTeX mapping and tweak a few things here and there.

The LaTeX package is hard to understand, at least if you're not used to the "new" latex3 programming syntax and style. I also can only guess what that stuff does. But as you say this is rather isolated from the citeproc processing, so I guess the main task would be to come up with an equivalent ConTeXt module that writes the citations to the tuc file, calls the citeproc engine, and uses the results for the output. Right? (I.e., AFAICS, everything that lives under https://github.com/zepinglee/citeproc-lua/tree/main/citeproc should pretty much just work, while https://github.com/zepinglee/citeproc-lua/tree/main/latex must be adapted.)

Does citeproc-lua/biblatex support using CSL files (https://citationstyles.org/). IMO, that is currently the biggest limitation of the context bib module that only a few citation styles are supported. Developing a new citation style is relatively easy but a lot of work. Being able to use CSL will simplify things (from the point of view of the user) a lot.

the main question is 'how easy is it to convertt one of these xml defs into a context rendering' ... probably somethng we can do stepwise using some external solution creates a dependency (and also maintainance issue i guess) and there is a good change for interferences (btw, in the apa one that we support the main complication is that it has many variants / border cases) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

Yes, Aditya. Citeproc-lua uses csl styles. Test suite coverage is not complete yet, though. Denis ________________________________________ Von: ntg-context im Auftrag von Hans Hagen via ntg-context Gesendet: Freitag, 6. Mai 2022 23:44:38 An: Denis Maier via ntg-context Cc: Hans Hagen Betreff: Re: [NTG-context] Citeproc-lua On 5/6/2022 11:02 PM, Denis Maier via ntg-context wrote:

I also have the impression that extending citeproc-formats.lua should be the easiest part. As an aside, I think the HTML is in there because that is needed for the test suite, but for ConTeXt we should be able to copy the LaTeX mapping and tweak a few things here and there.

The LaTeX package is hard to understand, at least if you're not used to the "new" latex3 programming syntax and style. I also can only guess what that stuff does. But as you say this is rather isolated from the citeproc processing, so I guess the main task would be to come up with an equivalent ConTeXt module that writes the citations to the tuc file, calls the citeproc engine, and uses the results for the output. Right? (I.e., AFAICS, everything that lives under https://github.com/zepinglee/citeproc-lua/tree/main/citeproc should pretty much just work, while https://github.com/zepinglee/citeproc-lua/tree/main/latex must be adapted.)

For the record: i don't think you should write something to the tuc file that doesn't come from context itself because you can mess it up (also performaance wise). Of course you could have another file (just like a bib file is independent). One problem could be that you need to make some extra installation to make it work as we're not going to add all kind of code to the distribution (we tend to go smaller) and someone needs to maintain that moduie then because users depend in it working. Did you look into what pandoc provides? It might be easier to take that output and include it. Some kind of html? That's easy to render. A bit like this: - Convert the whole bibliography database to html using pandoc in the preferred cs rendering. That should be fast. - Use the normal context commands for referencing a citation (the cite part is normally easy as there is not much variation in that; if needed one can cheat and also pregenerate that). That's then just some relatively small plugin mode. - When placing the bibliography, filter the right entries from that html file (easy) using info that got stored in the tuc. It is also fast. The only depdency then is pandoc but that is widely available (irr). But I would need to see an example of that kind of out first. We basically treat the (formatted) bibliography as an external resource but in some format that we can easily parse (and if needed tweak). Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___________________________________________________________________________________

On 5/7/2022 12:33 AM, denis.maier@unibe.ch wrote:

...

-----Ursprüngliche Nachricht----- Von: ntg-context Im Auftrag von Hans Hagen via ntg-context Gesendet: Freitag, 6. Mai 2022 23:45 An: Denis Maier via ntg-context Cc: Hans Hagen Betreff: Re: [NTG-context] Citeproc-lua

[...]

For the record: i don't think you should write something to the tuc file that doesn't come from context itself because you can mess it up (also performaance wise).

Really? But what about that stuff? Anyway, even better if the data coming from context itself can be used.

A bibliography is just about (cross) referencing data. If that data comes from elsewhere it can best kept indepdendent. One can of course store some reference but why the data if it is not used in some special way.

Of course you could have another file (just like a bib file

...

is independent). One problem could be that you need to make some extra installation to make it work as we're not going to add all kind of code to the distribution (we tend to go smaller) and someone needs to maintain that moduie then because users depend in it working.

Did you look into what pandoc provides? It might be easier to take that output and include it. Some kind of html? That's easy to render.

Pandoc can output context as well, so we can just take that.

Sure, but some parseble format might be better if one wants control over rendering. Basically we're talking about some pseudo formatted bibliography entry with some unique tag as reference.

Using the whole bibliography might give problems with regards to disambiguation.

In what sense? Aren't the 'tags' or id's unique? Anyway, one can - let \cite write some refe to soem file - that file gets pandoc'd to a bibliography list - that then gets included if that list is in some parsable format (like html width classes) we can easily pick up info (if needed) and do nmore

So, that would work a bit like Aditya's filter module: https://github.com/adityam/filter Right?

sort of

Another option would be to just use pandoc's citeproc directly, as described here: https://github.com/jgm/citeproc/blob/master/man/citeproc.1.md

We'd have to pass a list of citations as a JSON object to that citeproc and use the results. The results will include citations as well as the bibliography (all as JSON, content can be HTML formatted). we can handle json indeed, so hwo does such a json look like? can you make an example?

Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------

On 2022-05-06 17:44, Hans Hagen via ntg-context wrote:

[...] A bit like this:

- Convert the whole bibliography database to html using pandoc in the preferred cs rendering. That should be fast.

- Use the normal context commands for referencing a citation (the cite part is normally easy as there is not much variation in that; if needed one can cheat and also pregenerate that). That's then just some relatively small plugin mode.

- When placing the bibliography, filter the right entries from that html file (easy) using info that got stored in the tuc.

It is also fast. The only depdency then is pandoc but that is widely available (irr). But I would need to see an example of that kind of out first. We basically treat the (formatted) bibliography as an external resource but in some format that we can easily parse (and if needed tweak).

Hans

This is similar to my strategy today. I create a pandoc markdown file from my ConTeXt source using a simple script and generate ConTeXt output containing just the bibliography using pandoc. This output needs some minor tweaks, again scripted, to do things like replace the space in /Vol. 999/ with a non-breaking space to keep the following number on the same line and to convert /\em/ and such to taggable macros that I define in my environment. The result is ConText source that can be further edited (if necessary) and inserted where needed/./ I have not extended it to multiple bibliographies in  a single document, although that should not be difficult. Another possibility would be for ConTeXt, under control of a flag or setting, to generate an editable bibliography file that could be included under control of a second flag or setting. -- Rik