On 11 Aug 2016, at 10:18, Taco Hoekwater wrote:

Hi,

Working on the next issue of the CG Journal, I finally had a good reason to look at the new bibliography code. Even with a big manual, it was still a bit of challenge.

The article I was working on was written for mkii, and came with a prebuilt .bbl file for that (but not with the original .bib files). The original databases look like they were created for latex so there were some problems with embedded commands and it being 8-bit (ISO-latin), but that was to be expected.

Some other problems seems to be more related to differences between the mkii and the mkiv bibliography code.

First question: is there a ‘default’ or ‘standard’ dataset, or not? If so, is it named ‘default’ or ‘standard’? The bit of code on the wiki and the actual manual seem to disagree on that. My current solution looks like this:

\usebtxdataset[h-all-forced.bbl] \usebtxdefinitions[aps] \setupbtxrendering[alternative=aps] … \placelistofpublications[criterium=text]

That works, but almost all the documentation uses either [default] or [standard].

Second question: It appears I need both \usebtxdefinitions and \setupbtxrendering, which surprised me. I understand that right now, there is only ‘apa’ and ‘aps’, but it seems neither are preloaded? Without the \usebtxdefinitions, I did get a list of publications, but with horrible formatting, especially for ‘manual’ and ‘inproceedings’.

Contrary to what the manual says, \cite[key] does not seem to work. however, \citation[key] does. So, in the preamble I now have

\let\cite\citation

but isn’t it weird that I needed that?

More importantly (read: "more timeconsuming-ly"), there are some differences in the .bbl processing compared to mkii, which is not great. So far, I found this:

* mkii used \arttitle in some spots. To be exact: in ‘article’, ‘incollection’, and ‘inproceedings’ entries. I changed the ones in ‘article' to \title inside the .bll to get these to work.

* That fixed ‘article’s, but ‘inproceedings’ and ‘incollection’ are broken, because these *also* has an actual \title for the complete proceedings/collection, and I do not know how to make the module display both titles. Please help, because I definitely need the ‘inproceedings’ one to work properly! I now get formatted output like this:

[18] J.-M. Hufflen, Proc. 6th ConTEXt Meeting & EuroTEX 2012 (2012).

(actually twice the exact same entry, because there were two articles by J-M in that one proceedings). Do I need \booktitle? Not that I mind, but it is a little odd in the sense that a ‘collection’ has a title, but it is not really a ‘book title’, it is a ‘collection title’ (and likewise for 'proceedings’) ;)

* In mkii, the bbl uses e.h. \artauthor[]{John}[J.]{}{Doe}. The problem I have there is that the new biblio code (at least the ‘aps’ version) inserts dots after the initials in the output list, So I end up with “J.. Doe”. I like the ‘adding a dot’ in principle, but if it stays, then it should definitely be conditional on whether or not the initials already have one. Having to edit the datafile the fix the double dot was quite uncomfortable.

* I had to change the .bbl, replacing all \pubyear with \year. The extra \pub… prefix was there because of conflicts with the primitive \year in mkii and it makes sense to not require it any more in mkiv, but nevertheless it would be helpful if \pubyear could be used as an alias to \year. Not the hardest change though, it was a single global replace in the bbl, so a low priority request.

* The new ‘aps’ style does not print the publisher name for ‘book’ (at least, I probably missed others). I assume this is because of the ‘not quite ready yet’ state of new biblio code, but I hope that will be fixed at some point. And there are probably a few other (less common) fields still missing, right?. Probably because the .bbl uses e.g. \pubname, not ‘\publisher’? I don’t want to go looking for these missing fields just now, but I would be willing to do at some later date.

Finally: the manual says there are various ‘required fields’ for the entry types. That is great, but could there be some (optional) visual feedback in the formatted list if such a required field is in fact missing? traditionally, that would a black square, but a bold *publisher missing* would be very useful.

That’s it for now. Sorry about the long list of complaints. I do appreciate the amount of work needed to get bibliographies working even without having to worry about backward compatibility, but I feel it is important that, if there is supposed to be backward compatibility, that it ten be as close to perfect as technically possible. Even if it is a bit of pain.

Addendum: It seems \cite (aka \citation) does not work inside footnotes? I get output like this: … A good introduction to this scripting language is [<wall-etc2000>]. And the entry does not appear in the publication list.?! Best wishes, Taco

Hi Taco, On Thu, 11 Aug 2016 10:18:19 +0200 Taco Hoekwater wrote:

Hi,

Working on the next issue of the CG Journal, I finally had a good reason to look at the new bibliography code. Even with a big manual, it was still a bit of challenge.

The article I was working on was written for mkii, and came with a prebuilt .bbl file for that (but not with the original .bib files). The original databases look like they were created for latex so there were some problems with embedded commands and it being 8-bit (ISO-latin), but that was to be expected.

Some other problems seems to be more related to differences between the mkii and the mkiv bibliography code.

First question: is there a ‘default’ or ‘standard’ dataset, or not? If so, is it named ‘default’ or ‘standard’? The bit of code on the wiki and the actual manual seem to disagree on that. My current solution looks like this:

\usebtxdataset[h-all-forced.bbl] \usebtxdefinitions[aps] \setupbtxrendering[alternative=aps] … \placelistofpublications[criterium=text]

That works, but almost all the documentation uses either [default] or [standard].

I do not know anything about [standard]. For convenience, we define a "default" dataset. Hans has insisted, however, that it would be good practice for a user to explicitly define datasets.

Second question: It appears I need both \usebtxdefinitions and \setupbtxrendering, which surprised me. I understand that right now, there is only ‘apa’ and ‘aps’, but it seems neither are preloaded? Without the \usebtxdefinitions, I did get a list of publications, but with horrible formatting, especially for ‘manual’ and ‘inproceedings’.

The default (not to be confused with the dataset named "default") rendering is extremely minimal: only @book and @article are recognized. The rendering style is very simplified too. Hans wanted to keep this really minimal, so perhaps we can call it the "minimal" style. You do NOT need to setup the rendering after loading a more complete style. The manual states: An alternative set of specifications can be selected using, for example \usebtxdefinitions[apa] Alternately, the set of specifications can be loaded and (later) activated using \loadbtxdefinitionfile[apa] … \setupbtx[specification=apa] This is to allow loading multiple specifications (each in its own namespace) and then choosing specifications when needed. This is normally not very useful, but is used in the manual itself.

Contrary to what the manual says, \cite[key] does not seem to work. however, \citation[key] does. So, in the preamble I now have

\let\cite\citation

but isn’t it weird that I needed that?

??? line 1391 of publ-ini.mkiv has this definition.

More importantly (read: "more timeconsuming-ly"), there are some differences in the .bbl processing compared to mkii, which is not great. So far, I found this:

* mkii used \arttitle in some spots. To be exact: in ‘article’, ‘incollection’, and ‘inproceedings’ entries. I changed the ones in ‘article' to \title inside the .bll to get these to work.

* That fixed ‘article’s, but ‘inproceedings’ and ‘incollection’ are broken, because these *also* has an actual \title for the complete proceedings/collection, and I do not know how to make the module display both titles. Please help, because I definitely need the ‘inproceedings’ one to work properly! I now get formatted output like this:

[18] J.-M. Hufflen, Proc. 6th ConTEXt Meeting & EuroTEX 2012 (2012).

(actually twice the exact same entry, because there were two articles by J-M in that one proceedings). Do I need \booktitle? Not that I mind, but it is a little odd in the sense that a ‘collection’ has a title, but it is not really a ‘book title’, it is a ‘collection title’ (and likewise for 'proceedings’) ;)

* In mkii, the bbl uses e.h. \artauthor[]{John}[J.]{}{Doe}. The problem I have there is that the new biblio code (at least the ‘aps’ version) inserts dots after the initials in the output list, So I end up with “J.. Doe”. I like the ‘adding a dot’ in principle, but if it stays, then it should definitely be conditional on whether or not the initials already have one. Having to edit the datafile the fix the double dot was quite uncomfortable.

* I had to change the .bbl, replacing all \pubyear with \year. The extra \pub… prefix was there because of conflicts with the primitive \year in mkii and it makes sense to not require it any more in mkiv, but nevertheless it would be helpful if \pubyear could be used as an alias to \year. Not the hardest change though, it was a single global replace in the bbl, so a low priority request.

I noticed this in converting Idris' .bbl files. As I never used (nor know) the old .bbl format, I did not spend any time on this, nor did I bother Hans to look into it, but we probably need to tune the .bbl -> .bib (i.e. lua tables) conversion.

* The new ‘aps’ style does not print the publisher name for ‘book’ (at least, I probably missed others). I assume this is because of the ‘not quite ready yet’ state of new biblio code, but I hope that will be fixed at some point. And there are probably a few other (less common) fields still missing, right?. Probably because the .bbl uses e.g. \pubname, not ‘\publisher’? I don’t want to go looking for these missing fields just now, but I would be willing to do at some later date.

pubname is NOT a bibtex field name, whereas publisher is. This does not mean that pubname is incorrect for a .bbl, so this is an example of tuning the .bbl -> .bib conversion, not a shortcoming of the rendering specifications.

Finally: the manual says there are various ‘required fields’ for the entry types. That is great, but could there be some (optional) visual feedback in the formatted list if such a required field is in fact missing? traditionally, that would a black square, but a bold *publisher missing* would be very useful.

There are debugging tools. "required" vs. "optional" is a relative notion, and the rendering specifications are tuned to remain robust when fields are missing. The notion of required comes from the original bibtex specifications, and is reflected in tools such as jabref. But these do not necessarily correspond to the official APA specifications, nor on the APS guidelines.

That’s it for now. Sorry about the long list of complaints. I do appreciate the amount of work needed to get bibliographies working even without having to worry about backward compatibility, but I feel it is important that, if there is supposed to be backward compatibility, that it ten be as close to perfect as technically possible. Even if it is a bit of pain.

It sounds like the limitations that you see are in the .bbl conversions. We are also working with Idris right now who is also exercising the new system with his own requirements. Alan -- Alan Braslau CEA DSM-IRAMIS-SPEC CNRS UMR 3680 Orme des Merisiers 91191 Gif-sur-Yvette cedex FRANCE tel: +33 1 69 08 73 15 fax: +33 1 69 08 87 86 mailto:alan.braslau@cea.fr