[OT] Research into Generative Typesetting
Hi everyone, As some of you may know, in 2010 I attained a masters degree in New Media from the University of Amsterdam. The title of my thesis is "Grammars of Process: Agency, Collective Becoming, and the Organization of Software". * [PDF](http://drippingdigital.com/gop/grammars_of_process.pdf) * [TeX](http://drippingdigital.com/gop/grammars_of_process.tex) * [HTML](http://drippingdigital.com/gop) * [thesis-groomer.rb]( https://github.com/ab5tract/new_media/blob/master/thesis/infrastructure/thes...), in case you are interested in the glue. Linked above are the PDF and HTML versions of the thesis, which attempts to examine generative typesetting (ie, going from one input into multiple outputs with various properties) through the lens of new media theory and the 'transductive' cybernetics of Gilbert Simondon. The introduction is particularly heavy with new media theory, but as the chapters progress I think it becomes more readable for those unfamiliar with this discourse. The 'Operating Systems' chapter was quite fun to write and hopefully contains some interesting history and reflections. One of the things this list might be excited about is the typography itself. I use many of Robert Bringhurst's suggestions, and I think the output is much the better for his advice.[^1] The HTML version was never totally finished (I'd prefer it to have JavaScript interaction and more Web-specific functionality). In fact, I had quite some plans for it, but as the deadline of the thesis approached, I necessarily poured my efforts more into content than presentation, and where presentation was concerned I was much more preoccupied with the Context version. All in all, I am happy with the thesis but I also know that it could use some work. If you have any feedback, please let me know. [^1]: If anyone is interested, I'm thinking I might make a module that sets up the environment according to these conventions. Because I was concerned with both PDF and HTML output, I chose to work in Markdown with Pandoc as my input format. Through the course of trying to manage a generative workflow, I had to make some unfortunate concessions: on the one hand, I had to process some of the input in order to get some basic things I needed, such as a blockquote environment for Context, handling the title page layout and abstract, among other things; and on the other hand I was forced to avoid bibliographic automation and instead had to be very careful to manually cite all my references. (This was because, at the time, Pandoc did not have the any capacity to do citation management that would work within HTML; with its new CiteProc support it seems that this is now feasible). Tagged PDF support is out of the question due to Pandoc generally only having MkII support, including lacking newer stuff like \startchapter..\stopchapter and \hyphenatedURL. The main conclusion I had about generative typesetting is that we are missing a crucial "glue layer". I have written some on a system which I call Subtext, a mutable translation layer where one is in control of both the syntax and the translation effects from a configuration file. Pandoc is great, but Subtext approaches the edge cases of generative typesetting differently by encouraging output-specificities without requiring anyone to learn Haskell to gain a little more control. Looking over Hans' new `m-markdown` code, I am beginning to see a clearer vision of how to go about implementing this. I've done some presentations on generative typesetting: * [Generative Typesetting @ Libre Graphics Meeting 2011]( http://river-valley.tv/generative-typesetting-with-context/) * [Textual Liberation @ The Unbound Book Conference]( http://e-boekenstad.nl/unbound/index.php/john-haltiwanger-generative-typeset... ) * [Sozi notes]( http://drippingdigital.com/conf/unbound-book/textual-liberation.svg) Anyway, I thought that perhaps someone here might find some bit of this interesting. Have a great weekend! Sincerely, John
First of all, congratulations on your work!
Hi everyone,
As some of you may know, in 2010 I attained a masters degree in New Media from the University of Amsterdam. The title of my thesis is "Grammars of Process: Agency, Collective Becoming, and the Organization of Software".
* [PDF](http://drippingdigital.com/gop/grammars_of_process.pdf) * [TeX](http://drippingdigital.com/gop/grammars_of_process.tex) * [HTML](http://drippingdigital.com/gop) * [thesis-
It's always interesting to see what others do and especially how they do it with context. Really great that you are also linking to the source.
One of the things this list might be excited about is the typography itself. I use many of Robert Bringhurst's suggestions, and I think the output is much the better for his advice.[^1] The HTML version was never totally finished (I'd prefer it to have JavaScript interaction and more Web-specific functionality). In fact, I had quite some plans for it, but as the deadline of the thesis approached, I necessarily poured my efforts more into content than presentation, and where presentation was concerned I was much more preoccupied with the Context version. All in all, I am happy with the thesis but I also know that it could use some work. If you have any feedback, please let me know.
[^1]: If anyone is interested, I'm thinking I might make a module that sets up the environment according to these conventions.
I like the idea of pre-made styles. I'm not shure if a module would provide the necessary flexibility, though. Maybe a style collection (with commented code and linked sample output PDFs) in the wiki would do the job. Like there is for the biochemistry textbook. From there users could just use it as a whole or adapt it to their needs.
From scrolling through the pages I must say that your design is very appealing. I would change some minor details, though (But I'm sure you've had your reasons :) -Structure level 2 and 3 are not visually distinct (only by the numbers) see e.g. page 65. Maybe using italics for the 3rd level (like Bringhurst does it) would help to distinguish the titles. -No spacing after paragraphs if you already use indent. (Afaik the consensus is "indent or space, not both") -Indenting after a picture or line of code looks odd to me and can cause strange results. (See e.g. page 65. After "rm -rf /*" there is an indent in the middle of the sentence. -There are some issues with the links in the bibliography (e.g. page 99). Also, bibliographies should be set flushleft imho, due to their brief nature they don't really provide enough words to allow automated paragraph construction. The result is a somewhat uneven feel due to big word spacing.
But who am I to grouch? My thesis currently consists of 5 pages xD so I should really not be pointing fingers here^^ Again, congratulations on your thesis. All the best Christian
On Sat, Jul 23, 2011 at 9:37 AM, Christian
One of the things this list might be excited about is the typography itself. I use many of Robert Bringhurst's suggestions, and I think the output is much the better for his advice.[^1] The HTML version was never totally finished (I'd prefer it to have JavaScript interaction and more Web-specific functionality). In fact, I had quite some plans for it, but as the deadline of the thesis approached, I necessarily poured my efforts more into content than presentation, and where presentation was concerned I was much more preoccupied with the Context version. All in all, I am happy with the thesis but I also know that it could use some work. If you have any feedback, please let me know.
[^1]: If anyone is interested, I'm thinking I might make a module that sets up the environment according to these conventions.
I like the idea of pre-made styles. I'm not shure if a module would provide the necessary flexibility, though. Maybe a style collection (with commented code and linked sample output PDFs) in the wiki would do the job. Like there is for the biochemistry textbook. From there users could just use it as a whole or adapt it to their needs.
That is certainly one approach. Hoewever, I'd like to make it as easy as possible for people who just want to use defaults to do so. One solution is to have a script that generates a scaffold environment inside a Context source file. Then they do not need to keep multiple templates around or have internet access when creating a document. (I know that for experienced Context users, having some .env files or source templates around is no problem, but I'm thinking of users who are less experienced or just want to typeset Markdown documents without messing with TeX). ./context-style-gen.rb --style=bringhurst --input=myThesis.markdown myThesis.tex A style I am _really_ excited about is Tufte's, something along the lines of the 'tufte-latex' package (but without the dozen or so dependencies it requres in LaTeX, since we should be able to do all that functionality just from core Context :) So if anyone is interested in working on such an environment, please get in touch with me.
From scrolling through the pages I must say that your design is very appealing. I would change some minor details, though (But I'm sure you've had your reasons :) -Structure level 2 and 3 are not visually distinct (only by the numbers) see e.g. page 65. Maybe using italics for the 3rd level (like Bringhurst does it) would help to distinguish the titles. -No spacing after paragraphs if you already use indent. (Afaik the consensus is "indent or space, not both") -Indenting after a picture or line of code looks odd to me and can cause strange results. (See e.g. page 65. After "rm -rf /*" there is an indent in the middle of the sentence. -There are some issues with the links in the bibliography (e.g. page 99). Also, bibliographies should be set flushleft imho, due to their brief nature they don't really provide enough words to allow automated paragraph construction. The result is a somewhat uneven feel due to big word spacing.
All very good points, thank you!
But who am I to grouch? My thesis currently consists of 5 pages xD so I should really not be pointing fingers here^^ Again, congratulations on your thesis.
And good luck to you on yours! Cheers, John
On Wed, Jul 27, 2011 at 6:30 AM, John Haltiwanger
On Sat, Jul 23, 2011 at 9:37 AM, Christian
wrote: [^1]: If anyone is interested, I'm thinking I might make a module that sets up the environment according to these conventions.
I like the idea of pre-made styles. I'm not shure if a module would provide the necessary flexibility, though. Maybe a style collection (with commented code and linked sample output PDFs) in the wiki would do the job. Like there is for the biochemistry textbook. From there users could just use it as a whole or adapt it to their needs.
That is certainly one approach. Hoewever, I'd like to make it as easy as possible for people who just want to use defaults to do so. One solution is to have a script that generates a scaffold environment inside a Context source file. Then they do not need to keep multiple templates around or have internet access when creating a document. (I know that for experienced Context users, having some .env files or source templates around is no problem, but I'm thinking of users who are less experienced or just want to typeset Markdown documents without messing with TeX).
./context-style-gen.rb --style=bringhurst --input=myThesis.markdown myThesis.tex
Note: The following is untested. You don't need another script, context can handle this natively. For example, you can write a 'process-markdown' module that directly processes a markdown file: (I haven't looked at the new markdown module. I am assuming that it provides a macro, \inputmarkdown{...} to process a markdown file. \startmodule[process-markdown] \starttext \inputmarkdown{\inputfilename} \stoptext \stopmodule Then, you can call ConTeXt using context --usemodules=tufte,other-style,process-markdown --result=whatever file.markdown I followed a similar approach when I was processing markdown files using the filter module. See http://randomdeterminism.wordpress.com/2011/01/09/markdowntopdf/ I used modes instead of a separate module and an evinronment instead of a module because that fit my workflow better. @Hans: do you think that it is a good idea to include something like this in the m-markdown module so that a user could say context --usemodule=markdown --mode=process file.markdown and get a pdf output. Aditya
Am 28.07.2011 um 00:21 schrieb Aditya Mahajan:
@Hans: do you think that it is a good idea to include something like this in the m-markdown module so that a user could say
context --usemodule=markdown --mode=process file.markdown
and get a pdf output.
It’s already there: context --extra=markdown file.markdown There are limitations in Hans method because you can’t define completely customized styles with it and the basic markdown tags are not enough to produce a document title (should be simple with the multimarkdown extensions). Wolfgang
On Thu, 28 Jul 2011, Wolfgang Schuster wrote:
Am 28.07.2011 um 00:21 schrieb Aditya Mahajan:
@Hans: do you think that it is a good idea to include something like this in the m-markdown module so that a user could say
context --usemodule=markdown --mode=process file.markdown
and get a pdf output.
It’s already there:
context --extra=markdown file.markdown
Does not work. `context --extra=markdown filename` just gives a list of available extras (including markdown), but the file is not processed.
There are limitations in Hans method because you can’t define completely customized styles with it and the basic markdown tags are not enough to produce a document title (should be simple with the multimarkdown extensions).
Or pandoc's extensions. Aditya
On 29-7-2011 8:08, Aditya Mahajan wrote:
On Thu, 28 Jul 2011, Wolfgang Schuster wrote:
Am 28.07.2011 um 00:21 schrieb Aditya Mahajan:
@Hans: do you think that it is a good idea to include something like this in the m-markdown module so that a user could say
context --usemodule=markdown --mode=process file.markdown
and get a pdf output.
It’s already there:
context --extra=markdown file.markdown
Does not work. `context --extra=markdown filename` just gives a list of available extras (including markdown), but the file is not processed.
Is there a mtx-context-markdown.tex file on your system? What happens if you do: context --extra=markdown --help mtx-context | mtx-context | extra: markdown (c:/data/develop/context/sources/mtx-context-markdown.tex) mtx-context | mtx-context | usage: context --extra=markdown [options] list-of-files mtx-context | mtx-context | --sort : sort filenames first mtx-context | --paperoffset=dimension : left-top-offset mtx-context | --duplex : doublesided (singlesided is default) mtx-context | --backspace=dimension : extra left offset mtx-context | --topspace=dimension : extra top offset mtx-context | --bodyfont=specification : additional bodyfont specification mtx-context | --contents : add table of contents
There are limitations in Hans method because you can’t define completely customized styles with it and the basic markdown tags are not enough to produce a document title (should be simple with the multimarkdown extensions).
Or pandoc's extensions.
In due time we can extend the parser (given that I know the specs). Btw, the more extensions you have, the more one can consider other approached (context with export or just xml). Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On Fri, 29 Jul 2011, Hans Hagen wrote:
On 29-7-2011 8:08, Aditya Mahajan wrote:
On Thu, 28 Jul 2011, Wolfgang Schuster wrote:
Am 28.07.2011 um 00:21 schrieb Aditya Mahajan:
@Hans: do you think that it is a good idea to include something like this in the m-markdown module so that a user could say
context --usemodule=markdown --mode=process file.markdown
and get a pdf output.
It’s already there:
context --extra=markdown file.markdown
Does not work. `context --extra=markdown filename` just gives a list of available extras (including markdown), but the file is not processed.
Is there a mtx-context-markdown.tex file on your system?
Yes. But I needed to run mtxrun --generate (my update scripts were not running this). After that everything works fine. If I compile a file called 1.md, the output file is called context-extra.pdf. I think it will be better if the default output is <basename>.pdf.
In due time we can extend the parser (given that I know the specs). Btw, the more extensions you have, the more one can consider other approached (context with export or just xml).
Agreed. Personally, I have now moved from markdown to TeX markup for big projects (although I have thoroughly tested the xml export). Aditya
Am 30.07.2011 um 04:59 schrieb Aditya Mahajan:
On Fri, 29 Jul 2011, Hans Hagen wrote:
In due time we can extend the parser (given that I know the specs). Btw, the more extensions you have, the more one can consider other approached (context with export or just xml).
Agreed. Personally, I have now moved from markdown to TeX markup for big projects (although I have thoroughly tested the xml export).
There are a few extension which can be usefull but for complex documents TeX is the preferred input method (especially math). Wolfgang
On 30-7-2011 4:59, Aditya Mahajan wrote:
If I compile a file called 1.md, the output file is called context-extra.pdf. I think it will be better if the default output is <basename>.pdf.
that is problematic with other extras as there we often have temp output files (e.g. extra=listing) one can of course use --result=<basename> ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On Sat, 30 Jul 2011, Hans Hagen wrote:
On 30-7-2011 4:59, Aditya Mahajan wrote:
If I compile a file called 1.md, the output file is called context-extra.pdf. I think it will be better if the default output is <basename>.pdf.
that is problematic with other extras as there we often have temp output files (e.g. extra=listing)
one can of course use --result=<basename>
OK. Can the load order of mtx-context-markdown be changed? Suppose I have an environment file, env.tex that sets fonts, styles, and layout. If I use context --extra=markdown --environment=env file then the environment file is loaded *before* mtx-context-markdown.tex. Therefore, the font setup and layout set in the environment file are overwritten by mtx-context-markdown. I am not sure what is the best way to avoid this. Aditya
On Fri, 29 Jul 2011, Hans Hagen wrote:
In due time we can extend the parser (given that I know the specs).
A small feature is missing in the handling of figures. The markdown syntax[1] states that figures are specified as follows: ![Alt text](/path/to/img.jpg) Pandoc, when translating from markdown to latex/context, treats the alt-text as the figure caption. ConTeXt, just ignores the alt text. I would suggest changing the c_image function to (not completely sure about this) local function c_image(lab,src,tit) return { "\\placemarkdownfigure{", lab, "}", "{", "\\externalfigure[", src, "]", "}" } end where \placemarkdownfigure is a new type of float. I have not used any optional argument to \placemarkdownfigure so that the options can be set using \setupfloat. The default setup could be \setupfloat[markdownfigure][default={here,nonumber}] What do others think? Aditya [1]: http://daringfireball.net/projects/markdown/syntax#img
participants (5)
-
Aditya Mahajan
-
Christian
-
Hans Hagen
-
John Haltiwanger
-
Wolfgang Schuster