Running mtxrun in a continuous batch mode
Hello, I am trying to run mtxrun in a batch process and read the available documentation here - https://wiki.contextgarden.net/Running_ConTeXt_without_a_shell My usecase is being able to create PDF docs based on the input data that comes in a JSON format (in a continuous batch process). In ConTeXt, thanks to the Lua engine, I am able to inject a JSON read from a file - and use the JSON in the tex file (here variable `tab` holds the JSON). How do I make this as a continuous webservice - in comes JSON and out comes the PDF ? \startluacode require("test-json") \stopluacode The name of the document is \ctxlua{tex.print(tab['documentName'])} Any tips or pointers would be much appreciated. Thank you, Ramkumar
On 11/3/2020 1:55 AM, Ramkumar KB wrote:
Hello,
I am trying to run mtxrun in a batch process and read the available documentation here - https://wiki.contextgarden.net/Running_ConTeXt_without_a_shell
My usecase is being able to create PDF docs based on the input data that comes in a JSON format (in a continuous batch process). In ConTeXt, thanks to the Lua engine, I am able to inject a JSON read from a file - and use the JSON in the tex file (here variable `tab` holds the JSON).
How do I make this as a continuous webservice - in comes JSON and out comes the PDF ?
\startluacode require("test-json") \stopluacode
The name of the document is \ctxlua{tex.print(tab['documentName'])}
Any tips or pointers would be much appreciated. foo.cld :
require("util-jsn") -- local str = io.loaddata("somefile.json") local str = [[ { "title": "Some JSON", "text" : "Just an example.", "data" : [ { "a" : "first 1", "b" : "last 1" }, { "b" : "last 2", "a" : "first 2" } ] } ]] local tmp = utilities.json.tolua(str) context.starttext() context.starttitle { title = tmp.title } context(tmp.text) context.par() context.starttabulate { "|c|c|" } for i=1,#tmp.data do local d = tmp.data[i] context.NC() context(d.a) context.NC() context(d.b) context.NC() context.NR() end context.stoptabulate() context.stoptitle() context.stoptext() and then context foo.cld ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
Hans, Thank you so much for the kind response. Apologies that I was not very clear in my initial query. I have - 1. MyStatement.tex, with all the necessary fonts, static text, images, header, footer etc 2. MyStatement.tex reads from a JSON file for certain sections of the document for the dynamic content 3. context MyStatement.tex produces MyStatement.pdf How do I do the above steps 1 to 3 in a continuous fashion such that - - Output is MyStatement_1.pdf (reads from data_1.json), MyStatement_2.pdf (reads from data_2.json) and so on - The tex processing is fast as fonts etc are loaded once (as the Tex template is same for all the MyStatement_n.pdf)
From the sample code that you gave, I get a hint that this can be possibly done using ConTeXt Lua Document approach. Is this correct?
Thank you once again,
Best,
Ramkumar
On Tue, Nov 3, 2020 at 7:02 PM Hans Hagen
On 11/3/2020 1:55 AM, Ramkumar KB wrote:
Hello,
I am trying to run mtxrun in a batch process and read the available documentation here - https://wiki.contextgarden.net/Running_ConTeXt_without_a_shell
My usecase is being able to create PDF docs based on the input data that comes in a JSON format (in a continuous batch process). In ConTeXt, thanks to the Lua engine, I am able to inject a JSON read from a file - and use the JSON in the tex file (here variable `tab` holds the JSON).
How do I make this as a continuous webservice - in comes JSON and out comes the PDF ?
\startluacode require("test-json") \stopluacode
The name of the document is \ctxlua{tex.print(tab['documentName'])}
Any tips or pointers would be much appreciated. foo.cld :
require("util-jsn")
-- local str = io.loaddata("somefile.json")
local str = [[ { "title": "Some JSON", "text" : "Just an example.", "data" : [ { "a" : "first 1", "b" : "last 1" }, { "b" : "last 2", "a" : "first 2" } ] } ]]
local tmp = utilities.json.tolua(str)
context.starttext()
context.starttitle { title = tmp.title }
context(tmp.text) context.par()
context.starttabulate { "|c|c|" } for i=1,#tmp.data do local d = tmp.data[i] context.NC() context(d.a) context.NC() context(d.b) context.NC() context.NR() end context.stoptabulate()
context.stoptitle()
context.stoptext()
and then
context foo.cld
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
On 11/4/2020 4:48 AM, Ramkumar KB wrote:
Hans,
Thank you so much for the kind response.
Apologies that I was not very clear in my initial query. I have -
1. MyStatement.tex, with all the necessary fonts, static text, images, header, footer etc 2. MyStatement.tex reads from a JSON file for certain sections of the document for the dynamic content 3. context MyStatement.tex produces MyStatement.pdf
How do I do the above steps 1 to 3 in a continuous fashion such that -
* Output is MyStatement_1.pdf (reads from data_1.json), MyStatement_2.pdf (reads from data_2.json) and so on * The tex processing is fast as fonts etc are loaded once (as the Tex template is same for all the MyStatement_n.pdf)
From the sample code that you gave, I get a hint that this can be possibly done using ConTeXt Lua Document approach. Is this correct? i'd just write a script that calls context like
context mystatement.tex --myjsonsection=1 --batch context mystatement.tex --myjsonsection=2 --batch context mystatement.tex --myjsonsection=3 --batch etc .. maybe --once if no multipass is needed, or --runs=2 if you know how many runs are needed to stay in a run and kind of restart is asking for troubles because what should be reset? of course i could context make do that buit it doesn't pay off an alternative that you generate one document with all statements and use mutool to split of the pages but anyway, nowadays machines are fast enough to have separate runs (and one can run them in parallel) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
Hans,
Thank you for the suggestions. Let me try them out and feedback back to you
(as the batch set-up involves > 10,000 docs).
Best,
Ramkumar
On Wed, Nov 4, 2020 at 7:24 PM Hans Hagen
On 11/4/2020 4:48 AM, Ramkumar KB wrote:
Hans,
Thank you so much for the kind response.
Apologies that I was not very clear in my initial query. I have -
1. MyStatement.tex, with all the necessary fonts, static text, images, header, footer etc 2. MyStatement.tex reads from a JSON file for certain sections of the document for the dynamic content 3. context MyStatement.tex produces MyStatement.pdf
How do I do the above steps 1 to 3 in a continuous fashion such that -
* Output is MyStatement_1.pdf (reads from data_1.json), MyStatement_2.pdf (reads from data_2.json) and so on * The tex processing is fast as fonts etc are loaded once (as the Tex template is same for all the MyStatement_n.pdf)
From the sample code that you gave, I get a hint that this can be possibly done using ConTeXt Lua Document approach. Is this correct? i'd just write a script that calls context like
context mystatement.tex --myjsonsection=1 --batch context mystatement.tex --myjsonsection=2 --batch context mystatement.tex --myjsonsection=3 --batch
etc .. maybe --once if no multipass is needed, or --runs=2 if you know how many runs are needed
to stay in a run and kind of restart is asking for troubles because what should be reset? of course i could context make do that buit it doesn't pay off
an alternative that you generate one document with all statements and use mutool to split of the pages
but anyway, nowadays machines are fast enough to have separate runs (and one can run them in parallel)
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl -----------------------------------------------------------------
Hi Ramkumar, Maybe you could have some scripts that check when the JSON file is modified and if so launches the compilation. I did that sort of script in order to have a continuous compilation process by checking every second if my current .tex file is modified. Best Thierry On 06/11/2020 09:38, Ramkumar KB wrote:
Hans,
Thank you for the suggestions. Let me try them out and feedback back to you (as the batch set-up involves > 10,000 docs).
Best, Ramkumar
On Wed, Nov 4, 2020 at 7:24 PM Hans Hagen
mailto:j.hagen@xs4all.nl> wrote: On 11/4/2020 4:48 AM, Ramkumar KB wrote: > Hans, > > Thank you so much for the kind response. > > Apologies that I was not very clear in my initial query. I have - > > 1. MyStatement.tex, with all the necessary fonts, static text, images, > header, footer etc > 2. MyStatement.tex reads from a JSON file for certain sections of the > document for the dynamic content > 3. context MyStatement.tex produces MyStatement.pdf > > How do I do the above steps 1 to 3 in a continuous fashion such that - > > * Output is MyStatement_1.pdf (reads from data_1.json), > MyStatement_2.pdf (reads from data_2.json) and so on > * The tex processing is fast as fonts etc are loaded once (as the Tex > template is same for all the MyStatement_n.pdf) > > From the sample code that you gave, I get a hint that this can be > possibly done using ConTeXt Lua Document approach. Is this correct? i'd just write a script that calls context like
context mystatement.tex --myjsonsection=1 --batch context mystatement.tex --myjsonsection=2 --batch context mystatement.tex --myjsonsection=3 --batch
etc .. maybe --once if no multipass is needed, or --runs=2 if you know how many runs are needed
to stay in a run and kind of restart is asking for troubles because what should be reset? of course i could context make do that buit it doesn't pay off
an alternative that you generate one document with all statements and use mutool to split of the pages
but anyway, nowadays machines are fast enough to have separate runs (and one can run them in parallel)
Hans
----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl http://www.pragma-ade.nl | www.pragma-pod.nl http://www.pragma-pod.nl -----------------------------------------------------------------
___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___________________________________________________________________________________
On Fri, 6 Nov 2020, thierry horsin wrote:
Hi Ramkumar,
Maybe you could have some scripts that check when the JSON file is modified and if so launches the compilation. I did that sort of script in order to have a continuous compilation process by checking every second if my current .tex file is modified.
Here is a script which does this for general tasks: http://users.fred.net/tds/lab/ftp/atchange It is also possible to build something on top of inotifywait on linux. Aditya
Hi,
Thank you for the tips. Yes, I will probably write something similar (but
probably in Go https://github.com/fsnotify/fsnotify or Python
https://github.com/gorakhargosh/watchdog, as they are easier to maintain
in the environments that I work).
Thank you very much!
Ramkumar
On Fri, Nov 6, 2020 at 10:52 PM Aditya Mahajan
On Fri, 6 Nov 2020, thierry horsin wrote:
Hi Ramkumar,
Maybe you could have some scripts that check when the JSON file is modified and if so launches the compilation. I did that sort of script in order to have a continuous compilation process by checking every second if my current .tex file is modified.
Here is a script which does this for general tasks:
http://users.fred.net/tds/lab/ftp/atchange
It is also possible to build something on top of inotifywait on linux.
Aditya
___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net
___________________________________________________________________________________
participants (4)
-
Aditya Mahajan
-
Hans Hagen
-
Ramkumar KB
-
thierry horsin