Its entirely possible that I'm missing something, but I was merely inquiring about running the same timing tests that you did with mplib, but instead using the existing MKII and MKIV image inclusion code. How much of a speedup over the existing code does the mplib integration buy us for this benchmark? -----Original Message----- From: ntg-context-bounces@ntg.nl on behalf of Hans Hagen Sent: Fri 3/14/2008 12:46 PM To: mailing list for ConTeXt users Subject: Re: [NTG-context] mplib Santy, Michael wrote:

Hans,

Sounds pretty impressive. As a point of reference, do you have timing data for the existing MKII and MKIV image processing code?

you mean a difference in speed? it depends on how complex the search is, but in general the mkiv code is faster and more robust there and has more potential for plugins; also mkiv does runtime conversion if needed; of course inclusion itself is not faster while (eventually) manipulation will be ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl ----------------------------------------------------------------- ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Hans Hagen wrote:

Santy, Michael wrote:

...

Its entirely possible that I'm missing something, but I was merely inquiring about running the same timing tests that you did with mplib, but instead using the existing MKII and MKIV image inclusion code. How much of a speedup over the existing code does the mplib integration buy us for this benchmark?

well, processing 15K mp graphics either runtime or delayed would take way much time (no fun timing)

IIRC, beginning of the week we had a speedup of 50% against oldfashioned mkiv (mptopdf-in-lua) with a batched Metapost run, which is alread some 25% faster than batched Metapost in mkii mode (mptopdf-in-tex). And that was before we started optimizing, we are at least twice as fast now as we were then. And in the old system, batched metapost was a whole lot faster than runtime Metapost. Take a look at it this way: if you have a document stuffed full of backgrounds and borders and decorations, you may have a dozen unique decorations per page. On a 200-page book, the total runtime used by the new MPlib-based graphics system is then still less than a second (well, if you were using Hans' machine). Best wishes, Taco

On Fri, Mar 14, 2008 at 6:51 PM, Aditya Mahajan wrote:

On Fri, 14 Mar 2008, Hans Hagen wrote:

...

so, if you have specific wishes ... now is the time to let them know

Some questions (and hidden wishes)

Does this mean that text between metapost graphic and context can be really transparent? Currently \sometxt does not work with all macros, (like an old feature request by Mojca to have two optional arguments for setting the font in the gnuplot module). Will mplib mean that the metapost code can place any tex box at the desired location?

Now that you mention it: I still need to adapt the gnuplot module to mkiv, but I'm a bit afraid of asking for help. (The most important thing would be to get rid of that stupid catcode trickery inside \startGNUPLOTscript ... \stopGNUPLOTscript that always tends to fail when one needs it, and to have one-to-one conversion, so that %,#, $, ... will have no special meaning.) Then, it would probably be a nice benchmark to discover bugs in the library :) For example, demos from this page (I'm not saying that they're too useful since 100x100 is too low resolution anyway): http://gnuplot.sourceforge.net/demo_4.3/pm3d.html currently all fail because of the "out-of-memory" problem (10.000 times "fill unitsquare scaled ... shifted ... withcolor ..."), but this might be a ConTeXt problem as well (the text inside \startMPgraphic ... \stopMPgraphic probably needs to be stored somewhere in TeX, even if there's no need for that). Mojca

On Sat, Mar 15, 2008 at 11:18 AM, Hans Hagen wrote:

Mojca Miklavec wrote:

...

On Fri, Mar 14, 2008 at 6:51 PM, Aditya Mahajan wrote:

...

On Fri, 14 Mar 2008, Hans Hagen wrote:

...

so, if you have specific wishes ... now is the time to let them know

Some questions (and hidden wishes)

Does this mean that text between metapost graphic and context can be really transparent? Currently \sometxt does not work with all macros, (like an old feature request by Mojca to have two optional arguments for setting the font in the gnuplot module). Will mplib mean that the metapost code can place any tex box at the desired location?

Now that you mention it: I still need to adapt the gnuplot module to mkiv, but I'm a bit afraid of asking for help. (The most important thing would be to get rid of that stupid catcode trickery inside \startGNUPLOTscript ... \stopGNUPLOTscript that always tends to fail when one needs it, and to have one-to-one conversion, so that %,#, $, .... will have no special meaning.) Then, it would probably be a nice benchmark to discover bugs in the library :)

For example, demos from this page (I'm not saying that they're too useful since 100x100 is too low resolution anyway): http://gnuplot.sourceforge.net/demo_4.3/pm3d.html currently all fail because of the "out-of-memory" problem (10.000 times "fill unitsquare scaled ... shifted ... withcolor ..."), but this might be a ConTeXt problem as well (the text inside \startMPgraphic ... \stopMPgraphic probably needs to be stored somewhere in TeX, even if there's no need for that).

let's discuss this a few months from now when i have more code in place

btw, how difficult would it be to add a lua api to gnuplot, in which case we could communicate via lua tables

Peter Hedwig has already done some work on it: http://peter.affenbande.org/gnuplot/ What his patch does: one calls gnuplot normally, and then, when you say set term lua dashed fontsize 15 "dashed fontsize 15" is parsed by lua. Then, gnuplot does its work, and calls functions: set_linestyle(3) set_color(...) draw_line(...) What Peter did was a mapping from C to lua, so the actual code that does "draw_line" is actually implemented in lua and can easily be changed and/or adapted without recompiling gnuplot. He now "converts" those function calls into TikZ Code, but that could easily be adapted into anything. So: output has already been implemented (and would "only" need to be adapted), while input might need some tweaking. I'm sometimes using a rather simple C++ library which opens gnuplot and pipes "set terminal ...", "plot ..." commands to gnuplot, so one can pass a C table to gnuplot. I could take a look at how it's done there. How do you imagine passing lua tables to gnuplot? Mojca PS: no need to say that even though his terminal is much more popular (it's the only decent LaTeX terminal I know) than the one for ConTeXt, I have seen zero discussion on the gnuplot mailing list about any intentions to integrate that code into gnuplot core.

On Sun, Mar 16, 2008 at 10:58 AM, zs wrote:

On Sat, 15 Mar 2008 11:18:33 +0100 Hans Hagen wrote:

...

btw, how difficult would it be to add a lua api to gnuplot, in which case we could communicate via lua tables

Hans

btw: Some of you may not know about cairo library. (http://cairographics.org/) It's versatile graphic library with many output targets(pdf,svg,png,...) and bindings to different programming languages (including lua!). It can do almost the same things that metapost do. Development version of gnulpot should be able to use cairo library for rendering too.

Cairo (with pango, that needs lots of tweaking to be able to build it, I do not even dare dreaming about being able to build it on windows) is able to generate PDFs in gnuplot, but not to input any TeX labels yet, I'm affraid. Mojca