Hi This week there's been a nice discussion on comp.text.tex that started in the context of LuaTeX. It was on the difference between embedding a scripting language in an application and extending a scripting language with a module written in a compiled language. One of the best parts, in my view, was Zeljko Vrba pointing out, in effect, that Postgres both embedded scripting languages and provided an interface that allowed it to be used as an extension to a scripting language. This provides a wonderful dual example. One of the participants asked: "What was the rationale behind choosing the embed route by the LuaTeX team?" We'd be delighted if posted an answer to this question to comp.text.tex. To help you find the c.t.t discussion, here's a URL: http://groups.google.co.uk/group/comp.text.tex/tree/browse_frm/thread/d352212826544c1d/fdf9f61e693cac27?rnum=111&_done=%2Fgroup%2Fcomp.text.tex%2Fbrowse_frm%2Fthread%2Fd352212826544c1d%2F9900cd1dca621b18%3F#doc_2f7883ecc8e560fb And here's a shorter one: http://groups.google.co.uk/group/comp.text.tex/search?hl=en&group=comp.text.tex&q=embed+or+extend&qt_g=Search+this+group with best regards Jonathan
Jonathan Fine wrote:
One of the participants asked: "What was the rationale behind choosing the embed route by the LuaTeX team?"
the motives for doing ot the way we do has been discussed in articles and presentations several times the last couple of years so it makes no sense to repeat that again the reasons for choosing lua are (1) we like it, (2) it's small and stable, (3) it does not add updating, huge libraries and depencies mess to tex distributions, (4) it's fast, (5) it's made for this purpose anyhow, we consider tex to be the core application that we wanted to extend, and as lua is perfect as extensions language, we follow this route; the starting point is the user base and users use macro packages; users don't want to implement font, language, typesetting, math, whatever subsystems themselves maybe > 2012 we've reached a stage where one can isolate the tex component but in itself it's rather useless as it needs all these extra subsystems; taking just the par builder and making that a library component is quite doable but useless in perspective of tex usage; also, the tex community has a large code base, distributions etc etc and it made perfect sense to stay in that domain (i.e. keep macro language, be compatible, provide support for existing stuff, etc) so, in practice we're not talking of tex, but of the whole infrastructure around tex; also, we do it the way we like most (which is a pretty valid argument too) for other motives, discussions etc etc i refer to all things (to be) presented, discussed, published Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hans Hagen wrote:
Jonathan Fine wrote:
One of the participants asked: "What was the rationale behind choosing the embed route by the LuaTeX team?"
the motives for doing ot the way we do has been discussed in articles and presentations several times the last couple of years so it makes no sense to repeat that again
I have answered nonetheless in c.t.t (just like you did here)
Hello Hans Thank you for your prompt reply.
One of the participants asked: "What was the rationale behind choosing the embed route by the LuaTeX team?"
the motives for doing ot the way we do has been discussed in articles and presentations several times the last couple of years so it makes no sense to repeat that again
I'm sorry, Hans, but I've not been able to find the resources you refer to. I've done an internet search http://www.google.co.uk/search?q=embed+extend+luatex and I've not found anything yet. Please would you supply URLs explain why the LuaTeX team chose to embed rather than extend.
the reasons for choosing lua are (1) we like it, (2) it's small and stable, (3) it does not add updating, huge libraries and depencies mess to tex distributions, (4) it's fast, (5) it's made for this purpose
Ah, you may have misunderstood me. Perhaps I was not clear enough. My question is not why did you choose Lua, but why did you choose to embed rather than to extend. [snip] -- Jonathan
Jonathan Fine wrote:
Please would you supply URLs explain why the LuaTeX team chose to embed rather than extend.
tugboat, maps, gust magazine, dtk, context mailing list/meetings, etc; i have no clue what is online or not (nor time to figure it out) we extend tex by embedding lua so i don't see the point unless you mean that we'd extend lua with tex which never has been an issue at all as mentioned in the prev mail .. the starting point is simply tex and extending tex as tex macropackages is what users use and we float on the waves of the tex cummunity and distributions also, i don't want in retrospect to give the impression that we had (10 year lasting and thereby ending up into nothing) endless discussions; just to mention one argument that i presenbted in talks and articles right from the start: i ran into lua when i played with adding fuctionality to scite (editor), then after some mail exchanges hartmut hacked something similar into tex and we played with it for a while, then taco, hartmut and i decided to set the next steps and open op more and more of tex to lua (the extension language) and luatex was born .. we liked it so much that we went that route and never regret it (as it fits nicely into regular tex / macro writing as well as puts no burden on distributions and users) there is not much more to tell and the shortest answer possible really is: we found this quite natural and liked it; of course i can cook up arguments afterwards and add philosophical or pseudo scientific motives; we keep a history of the developments (mostly published) where decisions etc etc are argumented so that's the definite reference Hans (of course after the first experiments and along the route we saw more possibilities and got more ideas about the future but that's irrelevant here) ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On Sun, May 24, 2009 at 03:03:01PM +0100, Jonathan Fine wrote:
the reasons for choosing lua are (1) we like it, (2) it's small and stable, (3) it does not add updating, huge libraries and depencies mess to tex distributions, (4) it's fast, (5) it's made for this purpose
Ah, you may have misunderstood me. Perhaps I was not clear enough. My question is not why did you choose Lua, but why did you choose to embed rather than to extend.
I, personally, don't see the point of such question; it is clear that LuaTeX team, to whom I've no affiliation, are trying to extend and improve TeX and for some reasons they found extending it with a scripting language is nice idea and went with it. I fail to understand why they would be extending Lua by a typesetting engine when they are actually interested in the engine not Lua. Sure having TeX as a service is a neat idea and would make TeX available to many uses that is hard to be done now, but I can clearly see that this never have been one of the stated goals of LuaTeX. Regards, Khaled -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer
One of the best parts, in my view, was Zeljko Vrba pointing out, in effect, that Postgres both embedded scripting languages and provided an interface that allowed it to be used as an extension to a scripting language. This provides a wonderful dual example.
Taco and Hans already stated their opinions, and I won't speak on their behalf, but I read the entire thread on c.t.t. and would like to point out that in the discussion, Željko seems to say the exact contrary to what you report here: Postgres embeds Python as a scripting language -- just like LuaTeX embeds Lua -- and does not offer the database engine as an extension: it only gives access to it through an API, and does not extend Python with a database module. I'm quite sure there would be ways to do the same with LuaTeX (provided one knows what sort of result one expects from it exactly), and David Kastrup even described possible solutions, which Željko mentioned in his very same post. You did not care to comment about it. Arthur
Arthur Reutenauer wrote:
One of the best parts, in my view, was Zeljko Vrba pointing out, in effect, that Postgres both embedded scripting languages and provided an interface that allowed it to be used as an extension to a scripting language. This provides a wonderful dual example.
Taco and Hans already stated their opinions, and I won't speak on their behalf, but I read the entire thread on c.t.t. and would like to point out that in the discussion, Željko seems to say the exact contrary to what you report here: Postgres embeds Python as a scripting language -- just like LuaTeX embeds Lua --
Thank you for reading the c.t.t thread, Arthur, and for your comments. Željko pointed out that Postgres allows Python to be embedded as a scripting language. It is well-known that there are Python wrappers that allow one to extend Python with the Postgres database engine. Put the two together and you have the dual example. I don't think I was being too generous in giving Željko credit for this wonderful dual example. He certainly provided the part that was not obvious.
and [Postgres] does not offer the database engine as an extension: it only gives access to it through an API, and does not extend Python with a database module.
In Python I can write import pyscopg and then I have a module that allows me to connect to a Postgres database, and add, change and delete records. In other words I have imported into Python a database extension. I'd expect almost every LAMP web-site to do something like this. Not always Python, not always Postgres, but extend a scripting language to provide database access. This database will be accessed, of course, through its API. (How else would a sensible programmer do it?) The extension module pyscopg provides a wrapper around the API that makes it accessible to Python. As it happens, and for good reasons, pyscopg is not provided by Postgres but by a third party. So technically you are correct. Postgres does not provide a Python extension module, only an API. A third party wraps the API to provide a Python extension module. But all this is a little off the topic of embed and extend. Postgres can embed Python, and one can extend Python using Postgres. This is the dual example, and it helps show us what embed is good for, and what extend is good for. -- Jonathan
Jonathan Fine wrote:
In Python I can write import pyscopg and then I have a module that allows me to connect to a Postgres database, and add, change and delete records. In other words I have imported into Python a database extension.
connect ... so the database itself runs indepently (and has a comunication channel to a certain amount of clients)
But all this is a little off the topic of embed and extend. Postgres can embed Python, and one can extend Python using Postgres. This is the dual example, and it helps show us what embed is good for, and what extend is good for.
so, in your example its not even embedded or extended that we're talking of but connecting; well, it is no secret that luatex provides sockets so you can start a luatex session, connect to it, send some text, and get back whatever you want, in the sense that you can serialize for instance a node list, (which in itself is not that useful as it lacks resources) anyhow, this is exactly what arthur also mentioned so ... [end of thread] ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Hans Hagen wrote:
In Python I can write import pyscopg and then I have a module that allows me to connect to a Postgres database, and add, change and delete records. In other words I have imported into Python a database extension.
connect ... so the database itself runs indepently (and has a comunication channel to a certain amount of clients)
Yes, Hans. Pyscopg is a Python extension (probably written in C) that can be imported as a module and which allows a Python script to connect to a database (via a database server, of course).
But all this is a little off the topic of embed and extend. Postgres can embed Python, and one can extend Python using Postgres. This is the dual example, and it helps show us what embed is good for, and what extend is good for.
so, in your example its not even embedded or extended that we're talking of but connecting;
No, Hans, and my apologies for not being sufficiently clear in my previous post. 1. Pyscopg is a Python extension. 2. Postgres has an API that allows it to be used as a service. 3. Pysocpg is a Python extension that can connect to a Postgres database. So it is 'extending Python (with a connection to a database)'.
well, it is no secret that luatex provides sockets so you can start a luatex session, connect to it, send some text, and get back whatever you want, in the sense that you can serialize for instance a node list, (which in itself is not that useful as it lacks resources)
Yes, I agree. One can write Lua code that would allow LuaTeX to function as a server, and some additional work would be required to make that service useful. Or, because the C-libraries (used also by Lua) support sockets, one could write an extension of TeX that functions as a server. Each approach has its advantages and problems. To return the dual-example provided by Postgres. I could, as you suggest, write Lua code that allows LuaTeX to provide typesetting as a service, via a socket interface. Here, I would be using Lua as an embedded language in the typesetting program. It is akin to using a "stored procedure" in a database program. I could then wrap the API provided by LuaTeX (plus the code just mentioned) in a module that could be imported, say, by Ruby. This would be an example of extending a dynamic language (with a connection to a typesetting engine). -- Jonathan
participants (5)
-
Arthur Reutenauer -
Hans Hagen -
Jonathan Fine -
Khaled Hosny -
Taco Hoekwater