Using external libraries in LuaTeX
Dear All In a the recent Luatex 0.46.0 announcement http://tug.org/pipermail/luatex/2009-November/001118.html experimental support for loading external libraries was reported. I am running Windows Vista with ConTeXt ver: 2010.03.12 19:49 MKIV fmt: 2010.3.16 int: english/english with LuaTeX, Version beta-0.51.0-2010031309 (rev 3479) I have also downloaded Lua 5.1 from http://luaforwindows.luaforge.net/ and so have Lua5.1.dll installed. This distribution has a number of interesting extenal libraries (ie DLLs) that I would like to try out with LuaTeX. If I can make progress I would be happy to document and write-up my experiments for posting on the Wiki in case they may be of interest to others. I should say that my coding skills are somewhat rusty as I've been away from programming/TeX for a few years... so do please excuse my "simplistic" questions :-) So, I was wondering if anyone has a minimal example they would be kind enough to share, or can give me a few pointers/guidelines so that I can start experimenting. In particular, I'm interested to call external libraries and inject the output from an external library into TeX for typesetting. For example, is it possible to load a library once, keep it loaded and access the library functions from different commands during the TeX run? In outline... --- run some code to load the library (initialisation) --- keeping the library available --- some commands\definitions to accss library functions eg \calltolibrary{....) --- make callsto library and pass the output into TeX for typesetting I would be very grateful for any guidance/help that can be offered. Warm wishes to all Graham
On Thu, Mar 18, 2010 at 1:10 AM, Graham Douglas
Dear All
In a the recent Luatex 0.46.0 announcement
http://tug.org/pipermail/luatex/2009-November/001118.html
experimental support for loading external libraries was reported.
I am running Windows Vista with
ConTeXt ver: 2010.03.12 19:49 MKIV fmt: 2010.3.16 int: english/english with LuaTeX, Version beta-0.51.0-2010031309 (rev 3479)
I have also downloaded Lua 5.1 from http://luaforwindows.luaforge.net/ and so have Lua5.1.dll installed.
This distribution has a number of interesting extenal libraries (ie DLLs) that I would like to try out with LuaTeX. If I can make progress I would be happy to document and write-up my experiments for posting on the Wiki in case they may be of interest to others. I should say that my coding skills are somewhat rusty as I've been away from programming/TeX for a few years... so do please excuse my "simplistic" questions :-)
So, I was wondering if anyone has a minimal example they would be kind enough to share, or can give me a few pointers/guidelines so that I can start experimenting. In particular, I'm interested to call external libraries and inject the output from an external library into TeX for typesetting. For example, is it possible to load a library once, keep it loaded and access the library functions from different commands during the TeX run?
In outline...
--- run some code to load the library (initialisation) --- keeping the library available --- some commands\definitions to accss library functions eg \calltolibrary{....)
--- make callsto library and pass the output into TeX for typesetting
I would be very grateful for any guidance/help that can be offered.
http://www.ntg.nl/EuroTeX2009/slides/luigi-slides.pdf -- luigi
Hi Luigi That is very cool work. At present, it is a bit beyond my fuzzy skills, but I will dfinitely keep a copy of your slides for future reference as I slowly.... get a bit more familiar with LuaTeX etc. Thanks for sharing that. Warm wishes Graham
On Thu, Mar 18, 2010 at 11:29 PM, Graham Douglas
Hi Luigi
That is very cool work. At present, it is a bit beyond my fuzzy skills, but I will dfinitely keep a copy of your slides for future reference as I slowly.... get a bit more familiar with LuaTeX etc.
Thanks for sharing that.
We have 2 ways to interface with an external program 1) \write18 . In this way (lua/pdf)tex runs the program using the OS, ie something like os.cmd, . and a completely new task is created. We have a clear separation between luatex process an program process and usually if the program process aborts luatex can manage this situation nicely. It's robust but it's also slow, as the case for mkii with metapost shown; 2) dynamic loading of a library, a *so or a *dll. In this case there is not another process to start: luatex is extended with this new library at runtime. So, if a the library aborts while you are calling a function, then luatex aborts --- and you can't manage this situation. Now, if your library declare a symbol that is also defined by luatex or if you library has a function that load another library at runtime and this one has a symbol that is also defined by luatex , then you have a symbol collision and hence with high probability a segfault . Trace the symbol collision can be very hard, so actually it's less robust than 1) (remember that *tex must be portable to many SO, so robustness is really important). Of course there is a benefict: the gain in speed can be 1000 (one thousand times) and if I remember well, Hans reported 10000 with mplib (which is included in luatex at compilation time ). -- luigi
<snip> Hi Luigi Great stuff, thank you. If you ever had time to write an article which really explained the underlying programming concepts you demonstrated in your slides I am sure you would find a lot of people interested to read it. I think the ability to extend LuaTeX via Lua libraries really is a powerful feature. With LuaTeX there is so much going on, so many fascinating possibilities to explore, that it needs people like you, and other experts on this list, to "open the doors" to the rest of us, by showing the "big picture" and how it all really fits together. Leaving Taco + Hans to keep writing the goodies :-) But, of course, articles etc take much time to write and we all have busy day jobs etc. Once again, many thanks. Graham
On Fri, Mar 19, 2010 at 7:29 PM, Graham Douglas
<snip>
Hi Luigi
Great stuff, thank you. If you ever had time to write an article which really explained the underlying programming concepts you demonstrated in your slides I am sure you would find a lot of people interested to read it. I think the ability to extend LuaTeX via Lua libraries really is a powerful feature. With LuaTeX there is so much going on, so many fascinating possibilities to explore, that it needs people like you, and other experts on this list, to "open the doors" to the rest of us, by showing the "big picture" and how it all really fits together. Leaving Taco + Hans to keep writing the goodies :-)
But, of course, articles etc take much time to write and we all have busy day jobs etc. Well if you are really interested then you should join ntg + tug now. My article --- as many others about luatex & mkiv --- is in the last number of TUGBOAT
-- luigi
Hi Luigi "Well if you are really interested then you should join ntg + tug now. My article --- as many others about luatex & mkiv --- is in the last number of TUGBOAT" -- already made TUG membership application, plus made a donation to the LuaTeX fund. Warm regards Graham
Hi, Graham Douglas wrote:
So, I was wondering if anyone has a minimal example they would be kind enough to share, or can give me a few pointers/guidelines so that I can start experimenting. In particular, I'm interested to call external libraries and inject the output from an external library into TeX for typesetting. For example, is it possible to load a library once, keep it loaded and access the library functions from different commands during the TeX run?
Here is a trivial example using LuaExpat: \starttext \startluacode require "lxp"; text = '<p>hello world</p>' callbacks = { EndElement = false, StartElement = false, Default = function (p, s) tex.sprint(s) end } p = lxp.new(callbacks) \stopluacode Some xml output: \directlua{p:parse(text)}!. \directlua{p:close()} \stoptext Disclosure: for this test, I copied lxp.so into the local directory because I am not sure exactly how context mkiv searches for Lua cpath files. Best wishes, Taco
Hi, I have tested all samples in the arxive to use mathdesign fonts for math but no success has been reached. Any comment is wellcome.
Am 18.03.10 08:12, schrieb Mehdi Omidali:
Hi, I have tested all samples in the arxive to use mathdesign fonts for math but no success has been reached. Files: http://drp.ly/CC5aF
Unzip the archive in one of your local TeX folders. Sample: http://drp.ly/CC86L Result: http://drp.ly/CC92k Replace the font defintions in math-vfu.lua with this and regenerate the format. mathematics.make_font ( "charter-math", { -- { name = "file:bchr8a", features = "virtualmath", main = true }, { name = "file:bchri8a", features = "virtualmath", main = true }, -- { name = "md-chr7m.tfm", vector = "tex-mr" }, { name = "md-chri7m.tfm", vector = "tex-mi", skewchar=0x7F }, { name = "md-chri7m.tfm", vector = "tex-it", skewchar=0x7F }, { name = "md-chr7y.tfm", vector = "tex-sy", skewchar=0x30, parameters = true }, { name = "md-chr7v.tfm", vector = "tex-ex", extension = true }, -- { name = "md-chbma.tfm", vector = "tex-ma" }, -- { name = "md-chbmb.tfm", vector = "tex-mb" }, { name = "msam10.tfm", vector = "tex-ma" }, { name = "msbm10.tfm", vector = "tex-mb" }, } ) mathematics.make_font ( "garamond-math", { -- { name = "file:ugmr8a", features = "virtualmath", main = true }, { name = "file:ugmri8a", features = "virtualmath", main = true }, -- { name = "md-gmr7m.tfm", vector = "tex-mr" }, { name = "md-gmri7m.tfm", vector = "tex-mi", skewchar=0x7F }, { name = "md-gmri7m.tfm", vector = "tex-it", skewchar=0x7F }, { name = "md-gmr7y.tfm", vector = "tex-sy", skewchar=0x30, parameters = true }, { name = "md-gmr7v.tfm", vector = "tex-ex", extension = true }, -- { name = "md-gmmma.tfm", vector = "tex-ma" }, -- { name = "md-gmmmb.tfm", vector = "tex-mb" }, { name = "msam10.tfm", vector = "tex-ma" }, { name = "msbm10.tfm", vector = "tex-mb" }, } ) mathematics.make_font ( "utopia-math", { -- { name = "file:putr8a", features = "virtualmath", main = true }, { name = "file:putri8a", features = "virtualmath", main = true }, -- { name = "md-utr7m.tfm", vector = "tex-mr" }, { name = "md-utri7m.tfm", vector = "tex-mi", skewchar=0x7F }, { name = "md-utri7m.tfm", vector = "tex-it", skewchar=0x7F }, { name = "md-utr7y.tfm", vector = "tex-sy", skewchar=0x30, parameters = true }, { name = "md-utr7v.tfm", vector = "tex-ex", extension = true }, -- { name = "md-utbma.tfm", vector = "tex-ma" }, -- { name = "md-utbmb.tfm", vector = "tex-mb" }, { name = "msam10.tfm", vector = "tex-ma" }, { name = "msbm10.tfm", vector = "tex-mb" }, } ) Wolfgang
On Thu, Mar 18, 2010 at 10:40 AM, Wolfgang Schuster
Am 18.03.10 08:12, schrieb Mehdi Omidali:
Hi, I have tested all samples in the arxive to use mathdesign fonts for math but no success has been reached.
Files: http://drp.ly/CC5aF
Unzip the archive in one of your local TeX folders.
Sample: http://drp.ly/CC86L Result: http://drp.ly/CC92k
Replace the font defintions in math-vfu.lua with this and regenerate the format.
mathematics.make_font ( "charter-math", { -- { name = "file:bchr8a", features = "virtualmath", main = true }, { name = "file:bchri8a", features = "virtualmath", main = true }, -- { name = "md-chr7m.tfm", vector = "tex-mr" }, { name = "md-chri7m.tfm", vector = "tex-mi", skewchar=0x7F }, { name = "md-chri7m.tfm", vector = "tex-it", skewchar=0x7F }, { name = "md-chr7y.tfm", vector = "tex-sy", skewchar=0x30, parameters = true }, { name = "md-chr7v.tfm", vector = "tex-ex", extension = true }, -- { name = "md-chbma.tfm", vector = "tex-ma" }, -- { name = "md-chbmb.tfm", vector = "tex-mb" }, { name = "msam10.tfm", vector = "tex-ma" }, { name = "msbm10.tfm", vector = "tex-mb" }, } )
mathematics.make_font ( "garamond-math", { -- { name = "file:ugmr8a", features = "virtualmath", main = true }, { name = "file:ugmri8a", features = "virtualmath", main = true }, -- { name = "md-gmr7m.tfm", vector = "tex-mr" }, { name = "md-gmri7m.tfm", vector = "tex-mi", skewchar=0x7F }, { name = "md-gmri7m.tfm", vector = "tex-it", skewchar=0x7F }, { name = "md-gmr7y.tfm", vector = "tex-sy", skewchar=0x30, parameters = true }, { name = "md-gmr7v.tfm", vector = "tex-ex", extension = true }, -- { name = "md-gmmma.tfm", vector = "tex-ma" }, -- { name = "md-gmmmb.tfm", vector = "tex-mb" }, { name = "msam10.tfm", vector = "tex-ma" }, { name = "msbm10.tfm", vector = "tex-mb" }, } )
mathematics.make_font ( "utopia-math", { -- { name = "file:putr8a", features = "virtualmath", main = true }, { name = "file:putri8a", features = "virtualmath", main = true }, -- { name = "md-utr7m.tfm", vector = "tex-mr" }, { name = "md-utri7m.tfm", vector = "tex-mi", skewchar=0x7F }, { name = "md-utri7m.tfm", vector = "tex-it", skewchar=0x7F }, { name = "md-utr7y.tfm", vector = "tex-sy", skewchar=0x30, parameters = true }, { name = "md-utr7v.tfm", vector = "tex-ex", extension = true }, -- { name = "md-utbma.tfm", vector = "tex-ma" }, -- { name = "md-utbmb.tfm", vector = "tex-mb" }, { name = "msam10.tfm", vector = "tex-ma" }, { name = "msbm10.tfm", vector = "tex-mb" }, } )
Wolfgang
Wow, Wolfgang, this is very nice! (I was fighting his some weeks ago, and what I probably did wrong was that I did not regenerate the file name database. Now the math is indeed italic and not upright.) It is, however, not completely perfect. With the following example, I get the attached result. I think the prime is too close to the f and also, sin should be "upright" and not italic. Any ideas? % mdprim.tex \definetypeface[charter][mm][math] [charter] [default] \definetypeface[charter][rm][serif][charter] [default] \setupbodyfont[charter,10pt] \starttext $f'(x)=\sin(x)$. \stoptext % end of mdprim.tex Best regards, Mikael PS I hope it is OK to attach a 10kB pdf file
Hi Mikael, Mikael Persson wrote:
It is, however, not completely perfect. With the following example, I get the attached result. I think the prime is too close to the f and
This problem could be related to a luatex bug : http://tracker.luatex.org/view.php?id=345 First version with that fix was 0.52.0 Best wishes, Taco
Am 18.03.10 11:10, schrieb Taco Hoekwater:
Mikael Persson wrote:
I think the prime is too close to the f and
This problem could be related to a luatex bug :
This is LuaTeX, Version beta-0.53.0-2010031812 (rev 3520) Unrelated to the bug, the bad kerning remains. Wolfgang
On Thu, Mar 18, 2010 at 12:27 PM, Wolfgang Schuster
Am 18.03.10 11:10, schrieb Taco Hoekwater:
Mikael Persson wrote:
I think the prime is too close to the f and
This problem could be related to a luatex bug :
This is LuaTeX, Version beta-0.53.0-2010031812 (rev 3520)
Unrelated to the bug, the bad kerning remains.
Wolfgang
I have now compiled a larger article and so let me update my example a bit: % mdprim.tex \definetypeface[charter][mm][math] [charter][default][text=rm] \definetypeface[charter][rm][serif][charter][default] \setupbodyfont[charter,10pt] \starttext $f'(x)=\sin(x)$, $\partial x$, $\alpha x_2^2$. Define $A\colon \mathbb{R}\to\mathbb{R}$ as\dots. \stoptext % end of mdprim.tex The output is attached. In my opinion the spacing between \partial and x is too small, and the spacing in the x_2^2 does not look right either, but I don't really know what is the problem. Maybe, on a second look the 2s are too big? I'd also expected that the \colon gives a colon (with some space), but maybe that is a ams(la)tex macro I have do define on my own? (As it is now it does not complain about the \colon, but it also gives no colon). Best regards, Mikael PS: The kerning for mathdesign in LaTeX is OK. But that is probably, as someone wrote, due to vf files(?)
Wolfgang Schuster wrote:
Am 18.03.10 11:10, schrieb Taco Hoekwater:
Mikael Persson wrote:
I think the prime is too close to the f and
This problem could be related to a luatex bug :
This is LuaTeX, Version beta-0.53.0-2010031812 (rev 3520)
Unrelated to the bug, the bad kerning remains.
In that case, I don't quite understand either.
On Thu, 18 Mar 2010, Mikael Persson wrote:
Wow, Wolfgang, this is very nice! (I was fighting his some weeks ago, and what I probably did wrong was that I did not regenerate the file name database. Now the math is indeed italic and not upright.)
It is, however, not completely perfect. With the following example, I get the attached result. I think the prime is too close to the f and
I don't know. Do they come out correctly with latex? If so, it may be a bug in luatex's italic correction.
also, sin should be "upright" and not italic. Any ideas?
Try (Untested) \definetypeface[charter][mm][math] [charter] [default] [text=rm] Aditya
It is, however, not completely perfect. With the following example, I get the attached result. I think the prime is too close to the f and I don't know. Do they come out correctly with latex? If so, it may be a bug in luatex's italic correction. LaTeX use virtual fonts and we use the tfm files only. also, sin should be "upright" and not italic. Any ideas? Try (Untested)
\definetypeface[charter][mm][math] [charter] [default] [text=rm] The problem is the fonts do not contain roman letters and I use the italic
Am 18.03.10 11:14, schrieb Aditya Mahajan: style or the Charter fonts. { name = "file:bchri8a", features = "virtualmath", main = true }, LaTeX solves this with virtual fonts but we have only tfm’s. Wolfgang
On Thu, Mar 18, 2010 at 11:14 AM, Aditya Mahajan
On Thu, 18 Mar 2010, Mikael Persson wrote:
Wow, Wolfgang, this is very nice! (I was fighting his some weeks ago, and what I probably did wrong was that I did not regenerate the file name database. Now the math is indeed italic and not upright.)
It is, however, not completely perfect. With the following example, I get the attached result. I think the prime is too close to the f and
I don't know. Do they come out correctly with latex? If so, it may be a bug in luatex's italic correction.
also, sin should be "upright" and not italic. Any ideas?
Try (Untested)
\definetypeface[charter][mm][math] [charter] [default] [text=rm]
Aditya
Thanks Aditya! The sin problem goes away with this! Then, there is only the prime problem left (as far as I can see. I should try this setup on a larger paper to see if something else looks strange) Best regards, Mikael
On 3/18/2010 2:04 PM, Mikael Persson wrote:
On Thu, Mar 18, 2010 at 11:14 AM, Aditya Mahajan
wrote: On Thu, 18 Mar 2010, Mikael Persson wrote:
Wow, Wolfgang, this is very nice! (I was fighting his some weeks ago, and what I probably did wrong was that I did not regenerate the file name database. Now the math is indeed italic and not upright.)
It is, however, not completely perfect. With the following example, I get the attached result. I think the prime is too close to the f and
I don't know. Do they come out correctly with latex? If so, it may be a bug in luatex's italic correction.
also, sin should be "upright" and not italic. Any ideas?
Try (Untested)
\definetypeface[charter][mm][math] [charter] [default] [text=rm]
Aditya
Thanks Aditya! The sin problem goes away with this!
Then, there is only the prime problem left (as far as I can see. I should try this setup on a larger paper to see if something else looks strange)
Best regards, Mikael ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : http://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________
Thank you all. Still there is a problem with delimiters (which should be upright) and \mfunction. Please see the attached file. Mehdi
<<snip>> Taco wrote: ============= Here is a trivial example using LuaExpat: \starttext \startluacode require "lxp"; text = '<p>hello world</p>' callbacks = { EndElement = false, StartElement = false, Default = function (p, s) tex.sprint(s) end } p = lxp.new(callbacks) \stopluacode Some xml output: \directlua{p:parse(text)}!. \directlua{p:close()} \stoptext ============= Wonderful! Thanks Taco. Just for the record, for other Windows users, to get this working I downloaded luaexpat-1.1.win32-lua51.zip from http://luaforge.net/projects/luaexpat/files and from that I copied the following DLLs (from luaexpat-1.1.win32-lua51.zip) libexpat.dll lxp.dll into the following directory on my PC C:\Context\tex\texmf-mswin\bin\lib\lua (adapt to your ConText install) One question, in the minimal example below, the XML text: text = '<elem1> text <elem2/> more text </elem1>' parses fine *if* on a single line. If I break the text onto multiple lines, eg text = '<elem1> text <elem2/> more text </elem1>' it fails with ! LuaTeX error <main ctx instance>:3: unfinished string near ''<elem1> text <elem2/>'. } = lxp.new(callbacks)d \dodostartluacode ...d \directlua \zerocount {#1}} l.21 \stopluacode Why is this --- is this a catcode issue? Can you suggest a fix? Once again, thank you, as always, for sharing your expertise. Warm wishes Graham =============== Minimal Example ================ \startluacode require "lxp"; text = '<elem1> text <elem2/> more text </elem1>' callbacks = { StartElement = function (p, s) tex.sprint("Starting element: " ..s) end, EndElement = function (p, s) tex.sprint("Finishing element: " ..s) end, CharacterData = function (p, s) tex.sprint(s) end } p = lxp.new(callbacks) \stopluacode
Graham Douglas wrote:
One question, in the minimal example below, the XML text:
text = '<elem1> text <elem2/> more text </elem1>'
parses fine *if* on a single line. If I break the text onto multiple lines, eg
text = '<elem1> text <elem2/> more text </elem1>'
it fails with
! LuaTeX error <main ctx instance>:3: unfinished string near ''<elem1> text <elem2/>'. } = lxp.new(callbacks)d
\dodostartluacode ...d \directlua \zerocount {#1}}
l.21 \stopluacode
Why is this --- is this a catcode issue?
No, not at all. Lua supports only single-line '' or "" strings.
Can you suggest a fix?
Multi-line strings in Lua use square brackets text = [[<elem1> text <elem2/> more text </elem1>]] See http://www.lua.org/manual/5.1/manual.html#2.1 for details. Best wishes, Taco
Hello, I like to use Latin Modern Math as my math font but it lacks some glyphs like \cong=0x02245. I would like to know (if possible, with a working example) if font fallbacks feature is available for math fonts. Mehdi
participants (7)
-
Aditya Mahajan
-
Graham Douglas
-
luigi scarso
-
Mehdi Omidali
-
Mikael Persson
-
Taco Hoekwater
-
Wolfgang Schuster