Hi, there is a new project on supelec: Luajittex. Briefly, it's like luatex with Lua replaced by LuaJIT. See http://foundry.supelec.fr/gf/project/luajittex/ for the source code. Please note that LuaJIT is based on lua 5.1, while luatex is now based on lua 5.2 --- this means that binaries modules must be kept separate. The tag 0.72.0 means that luajittex is in sinc with the tag 0.72.0 of luatex; I'm working to keep in sinc with the last tag. I've compiled the source for Linux 64bit (Ubuntu 12.10) Linux 32bit (Opensuse 12.1) and crosscompiled for Mingw32bit and they seem to work, but of course any feedback is appreciate. -- luigi
Hi Luigi,
I've compiled the source for Linux 64bit (Ubuntu 12.10) Linux 32bit (Opensuse 12.1) and crosscompiled for Mingw32bit and they seem to work, but of course any feedback is appreciate.
compilation worked fine, but after that I've got (in my publisher) Can't create the Lua state. I'll try to find out where this happens. Patrick (sounds very promising!!!)
On Tue, Dec 25, 2012 at 9:02 PM, Patrick Gundlach
Hi Luigi,
I've compiled the source for Linux 64bit (Ubuntu 12.10) Linux 32bit (Opensuse 12.1) and crosscompiled for Mingw32bit and they seem to work, but of course any feedback is appreciate.
compilation worked fine, but after that I've got (in my publisher)
Can't create the Lua state.
I'll try to find out where this happens.
Patrick Hm what format are you using ?
-- luigi
On Tue, Dec 25, 2012 at 9:25 PM, luigi scarso wrote:
On Tue, Dec 25, 2012 at 9:17 PM, Patrick Gundlach wrote:
Can't create the Lua state.
Hm what format are you using ?
non, I run luatex --ini --lua xxx.lua myfile.tex
Which platform ?
For me: 64-bit Mac OS X.
gdb luajittex GNU gdb 6.3.50-20050815 (Apple version gdb-1752) (Sat Jan 28 03:02:46 UTC 2012) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared libraries ... done
(gdb) r --luaonly "/Users/ConTeXt/tex/texmf-osx-64/bin/mtxrun" --script context a.tex Starting program: /Users/ConTeXt/tex/texmf-osx-64/bin/luajittex --luaonly "/Users/ConTeXt/tex/texmf-osx-64/bin/mtxrun" --script context a.tex Reading symbols for shared libraries ++......................... done Can't create the Lua state. Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000018 0x00000001002e8a8d in lua_checkstack (L=0x0, size=9) at lj_api.c:87 87 if (size > LUAI_MAXCSTACK || (L->top - L->base + size) > LUAI_MAXCSTACK) { (gdb) backtrace #0 0x00000001002e8a8d in lua_checkstack (L=0x0, size=9) at lj_api.c:87 #1 0x00000001002e8b75 in luaL_checkstack (L=0x0, size=9, msg=0x100869811 "too many arguments to script") at lj_api.c:97 #2 0x00000001000b80fd in lua_initialize (ac=6, av=0x7fff5fbffa08) at luainit.w:856 #3 0x00000001000022a0 in main (ac=6, av=0x7fff5fbffa08) at luatex.c:468 (gdb) The problem seems to be around lj_alloc_create () at lj_alloc.c:1134 1134 tbase = (char *)(CALL_MMAP(tsize)); (gdb) step CALL_MMAP [inlined] () at /Users/ConTeXt/luajittex/buildjit/libs/luajit/luajit-build/src/lj_alloc.c:208 ... 222 void *p = mmap((void *)alloc_hint, size, MMAP_PROT, MMAP_FLAGS, -1, 0); (gdb) p p $13 = (void *) 0x1010af000 ... 235 return CMFAIL; ... 1135 if (tbase != CMFAIL) { (gdb) p tbase $18 = 0xffffffffffffffff
Possibly related post: http://lua-users.org/lists/lua-l/2012-03/msg00656.html Mojca
On Thu, Jan 10, 2013 at 12:55 AM, Mojca Miklavec
On Tue, Dec 25, 2012 at 9:25 PM, luigi scarso wrote:
On Tue, Dec 25, 2012 at 9:17 PM, Patrick Gundlach wrote:
Can't create the Lua state.
Hm what format are you using ?
non, I run luatex --ini --lua xxx.lua myfile.tex
Which platform ?
For me: 64-bit Mac OS X.
gdb luajittex GNU gdb 6.3.50-20050815 (Apple version gdb-1752) (Sat Jan 28 03:02:46 UTC 2012) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared libraries ... done
(gdb) r --luaonly "/Users/ConTeXt/tex/texmf-osx-64/bin/mtxrun" --script context a.tex Starting program: /Users/ConTeXt/tex/texmf-osx-64/bin/luajittex --luaonly "/Users/ConTeXt/tex/texmf-osx-64/bin/mtxrun" --script context a.tex Reading symbols for shared libraries ++......................... done Can't create the Lua state.
Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000018 0x00000001002e8a8d in lua_checkstack (L=0x0, size=9) at lj_api.c:87 87 if (size > LUAI_MAXCSTACK || (L->top - L->base + size) > LUAI_MAXCSTACK) { (gdb) backtrace #0 0x00000001002e8a8d in lua_checkstack (L=0x0, size=9) at lj_api.c:87 #1 0x00000001002e8b75 in luaL_checkstack (L=0x0, size=9, msg=0x100869811 "too many arguments to script") at lj_api.c:97 #2 0x00000001000b80fd in lua_initialize (ac=6, av=0x7fff5fbffa08) at luainit.w:856 #3 0x00000001000022a0 in main (ac=6, av=0x7fff5fbffa08) at luatex.c:468 (gdb)
The problem seems to be around lj_alloc_create () at lj_alloc.c:1134 1134 tbase = (char *)(CALL_MMAP(tsize)); (gdb) step CALL_MMAP [inlined] () at /Users/ConTeXt/luajittex/buildjit/libs/luajit/luajit-build/src/lj_alloc.c:208 ... 222 void *p = mmap((void *)alloc_hint, size, MMAP_PROT, MMAP_FLAGS, -1, 0); (gdb) p p $13 = (void *) 0x1010af000 ... 235 return CMFAIL; ... 1135 if (tbase != CMFAIL) { (gdb) p tbase $18 = 0xffffffffffffffff
Possibly related post: http://lua-users.org/lists/lua-l/2012-03/msg00656.html I see ... not a simple problem.
-- luigi
On Thu, Jan 10, 2013 at 12:55 AM, Mojca Miklavec wrote:
On Tue, Dec 25, 2012 at 9:25 PM, luigi scarso wrote:
On Tue, Dec 25, 2012 at 9:17 PM, Patrick Gundlach wrote:
Can't create the Lua state.
Hm what format are you using ?
non, I run luatex --ini --lua xxx.lua myfile.tex
Which platform ?
For me: 64-bit Mac OS X.
gdb luajittex GNU gdb 6.3.50-20050815 (Apple version gdb-1752) (Sat Jan 28 03:02:46 UTC 2012) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared libraries ... done
(gdb) r --luaonly "/Users/ConTeXt/tex/texmf-osx-64/bin/mtxrun" --script context a.tex Starting program: /Users/ConTeXt/tex/texmf-osx-64/bin/luajittex --luaonly "/Users/ConTeXt/tex/texmf-osx-64/bin/mtxrun" --script context a.tex Reading symbols for shared libraries ++......................... done Can't create the Lua state.
Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000018 0x00000001002e8a8d in lua_checkstack (L=0x0, size=9) at lj_api.c:87 87 if (size > LUAI_MAXCSTACK || (L->top - L->base + size) > LUAI_MAXCSTACK) { (gdb) backtrace #0 0x00000001002e8a8d in lua_checkstack (L=0x0, size=9) at lj_api.c:87 #1 0x00000001002e8b75 in luaL_checkstack (L=0x0, size=9, msg=0x100869811 "too many arguments to script") at lj_api.c:97 #2 0x00000001000b80fd in lua_initialize (ac=6, av=0x7fff5fbffa08) at luainit.w:856 #3 0x00000001000022a0 in main (ac=6, av=0x7fff5fbffa08) at luatex.c:468 (gdb)
The problem seems to be around lj_alloc_create () at lj_alloc.c:1134 1134 tbase = (char *)(CALL_MMAP(tsize)); (gdb) step CALL_MMAP [inlined] () at /Users/ConTeXt/luajittex/buildjit/libs/luajit/luajit-build/src/lj_alloc.c:208 ... 222 void *p = mmap((void *)alloc_hint, size, MMAP_PROT, MMAP_FLAGS, -1, 0); (gdb) p p $13 = (void *) 0x1010af000
This number (p) seems way behind "if ((uintptr_t)p + size < MMAP_REGION_END)" where MMAP_REGION_END points to ((uintptr_t)0x80000000). I have found a partial solution: Following these instructions: #elif LJ_TARGET_OSX || defined(__FreeBSD__) || defined(__FreeBSD_kernel__) || defined(__OpenBSD__) /* OSX and FreeBSD mmap() use a naive first-fit linear search. ** That's perfect for us. Except that -pagezero_size must be set for OSX, ** otherwise the lower 4GB are blocked. And the 32GB RLIMIT_DATA needs ** to be reduced to 250MB on FreeBSD. */ export LDFLAGS=" -pagezero_size 10000 -image_base 100000000" ./buildjit.sh --debug The code now initialises properly and only breaks much later on at
luajittex --luaonly "/Users/ConTeXt/tex/texmf-osx-64/bin/mtxrun" --script context a.tex
mtx-context | run 1: luatex --fmt="/Users/ConTeXt/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luajittex/cont-en" --jobname="a" --lua="/Users/ConTeXt/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luajittex/cont-en.lui" --no-parse-first-line --c:currentrun=1 --c:fulljobname="./a.tex" --c:input="./a.tex" --c:kindofrun=1 "cont-yes.mkiv" This is LuaTeX, Version beta-0.74.0-2012122517 (rev 4541) \write18 enabled. This went wrong: ...0bfe781ce0dde776fb1556f32e/formats/luajittex/cont-en.lui:107: bad bytecode register . <*> cont-yes.mkiv ? That one needs further inspection (which I'm probably not going to do right now). Mojca
On Thu, Jan 10, 2013 at 1:41 AM, Mojca Miklavec
On Thu, Jan 10, 2013 at 12:55 AM, Mojca Miklavec wrote: I have found a partial solution:
Following these instructions:
#elif LJ_TARGET_OSX || defined(__FreeBSD__) || defined(__FreeBSD_kernel__) || defined(__OpenBSD__)
/* OSX and FreeBSD mmap() use a naive first-fit linear search. ** That's perfect for us. Except that -pagezero_size must be set for OSX, ** otherwise the lower 4GB are blocked. And the 32GB RLIMIT_DATA needs ** to be reduced to 250MB on FreeBSD. */
export LDFLAGS=" -pagezero_size 10000 -image_base 100000000" ./buildjit.sh --debug I suggest to build the stock luajit , (see http://lua-users.org/lists/lua-l/2012-03/msg00756.html ) if it works than it's a problem of luajittex, otherwise of luajit.
The code now initialises properly and only breaks much later on at
luajittex --luaonly "/Users/ConTeXt/tex/texmf-osx-64/bin/mtxrun" --script context a.tex
mtx-context | run 1: luatex --fmt="/Users/ConTeXt/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luajittex/cont-en" --jobname="a" --lua="/Users/ConTeXt/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luajittex/cont-en.lui" --no-parse-first-line --c:currentrun=1 --c:fulljobname="./a.tex" --c:input="./a.tex" --c:kindofrun=1 "cont-yes.mkiv" This is LuaTeX, Version beta-0.74.0-2012122517 (rev 4541)
^^^^^^^^^^^^^^^ Why LuaTeX ? -- luigi
On Thu, Jan 10, 2013 at 1:51 AM, luigi scarso wrote:
On Thu, Jan 10, 2013 at 1:41 AM, Mojca Miklavec wrote:
On Thu, Jan 10, 2013 at 12:55 AM, Mojca Miklavec wrote: I have found a partial solution:
Following these instructions:
#elif LJ_TARGET_OSX || defined(__FreeBSD__) || defined(__FreeBSD_kernel__) || defined(__OpenBSD__)
/* OSX and FreeBSD mmap() use a naive first-fit linear search. ** That's perfect for us. Except that -pagezero_size must be set for OSX, ** otherwise the lower 4GB are blocked. And the 32GB RLIMIT_DATA needs ** to be reduced to 250MB on FreeBSD. */
export LDFLAGS=" -pagezero_size 10000 -image_base 100000000" ./buildjit.sh --debug I suggest to build the stock luajit , (see http://lua-users.org/lists/lua-l/2012-03/msg00756.html ) if it works than it's a problem of luajittex, otherwise of luajit.
I don't know how to properly test luajit. Simple lua scripts run fine. I suspect that it's really about missing LDFLAGS in LuaJITTeX (or about changing the code upstream in such a way that those flags wouldn't be required ;).
luajittex --luaonly "/Users/ConTeXt/tex/texmf-osx-64/bin/mtxrun" --script context a.tex
mtx-context | run 1: luatex --fmt="/Users/ConTeXt/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luajittex/cont-en" --jobname="a" --lua="/Users/ConTeXt/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luajittex/cont-en.lui" --no-parse-first-line --c:currentrun=1 --c:fulljobname="./a.tex" --c:input="./a.tex" --c:kindofrun=1 "cont-yes.mkiv" This is LuaTeX, Version beta-0.74.0-2012122517 (rev 4541) ^^^^^^^^^^^^^^^ Why LuaTeX ?
I'm sorry and thanks for pointing it out. My bad. I only copy-pasted part of the command (starting with "luajittex") for debugging purposes. I now tried again with "context a.tex" and it seems to work properly. So it seems that export LDFLAGS=" -pagezero_size 10000 -image_base 100000000" solves the problem completely, at least on hello-world documents. But those two flags should probably be added by default then? Mojca
On Thu, Jan 10, 2013 at 2:45 AM, Mojca Miklavec
So it seems that export LDFLAGS=" -pagezero_size 10000 -image_base 100000000" solves the problem completely, at least on hello-world documents. But those two flags should probably be added by default then?
Could be. These flags are already set only for Darwin x64 in the Makefile and if you match this configuration than maybe you can export LDFLAGS=" -pagezero_size 10000 -image_base 100000000" and build the binary. And let's see if it fails again. -- luigi
Hi Luigi,
there is a new project on supelec: Luajittex.
more feedback from me: a sample project with my publisher: * normal luatex: 1m13sec * luajittex: 45 sec * luajittex --jiton. 42 sec This is great! (Mac OS X) To use it, I must be able to compile it on Linux/Mac/windows(mingw). I see that there's a mingw build file in the repository, so this looks like it's doable, right? Patrick
Hi Luigi,
there is a new project on supelec: Luajittex.
more feedback from me:
a sample project with my publisher:
* normal luatex: 1m13sec * luajittex: 45 sec * luajittex --jiton. 42 sec hm, --jiton is better that default...strange but possible, if you are not using the NYI functions in http://wiki.luajit.org/NYI Of course if your publisher now uses ffi http://luajit.org/ext_ffi.html (a kind of C code) than performance is superior ---but is better to
On Wed, Feb 20, 2013 at 11:25 AM, Patrick Gundlach
To use it, I must be able to compile it on Linux/Mac/windows(mingw). I see that there's a mingw build file in the repository, so this looks like it's doable, right?
yes, the rules are 1) Linux : buildjit.sh 2) Mac OS X : buildjit-clang.sh 3) Windows buildjit-mingw32.sh cross compiled under Linux buildjit-clang.sh should work also with Linux, assuming clang on standard PATH; buildjit.sh should work under Mac OS X, assuming gcc on standard PATH. I've not tried to build the luajittex.exe under Windows with mingw; the executalbe from http://www.w32tex.org/ is up-to-date and is compiled with the ms compiler, which gives a slightly better results. The new luajit 2.0.1 was published yesterday: I will update luajittex soon but not now --- I wait a liitle for some fixes . -- luigi
On 2/20/2013 11:25 AM, Patrick Gundlach wrote:
Hi Luigi,
there is a new project on supelec: Luajittex.
more feedback from me:
a sample project with my publisher:
* normal luatex: 1m13sec * luajittex: 45 sec * luajittex --jiton. 42 sec
This is great!
20-40 % is indeed what can be expected (mostly due to the faster vm) plus a somewhat smaller memory usage jiton only has an advantage when many internal commands/standard libs are used (type, a few string.*, most math.*)
(Mac OS X)
To use it, I must be able to compile it on Linux/Mac/windows(mingw). I see that there's a mingw build file in the repository, so this looks like it's doable, right?
we've done all tests with mingw, so it should be no problem Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
participants (4)
-
Hans Hagen -
luigi scarso -
Mojca Miklavec -
Patrick Gundlach