On 09/25/2015 03:04 PM, Hans Hagen wrote:

On 9/25/2015 4:09 PM, Orion Poplawski wrote:

I'm just starting to take a look how luatex makes use of lua.  And it
appears that luatex does not merely use lua, but extends it in various
ways, essentially creating new lua dialect.  Is the assessment correct?
  Is there any documentation on this?

Indeed we have some more on board. Keep in mind that lua is used as
extension language so whenever it's used in an application there can be
extensions.

Btw, a dialect implies a different language. Adding a couple of
functions is not making it a dialect.

You appear to be extending it in at least four areas by adding new functions, and replacing some as described below.  liolibext is the most "entangled" due to duplicating liolib code, lstrlibext also uses some Lua internals that are not normally exposed to consumers of Lua (e.g. lua_lock).  I'm fine with not calling this a new "dialect", but it is certainly extended, though modifying popen()/read()/lines() is interesting.  I'm interested to know if there has been any discussion with the Lua maintainers about incorporating any of these changes in Lua proper?

liolibext.c:
/* This extension only replaces one function: io.popen() and two
   metatable entries: f:read() and f:lines() but unfortunately
   it has to copy quite a bunch of lines from the original liolib.c
   to do so.
*/

llfslibext.c: adds to lfs:
    lua_setfield(L, -2, "isdir");
    lua_pushcfunction(L, file_is_file);
    lua_setfield(L, -2, "isfile");
    lua_pushcfunction(L, read_link);
    lua_setfield(L, -2, "readlink");
    lua_pushcfunction(L, get_short_name);
    lua_setfield(L, -2, "shortname");

loslibext.c:
    lua_getglobal(L, "os");
    lua_pushcfunction(L, ex_sleep);
    lua_setfield(L, -2, "sleep");
    lua_pushliteral(L, OS_PLATTYPE);
    lua_setfield(L, -2, "type");
    lua_pushliteral(L, OS_PLATNAME);
    lua_setfield(L, -2, "name");
    lua_pushcfunction(L, ex_uname);
    lua_setfield(L, -2, "uname");
#if (! defined (_WIN32))  && (! defined (__SUNOS__))
    lua_pushcfunction(L, os_times);
    lua_setfield(L, -2, "times");
#endif
#if ! defined (__SUNOS__)
    lua_pushcfunction(L, os_gettimeofday);
    lua_setfield(L, -2, "gettimeofday");
#endif

    if (!safer) {
        lua_pushcfunction(L, os_setenv);
        lua_setfield(L, -2, "setenv");
        lua_pushcfunction(L, os_exec);
        lua_setfield(L, -2, "exec");
        lua_pushcfunction(L, os_spawn);
        lua_setfield(L, -2, "spawn");
        lua_pushcfunction(L, os_execute);
        lua_setfield(L, -2, "execute");
        lua_pushcfunction(L, os_tmpdir);
        lua_setfield(L, -2, "tmpdir");
    }

lstrlibext.c
  {"utfvalues", str_utfvalues},
  {"utfcharacters", str_utfcharacters},
  {"characters", str_characters},
  {"characterpairs", str_characterpairs},
  {"bytes", str_bytes},
  {"bytepairs", str_bytepairs},
  {"explode", str_split},
  {"dump", str_dump},

Also, is luajit used in the same way, or would it be possible to use a
system installed luajit in building luatex?

The only benefit of luajit is the faster virtual machine. A downside is
that there are some limitations in memory cq. stack usage. We use a
patch that makes the string hashing faster (we use the normal lua method
instead of the luajit one that is optimizied for urls and that can make
luajit slower than lua).

Becaus eluatex uses lua as extension language it will always be shipped
with a specific version (linked in or alongside the binary).

Hans

FWIW - Here are some changes I had to make while trying to compile against lua 5.3: