Hello, I'm playing with LuaTeX and I found a very strange behaviour that you can reproduce with the following steps: - compile tetest.c (it's just a tiny program writing utf8 in a file with wide characters) - 'texlua tetest.lua' (it executes the program and outputs the file) - 'lua tetest.lua' - compare... Is there a way for texlua to behave like lua in this case? Thank you, -- Elie
On Tue, Jun 8, 2010 at 16:49, Elie Roux wrote:
Hello,
I'm playing with LuaTeX and I found a very strange behaviour that you can reproduce with the following steps:
- compile tetest.c (it's just a tiny program writing utf8 in a file with wide characters) - 'texlua tetest.lua' (it executes the program and outputs the file) - 'lua tetest.lua' - compare...
Is there a way for texlua to behave like lua in this case?
It does in my case (on Mac). Mojca
Mojca Miklavec a écrit :
It does in my case (on Mac).
I should have put more details: I'm on debian/sid 32bit, using a binary compiled from svn. Here are the results: eroux@norgz-laptop:~/booklets/newbenedictiones$ lua tetest.lua aóa aóa eroux@norgz-laptop:~/booklets/newbenedictiones$ texlua tetest.lua aóa a eroux@norgz-laptop:~/booklets/newbenedictiones$ Thank you, -- Elie
Mojca Miklavec wrote:
On Tue, Jun 8, 2010 at 16:49, Elie Roux wrote:
Hello,
I'm playing with LuaTeX and I found a very strange behaviour that you can reproduce with the following steps:
- compile tetest.c (it's just a tiny program writing utf8 in a file with wide characters) - 'texlua tetest.lua' (it executes the program and outputs the file) - 'lua tetest.lua' - compare...
Is there a way for texlua to behave like lua in this case?
I don't think this is a bug, it is documented behavior (p 33 of the manual): In stock Lua, many things depend on the current locale. In LuaTEX, we can't do that, because it makes documents unportable. While LuaTEX is running if forces the following locale settings: LC_CTYPE=C LC_COLLATE=C LC_NUMERIC=C
Taco Hoekwater a écrit :
I don't think this is a bug, it is documented behavior (p 33 of the manual):
In stock Lua, many things depend on the current locale. In LuaTEX, we can't do that, because it makes documents unportable. While LuaTEX is running if forces the following locale settings: LC_CTYPE=C LC_COLLATE=C LC_NUMERIC=C
Ok, thank you! Is there a workaround for this? Or how should I change the program so that it works? Thank you, -- Elie
Elie Roux wrote:
Taco Hoekwater a écrit :
I don't think this is a bug, it is documented behavior (p 33 of the manual):
In stock Lua, many things depend on the current locale. In LuaTEX, we can't do that, because it makes documents unportable. While LuaTEX is running if forces the following locale settings: LC_CTYPE=C LC_COLLATE=C LC_NUMERIC=C
Ok, thank you! Is there a workaround for this? Or how should I change the program so that it works?
don't use an empty second argument to setlocale(). If you already know that you want UTF-8, set it to a suitable value instead. Best wishes, Taco
taco a écrit :
don't use an empty second argument to setlocale(). If you already know that you want UTF-8, set it to a suitable value instead.
The problem is that, IIUC, it's weird to put fr_FR.UTF-8 as a locale, as it's specific to my system... But I discovered that os.setenv("LC_CTYPE", nil) os.execute("myprogram") os.setenv("LC_CTYPE", "C") does the trick... with this it's the LANG environment variable (fr_FR.UTF-8) that will be set as LC_CTYPE. Setting setlocale (LC_CTYPE, "UTF-8"); in tetest.c does not solve the problem (strangely...). Is there another way? Thank you, -- Elie
On Thu, Jun 10, 2010 at 05:58:31PM +0200, Elie Roux wrote:
taco a écrit :
don't use an empty second argument to setlocale(). If you already know that you want UTF-8, set it to a suitable value instead.
The problem is that, IIUC, it's weird to put fr_FR.UTF-8 as a locale, as it's specific to my system... But I discovered that
os.setenv("LC_CTYPE", nil) os.execute("myprogram") os.setenv("LC_CTYPE", "C")
does the trick... with this it's the LANG environment variable (fr_FR.UTF-8) that will be set as LC_CTYPE.
Setting setlocale (LC_CTYPE, "UTF-8"); in tetest.c does not solve the problem (strangely...). Is there another way?
UTF-8 is not a locale, en_US.UTF-8 etc. are. -- Khaled Hosny Arabic localiser and member of Arabeyes.org team Free font developer
Elie Roux wrote:
taco a écrit :
don't use an empty second argument to setlocale(). If you already know that you want UTF-8, set it to a suitable value instead.
The problem is that, IIUC, it's weird to put fr_FR.UTF-8 as a locale, as it's specific to my system... But I discovered that
os.setenv("LC_CTYPE", nil) os.execute("myprogram") os.setenv("LC_CTYPE", "C")
does the trick... with this it's the LANG environment variable (fr_FR.UTF-8) that will be set as LC_CTYPE.
Setting setlocale (LC_CTYPE, "UTF-8"); in tetest.c does not solve the problem (strangely...). Is there another way?
Quite seriously: your C code should not be using libc functions that depend on the current locale if you want it to produce reproducible results. That's the core of the problem: LOCALE settings are unreliable, unless your program explicitly sets the LOCALE itself. So don't use that 'smart' printf() format, but just write (or reuse) an int2utf8 C function. Best wishes, Taco
participants (5)
-
Elie Roux -
Khaled Hosny -
Mojca Miklavec -
taco -
Taco Hoekwater