Hi, I have encountered a problem with os.spawn(arg) on Windows. If arg[1]='foo bar' and arg[0]=nil, then the spawned process gets 'foo' as argv[0] and 'bar' as argv[1]. This might be expected, since according to docs, os.spawn doesn't do any argument processing. So I tried double quoting arg[1]='"foo bar"' (i.e. double quotes are part of arg[1]) but then os.spawn fails with invalid argument error. Is this expected? Furthermore, passing a string '"foo bar"' to os.spawn (including the double quotes) instead of a table, results in spawning 'foo bar' but again with 'foo' as argv[0] and 'bar' as argv[1]. The only way I found to properly spawn a command with spaces is by giving arg[0] explicitly: arg={[0]='foo bar', '"foo bar"', ...} Cheers, Tomek
Hi Tomek, T T wrote:
Hi,
I have encountered a problem with os.spawn(arg) on Windows. If arg[1]='foo bar' and arg[0]=nil, then the spawned process gets 'foo' as argv[0] and 'bar' as argv[1]. This might be expected, since according to docs, os.spawn doesn't do any argument processing.
Let's put it this way: windows command line processing does not agree with my neurons, so I have stayed way from it as much as possible (and will continue to do so). A good, clean patch would be welcome of course, but I myself will not spend any more time on the stuff, not even for bug reports, as all that would be accomplished by doing so is that it would give me a headache, and there are much more pleasing ways to reach that state. Best wishes, Taco
2009/11/23 Taco Hoekwater
Let's put it this way: windows command line processing does not agree with my neurons, so I have stayed way from it as much as possible (and will continue to do so).
Yes, I know it's PITA.
A good, clean patch would be welcome of course, but I myself will not spend any more time on the stuff, not even for bug reports, as all that would be accomplished by doing so is that it would give me a headache, and there are much more pleasing ways to reach that state.
OK, I'm looking into that. Cheers, Tomek
2009/11/23 T T
OK, I'm looking into that.
After some digging through the sources and googling around I think the problem is not with luatex's implementation but with _spawnvpe function on Windows, which can't handle arguments with spaces (how pathetic). So I should write to M$ now, I guess. Sigh. At least there is a workaround with setting arg[0] explicitly, but some note to unwary in the documentation would be good as well. Cheers, Tomek
T T wrote:
2009/11/23 T T
: OK, I'm looking into that.
After some digging through the sources and googling around I think the problem is not with luatex's implementation but with _spawnvpe function on Windows, which can't handle arguments with spaces (how pathetic). So I should write to M$ now, I guess. Sigh. At least there is a workaround with setting arg[0] explicitly, but some note to unwary in the documentation would be good as well.
maybe it's an option to reserve arg[0] for the (optional) path and set the path explicitely (http://msdn.microsoft.com/en-us/library/aa273351%28VS.60%29.aspx) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
Taco Hoekwater wrote:
Hi Tomek,
T T wrote:
Hi,
I have encountered a problem with os.spawn(arg) on Windows. If arg[1]='foo bar' and arg[0]=nil, then the spawned process gets 'foo' as argv[0] and 'bar' as argv[1]. This might be expected, since according to docs, os.spawn doesn't do any argument processing.
Let's put it this way: windows command line processing does not agree with my neurons, so I have stayed way from it as much as possible (and will continue to do so).
A good, clean patch would be welcome of course, but I myself will not spend any more time on the stuff, not even for bug reports, as all that would be accomplished by doing so is that it would give me a headache, and there are much more pleasing ways to reach that state.
The Python community have solved this problem, with the subprocess module. http://docs.python.org/library/subprocess.html To quote from that URL ==== The subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several other, older modules and functions, such as: os.system os.spawn* os.popen* popen2.* commands.* ==== The specification for subprocess is at http://www.python.org/dev/peps/pep-0324/ If anyone would like to create a module for lua to solve the problem of filenames with spaces then Python's subprocess would be a good place to start. -- Jonathan
On 23 November 2009 Taco Hoekwater wrote:
Let's put it this way: windows command line processing does not agree with my neurons, so I have stayed way from it as much as possible (and will continue to do so).
A good, clean patch would be welcome of course, [...]
Hi Taco, if you change anything though, please inform the TeX Live team in advance. We probably have to adapt our scripts then. The scripts already provide a function: function fixwin (args_unix) if os.type == 'windows' then local args_win={} -- new table args_win[0]=args_unix[1] for i=1, #args_unix do args_win[i]='"'..args_unix[i]..'"' end return args_win else return args_unix end end Regards, Reinhard -- ---------------------------------------------------------------------------- Reinhard Kotucha Phone: +49-511-3373112 Marschnerstr. 25 D-30167 Hannover mailto:reinhard.kotucha@web.de ---------------------------------------------------------------------------- Microsoft isn't the answer. Microsoft is the question, and the answer is NO. ----------------------------------------------------------------------------
Reinhard Kotucha wrote:
On 23 November 2009 Taco Hoekwater wrote:
Let's put it this way: windows command line processing does not agree with my neurons, so I have stayed way from it as much as possible (and will continue to do so).
A good, clean patch would be welcome of course, [...]
Hi Taco, if you change anything though, please inform the TeX Live team in advance. We probably have to adapt our scripts then. The scripts already provide a function:
don't worry changes in these mechanism will never show up in stable versions unannounced as it will likely influence many; of course, as long as luatex functionality is not declared frozen we take the freedom to change things, but in this case the fact that texlua is used for scripts in tex distributions asks for extra care as far as i can see the main problem is *not* in the interface, it's just that the underlying windows function cannot deal with spaces in the program location specification and that (at some point) probably be solved in a compatible way Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
2009/11/24 Hans Hagen
Reinhard Kotucha wrote:
On 23 November 2009 Taco Hoekwater wrote:
> Let's put it this way: windows command line processing does not > agree with my neurons, so I have stayed way from it as much as > possible (and will continue to do so). > > A good, clean patch would be welcome of course, [...]
Hi Taco, if you change anything though, please inform the TeX Live team in advance. We probably have to adapt our scripts then. The scripts already provide a function:
don't worry changes in these mechanism will never show up in stable versions unannounced as it will likely influence many; of course, as long as luatex functionality is not declared frozen we take the freedom to change things, but in this case the fact that texlua is used for scripts in tex distributions asks for extra care
as far as i can see the main problem is *not* in the interface, it's just that the underlying windows function cannot deal with spaces in the program location specification and that (at some point) probably be solved in a compatible way
I could it give a go and try to implement something that makes sense
on Windows. The _spawnvpe function takes four arguments and two of
them, cmdname (command path/name) and argv array, are of interest
here.
There are two cases to consider:
(1) String argument cmd_string to os.exec/os.spawn
It doesn't make sense to parse this string into argv, because
_spawnvpe will concatenate it back into single string anyway (because
that's required by CreateProcess API), but any quotation of arguments
will be lost on the way (because _spawnvpe is incredibly dumb).
Instead we can take:
argv[0] := cmd_string
argv[1] := NULL
cmdname = <first argument from cmd_string striped from quotes>
This way we can full _spawnvpe to pass the command string as is
directly to the CreateProcess API.
(2) Table argument cmd_table to os.exec/os.spawn
argv := cmd_table
if argv[0] == NULL {
argv[0] =
T T wrote:
The above changes do not change the behaviour of currently working cases, only the failing ones and those resulting in wrongly parsed arguments being passed to the child process are affected.
If that proposal sounds OK to you, I will write a patch implementing it.
Yes, please do! Best wishes, Taco
2009/11/27 Taco Hoekwater
T T wrote:
The above changes do not change the behaviour of currently working cases, only the failing ones and those resulting in wrongly parsed arguments being passed to the child process are affected.
If that proposal sounds OK to you, I will write a patch implementing it.
Yes, please do!
OK. I need to test the patch now, so a few questions more. A few days ago I managed to compile the current trunk on mingw (am I being lucky or is this officially supported?) but this takes ages. My changes are limited to just one file (loslibext.c). Is there some way to quickly rebuild only what's needed? Cheers, Tomek
T T wrote:
2009/11/27 Taco Hoekwater
: T T wrote:
The above changes do not change the behaviour of currently working cases, only the failing ones and those resulting in wrongly parsed arguments being passed to the child process are affected.
If that proposal sounds OK to you, I will write a patch implementing it. Yes, please do!
OK. I need to test the patch now, so a few questions more.
A few days ago I managed to compile the current trunk on mingw (am I being lucky or is this officially supported?) but this takes ages. My changes are limited to just one file (loslibext.c). Is there some way to quickly rebuild only what's needed?
$ ./build.sh --make (for a somewhat broad definition of 'only', but still much better than re-running configure). Best wishes, Taco
2009/11/27 Taco Hoekwater
T T wrote:
2009/11/27 Taco Hoekwater
: T T wrote:
The above changes do not change the behaviour of currently working cases, only the failing ones and those resulting in wrongly parsed arguments being passed to the child process are affected.
If that proposal sounds OK to you, I will write a patch implementing it. Yes, please do!
OK. I need to test the patch now, so a few questions more.
A few days ago I managed to compile the current trunk on mingw (am I being lucky or is this officially supported?) but this takes ages. My changes are limited to just one file (loslibext.c). Is there some way to quickly rebuild only what's needed?
$ ./build.sh --make
(for a somewhat broad definition of 'only', but still much better than re-running configure).
Indeed. The patch is attached. It did work for me as intended, but please review. I also made adjustments to build.sh for mingw, see below: Cheers, Tomek Index: build.sh =================================================================== --- build.sh (revision 3195) +++ build.sh (working copy) @@ -51,10 +51,10 @@ STRIP=strip LUATEXEXE=luatex -if [ `uname` = "Darwin" ] ; -then - export MACOSX_DEPLOYMENT_TARGET=10.4 -fi; +case `uname` in + Darwin ) export MACOSX_DEPLOYMENT_TARGET=10.4 ;; + MINGW32* ) LUATEXEXE=luatex.exe ;; +esac B=build CONFHOST=
T T wrote:
Indeed. The patch is attached. It did work for me as intended, but please review.
Applied and looks ok, but I cannot actually test this.
I also made adjustments to build.sh for mingw, see below:
Added the MINGW part of that patch also. Best wishes, Taco
T T wrote:
2009/11/27 Taco Hoekwater
: T T wrote:
The above changes do not change the behaviour of currently working cases, only the failing ones and those resulting in wrongly parsed arguments being passed to the child process are affected.
If that proposal sounds OK to you, I will write a patch implementing it. Yes, please do!
OK. I need to test the patch now, so a few questions more.
A few days ago I managed to compile the current trunk on mingw (am I being lucky or is this officially supported?) but this takes ages. My changes are limited to just one file (loslibext.c). Is there some way to quickly rebuild only what's needed?
build --make doesn't work for you? -- Pawe/l Jackowski P.Jackowski@gust.org.pl
T T wrote:
Hi,
I have encountered a problem with os.spawn(arg) on Windows. If arg[1]='foo bar' and arg[0]=nil, then the spawned process gets 'foo' as argv[0] and 'bar' as argv[1]. This might be expected, since according to docs, os.spawn doesn't do any argument processing.
So I tried double quoting arg[1]='"foo bar"' (i.e. double quotes are part of arg[1]) but then os.spawn fails with invalid argument error. Is this expected?
Furthermore, passing a string '"foo bar"' to os.spawn (including the double quotes) instead of a table, results in spawning 'foo bar' but again with 'foo' as argv[0] and 'bar' as argv[1]. The only way I found to properly spawn a command with spaces is by giving arg[0] explicitly:
arg={[0]='foo bar', '"foo bar"', ...}
arg = { "dir", "bar", '"foo bar"', } os.spawn(arg) works ok, so without arg[0] Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
T T wrote:
2009/11/23 Hans Hagen
: arg = { "dir", "bar", '"foo bar"', }
os.spawn(arg)
works ok, so without arg[0]
Yes, but your arg[1] has no spaces in it. Try running some program under %ProgramFiles% or other path with spaces.
hm, hard to find one, but "c:/Program Files/WinRAR/rar.exe", as arg[1] runs rar Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
participants (6)
-
Hans Hagen
-
Jonathan Fine
-
Paweł Jackowski
-
Reinhard Kotucha
-
T T
-
Taco Hoekwater