On 2 March 2010 09:21, Hans Hagen wrote:

On 2-3-2010 8:01, Taco Hoekwater wrote:

...

T T wrote:

...

With the other thread asking to add readlink, I would also like to ask for an addition of Windows specific GetShortPathName. Without this function it is impossible to check if two paths point to the same place.

Is that because of reparse points or so? I though GetShortPathName was more or less obsolete now.

indeed and luatex will not run nicely on windows < xp anyway (or os-9 or whatever old is around)

Win2000 should be OK too (or generally NT-based systems, although we don't test anything prior to Win2k). Support for Win98 and below have been officially dropped in TL'08 I think.

best is to settle on long names consistently (i noticed some short name issues a while ago)

This is not possible for two reasons: 1) there is still some old software breaking on long paths occasionally (or rather on names with spaces); we struggled recently with dviout but this is resolved now; I'm not aware of any other cases, atm. 2) even if we ignore old programs, short paths are accepted thoughout the Windows API and pop-up in various places from time to time (e.g. %TEMP% var); moreover, paths specified in mixed short/long convention are OK too and the only way to get a cannonical path on Windows is through GetShortPathName (or it's twin GetLongPathName, but this one is not nearly as useful) On 2 March 2010 07:01, Taco Hoekwater wrote:

T T wrote:

...

However, I'm not sure if beefing up LuaTeX with this kind of functionality is the best thing to do (there will be undoubtedly other things worth adding).

I am less hesitant than usual about adding stuff when the request is about scripting for texlive. If GetShortPathName is useful in such a context, I do not mind adding it.

From less essential functionality I would also love to have access to MessageBox for a minimal GUI functionality. I just introduced a few wrappers that run without a console (for gui programs), so there is no

I'm really glad to hear that. And yes, I think that GetShortPathName is one of those essential functions you can't work around when you need it. It's either there or not. place to output error messages. I worked around this with an ugly hack but a proper solution would be nicer.

...

Perhaps external modules? What's the status of loading binary modules? Will this be available for the coming TL release?

Yes, but I expect there will not be any binary modules actually included.

That might easily change once the machinery is in place ;) Adding/updating a module if definitely less troublesome and easier to experiment with than updating the whole luatex stack (which in practice we can't do in TL in between the releases).

...

And another request if I may. With kpsewhich it is possible to specify a search path explicitly with the -path switch, but this functionality is not exposed through lua kpse lib. Could this be added?

Sure. If there is anything else from kpathsea people want, please say so.

There are a few other kpsewhich options not covered thus far: 1) -all output all matches, one per line. 2) -engine=STRING set engine name to STRING. 3) [-no]-mktex=FMT disable/enable mktexFMT generation (FMT=pk/mf/tex/tfm). 4) -subdir=STRING only output matches whose directory ends with STRING. (1) and (4) could be occasionally useful (although the latter one can be trivially implemented in terms of the former one in Lua). For the remaining ones I have no opinion, I've never encountered the need for them myself. Perhaps also kpse.version should be added for version checking? Cheers, Tomek

On 2 March 2010 12:57, Hans Hagen wrote:

isn't there a flag somewhere that makes env vars expand into long paths?

Not to my knowledge but even if such a flag exists it doesn't help much. Short paths can appear in any place where file name is expected and to write correct code one can never assume that long convention is followed. This short/long file naming duality is a legacy of an ugly hack MS once pulled, and it is unlikely to ever disappear, I'm afraid. Cheers, Tomek

Taco Hoekwater wrote:

T T wrote:

...

followed. This short/long file naming duality is a legacy of an ugly hack MS once pulled, and it is unlikely to ever disappear, I'm afraid.

Tracker item added:

http://tracker.luatex.org/view.php?id=338

This is now in TRUNK, please test. (I verified it works under wine) Best wishes, Taco

T T wrote:

Thanks. Tried both of these and haven't noticed any problems.

Great.

I had some problems building on msys/mingw - the condition in getluatexsvnversion.sh:

if ([ -x `which svnversion` ] && [ -d ./.svn ])

This version was suggested bu Elie Roux.

evaluated always true even though I had no svnversion command in the path. I changed it to:

If [ -x `which svnversion` ] evaluates to true where there is no subversion, I think that counts as a bug in mingw that maybe you should file a bug report about.

if ( [ -d ./.svn ] && svnversion > /dev/null )

But this doesn't look bad either. Elie, is this ok? (my shell script skills are not that high either).

I would also suggest to remove the 'svn up' from the script. While working on/testing local changes you do not necessarily want to keep the working copy in sync with the head. And perhaps

Perhaps the call to getluatexsvnversion.sh should be an explicit option for build.sh instead of implied, but I prefer the rev. number to reflect the actual state of the repository, so I would like to leave the svn up command in it.

luatex_svnversion.h could be created/updated only when the revision number actually changes rather than every time (i.e. read the old

Probably. As I wrote above, I am not that good in shell either, and the generated header is not in the repository so I don't think it matters a whole lot, but if someone writes a patch I will apply it. Best wishes, Taco

T T wrote:

On 15 March 2010 07:47, Taco Hoekwater wrote:

...

T T wrote:

...

I would also suggest to remove the 'svn up' from the script. While working on/testing local changes you do not necessarily want to keep the working copy in sync with the head. And perhaps Perhaps the call to getluatexsvnversion.sh should be an explicit option for build.sh instead of implied, but I prefer the rev. number to reflect the actual state of the repository, so I would like to leave the svn up command in it.

This is just a suggestion, but I think that the decision whether to svn up or not should be configurable somehow. There might be justified reasons why one doesn't want to do that (e.g. svn doesn't work for me here because of proxy -- every time it stalls until timeout making it quite annoying).

Oh, alright then. Patch applied (#3494) Best wishes, Taco

T T wrote:

On 15 March 2010 12:03, Taco Hoekwater wrote:

...

Oh, alright then. Patch applied (#3494)

Thanks. One more patch for your consideration for something I needed recently: getting absolute, canonical file paths. In principle this can be implemented in pure Lua with the current lfs module, but getting all the corner cases right (e.g. foo/bar/./../baz) is not so easy. Perhaps having a convenience function for this is not such a bad idea (and using proper system APIs will guarantee that the result is correct).

There is no Unix counterpart in the patch, because I didn't know how to implement it (with a call to realpath?) and I would have no way to test it anyway.

My man page for realpath(3) says: BUGS: Avoid using this function. Its is broken by design But it does not offer an alternate solution. Looks like it would need private code, which is not a lot of fun to write (unless there is something in kpathsea?).

On 15 March 2010 15:54, Taco Hoekwater wrote:

My man page for realpath(3) says:

BUGS:  Avoid using this function. Its is broken by design

But it does not offer an alternate solution.

Bummer.

Looks like it would need private code, which is not a lot of fun to write (unless there is something in kpathsea?).

I agree, if you can't trust the API call, there is no point in hand coding this in C. It is not trivial and it will be easier to implement on the Lua side anyway. On 15 March 2010 16:00, Taco Hoekwater wrote:

What did you need it for anyway? As perhaps the problem it solves is win32-only?

The problem was to create a directory path with all intermediate ones if they don't exist. lfs.mkdir cannot create a hierarchy. Once I started coding this, I realized it's not so simple in general, because one needs to resolve the path first, get rid of all the '..' and '.' etc. GetFullPathName call is very handy for this, so I thought it would be nice to access it. Too bad that there is no reliable equivalent on Unix. For now I ended up with 'cmd.exe /c mkdir "some\dir\hierarchy\to\create"' :->. I'm not very proud of this solution but it gets the job done for this particular problem. Cheers, Tomek PS. Perhaps Unix's realpath is not the best match for what GetFullPathName does. With GetFullPathName the resolved path does not have to exist.

2010/3/15 Taco Hoekwater :

Ah, I see. I any case, I have applied your patch already, exactly as-is. I'll think about a unix solution.

Hello, I just thought about something: in a script we need to detect if, under windows, luatex is run under cygwin or not (cygwin and non-cygwin are the same binary for me). Currently we do something like local system = io.popen("uname -s"):read("*l") if system then if system:match("^CYGWIN") then system = 'cygwin' end end but having os.type == 'cygwin' as a condition would be cleaner... The thing is the check must be done at runtime, and I don't know anything about the Windows API nor compilation under cygwin/mingw... Tomek, would you know how to patch LuaTeX to do that? Thank you, -- Elie

2010/3/16 Taco Hoekwater :

...

local system = io.popen("uname -s"):read("*l")

You could test against os.uname().

Hello, here is what I get from os.uname() now: os.uname().sysname: Windows XP os.uname().machine: i686 os.uname().release: build 2600 os.uname().version: 5.01 Service Pack 3 os.uname().nodename: ROTIMN0L020707 No trace of cygwin, which is normal I guess, as the system calls go directly to windows, being cygwin-transparent... Any idea of how to detect cygwin? I must say I don't have any, as you cannot call cygwin-specific things in the binary... Thank you! -- Elie

On 16 March 2010 08:01, Élie Roux wrote:

I just thought about something: in a script we need to detect if, under windows, luatex is run under cygwin or not (cygwin and non-cygwin are the same binary for me). Currently we do something like

I'm not sure if I understand you correctly. I've just checked cygwin binaries and they are are different from the win32 ones we ship. Do you mean that the cygwin binaries can be run in two modes: under cygwin and under native windows? That would be strange, but then again I don't know enough about cygwin to tell.

local system = io.popen("uname -s"):read("*l") if system then    if system:match("^CYGWIN") then        system = 'cygwin'    end end

but having os.type == 'cygwin' as a condition would be cleaner... The thing is the check must be done at runtime, and I don't know anything about the Windows API nor compilation under cygwin/mingw...

Tomek, would you know how to patch LuaTeX to do that?

Would Taco's suggestion with os.uname() work for you?

From what I see patching luatex should be straightforward, the relevant file is:

trunk\source\texk\web2c\luatexdir\lua\loslibext.c which have defines: OS_PLATTYPE => os.type OS_PLATNAME => os.name All I need to know is what symbol to test to distinguish between windows and cygwin and how cygwin should identify itself (os.type = 'unix', os.name = 'cygwin'?). Note that os.type only gives the three basic types: 'msdos', 'windows' and 'unix', it is os.name that diffrentiates further. Note also that this is a compile time decision, i.e. the same binary will always identify itself in the same way (similarly with os.uname). I'm not sure if this is what you want, though. Cheers, Tomek

On 16 March 2010 08:20, Taco Hoekwater wrote:

Just wondering: what does it return for ".." then?

Path to the one dir up from the current dir.

I've given up on a Unix C function for this there are just too many potential problems with symlinks. As you are using mkdir now also, would it be ok if I undo your patch? (as you'll need mkdir on unix anyway)?

Yes, please do. FYI, in our perl code we use the "abs_path" function from the cwd module for the functionality I was after but I don't know how's that implemented. Cheers, Tomek

Hi Tomek. T T wrote:

From less essential functionality I would also love to have access to MessageBox for a minimal GUI functionality.

This is something that will not go in the core, so that would have to be an external binary module (there are just too many threading complications in relation to windows gui programming, and we do want to start using threading at some point in the not too distant future).

...

...

And another request if I may. With kpsewhich it is possible to specify a search path explicitly with the -path switch, but this functionality is not exposed through lua kpse lib. Could this be added? Sure. If there is anything else from kpathsea people want, please say so.

Tracker item: http://tracker.luatex.org/view.php?id=339 Best wishes, Taco

Taco Hoekwater wrote:

Tracker item:

http://tracker.luatex.org/view.php?id=339

New functionality in TRUNK: \subsection{\luatex{lookup}} A more powerful (but slower) generic method for finding files is also available (since 0.51). \startfunctioncall <string> f, ... = kpse.lookup(<string> filename, <table> options) \stopfunctioncall The options match commandline arguments from \type{kpsewhich}: \starttabulate[|l|l|p|] \NC \ssbf key \NC \ssbf type \NC \ssbf description \NC \NR \NC debug \NC number \NC set debugging flags for this lookup\NC \NR \NC format \NC string \NC use specific file type (see list above)\NC \NR \NC dpi \NC number \NC use this resolution for this lookup; default 600\NC \NR \NC path \NC string \NC search in the given path\NC \NR \NC all \NC boolean \NC output all matches, not just the first\NC \NR \NC must-exist\NC boolean \NC search the disk as well as ls-R if necessary\NC \NR \NC mktexpk \NC boolean \NC disable/enable mktexpk generation for this lookup\NC \NR \NC mktextex \NC boolean \NC disable/enable mktextex generation for this lookup\NC \NR \NC mktexmf \NC boolean \NC disable/enable mktexmf generation for this lookup\NC \NR \NC mktextfm \NC boolean \NC disable/enable mktextfm generation for this lookup\NC \NR \NC subdir \NC string or table \NC only output matches whose directory part ends with the given string(s) \NC \NR \stoptabulate \subsection{\luatex{version}} Returns the kpathsea version string (new in 0.51) \startfunctioncall <string> r = kpse.version() \stopfunctioncall Best wishes, Taco