- dev-luatex - NTG Mailing Lists

valid Unicode character treatment?
by David Kastrup 24 Apr '07

24 Apr '07

Hi, I've been wondering about several things with regard to Unicode/utf-8. What is the situation in fonts (of any type) in general concerning the non-existing Unicode codepoints corresponding to the 16-bit surrogate codes D800-DFFF? Are there any fonts actually putting stuff there, if only ligatures? A compliant utf-8 file is not supposed to contain any codes in that area, so we would not want to have them appear as part of "overfull hbox" messages and similar if I am not mistaken. I am currently thinking about a utf-8 strategy that would be least prone to causing internal inconsistencies: basically I think that certain properties of _legal_ utf-8 should be guaranteed inside of LuaTeX, like that the number of characters in a string being equal to the number of bytes outside of the 80-BF code range, that characters are encoded with minimal length, that the number of bytes never exceeds 4 times the number of characters and similar things. I'd tend to move the special "output in byte-sized chunks" characters to "11xxxx: after all, fonts may contain stuff in the "10ffxx area, and overfull box messages will output those characters. Those can be represented internally by (basically out of range) utf-8 sequences in the obvious way. If the input reader for utf-8 cranks out the corresponding "output in byte-sized chunks" characters for illegal utf-8 byte sequences, then inputting them accidentally will usually lead to "missing character" errors, but it will be possible to write stuff like \message{^^11xxxx} to produce verbatim output, and \message{illegal byte sequence} will reproduce the illegal byte sequence unchanged, without having any illegal byte sequence (apart from the codes for "11xxxx) present in the innards of LuaTeX. I'd think it reasonable not to permit those characters "11xxxx into the normal character arrays (lccode, uccode, chardef ...) in a manner similar to how the codes from "80 to "ff were treated in TeX-2.x (which had 7-bit arrays inside, but accepted 256 characters in fonts and input). When I write "11xxxx instead of "1100xx it is because I don't yet have a clear idea about whether or how one would bother thinking about transparent word output when using UCS-16 (which has surrogate characters and stuff). Possibly one should just completely forget about facilitating UCS-16 output, whether through callbacks or otherwise. Ok, this is just a sketch (I consider the prospect disturbing of having to code without being able to rely internally on legal utf-8 sequences as long as possibly involved callbacks are bugfree), but the main question of this posting was what how surrogate code points are treated in fonts. Thanks, David -- David Kastrup

2 2

few font setup related questions
by Igor Khomyakov 24 Apr '07

24 Apr '07

Hi All, I hope this is the right mailing list. I am trying to typeset a multilingual document with LuaTeX in pdftex mode (pdfoutput=1). The document contains english, russian and japanese texts (UTF-8 encoding). Does LuaTeX support large True Type and/or Open Type fonts? What is the best way to set these fonts up with LuaTeX ? Can they be set up "directly", or via virtual font (OVF) ? Does one need to write any Lua code in order to achieve this? Are there any samples available ? Would appreciate any pointers. Kind regards, Igor Khomyakov

2 6

\input does not look at open_read_file callback?
by David Kastrup 24 Apr '07

24 Apr '07

Hi, in the following code, the callback is never reached when called with luatex -ini bug -- David Kastrup

3 4

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 23 Apr '07

23 Apr '07

U luatex/src/texk/web2c/luatexdir/luatex.web U luatex/src/texk/web2c/luatexdir/lua/lstatslib.c U luatex/src/texk/web2c/luatexdir/font/ttflua.c Checked out revision 424.

1 0

Three fixes
by David Kastrup 23 Apr '07

23 Apr '07

Hi, here is a patch to luatex.web. The first hunk fixes the FSF's address to the current one, the second one makes detection of illegal utf-8 sequences slightly more attentive (though I actually would prefer a different approach), the third fixes a typo in the comment for the nc/nd patch. -- David Kastrup

2 1

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 20 Apr '07

20 Apr '07

U luatex/build.sh.cross U luatex/build.sh.linux U luatex/src/texk/web2c/luatexdir/luatex.mk U luatex/src/texk/web2c/luatexdir/luatex.web U luatex/src/texk/web2c/luatexdir/lua/lstatslib.c U luatex/src/texk/web2c/luatexdir/lua/lfontlib.c U luatex/src/texk/web2c/luatexdir/lua/lpdflib.c U luatex/build.sh Checked out revision 421.

1 0

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 19 Apr '07

19 Apr '07

U luatex/src/texk/web2c/luatexdir/font/ttflua.c Checked out revision 416.

1 0

SVN tree snapshot?
by David Kastrup 18 Apr '07

18 Apr '07

Hi, when trying to follow the "download nightly SVN Tree Snapshot Link" on page <URL:https://foundry.supelec.fr/scm/?group_id=10> which points to <URL:https://foundry.supelec.fr/snapshots.php?group_id=10>, I get the message PAGE NOT FOUND Does anybody know what is up with that? Background is that I am trying to use git-svnimport and it does not like the anonymous access, and it does not like the certificates for the https access. Downloading the repository locally would presumably work, but the link is dead. Thanks, -- David Kastrup

2 1

Cron <luatex@server> /home/luatex/luatex-svn
by root＠aanhet.net 17 Apr '07

17 Apr '07

U luatex/src/texk/web2c/luatexdir/font/ttflua.c Checked out revision 415.

1 0

Question about localleftbox/localrightbox/local*
by David Kastrup 17 Apr '07

17 Apr '07

Hi, when creating some boxes, specials recording the current values of localleftbox etc get inserted. Is there anybody on this list who actually uses those constructs? I am trying to figure out sensible semantics for them, and I have my doubts that the current semantics actually are fully operative for the purposes for which they have been created. So I'd be interested in hearing of any existing use cases/TeX code for them. Part of the reason is that they may interfere with normal typesetting in a manner similar to source specials in the current manner of implementation. Part of the reason is that the purpose for which I think they have been created (running quote characters) might be defeated by the current implementation if one unboxes material into a running paragraph. Analysing this would be facilitated by existing code making use of those features. Thanks, -- David Kastrup

2 2