Possible issue with input from external libraries (page break problem?)
Hi Taco I have been experimenting with external C libraries by writing Lua bindings under Windows --- DLLs. I have noticed something I'd like to ask about --- to know if it is expected behavior. When these DLLs return text I'm doing a lot of tex.write(...) of the data returned from the DLL functions (for testing). What I've noticed is that when the text data from the DLL calls is output (by tex.write()) and a page break occurs in the middle of the tex.write() outputs calls, then the text is lost across the break. Here I am guessing....> It looks like if the page break machinery is called whilst the text from the DLL is being output by tex.write() then something temporarily "forgets??" that text data is still coming and so the page break is losing data --- actually it looks like the incoming tex.write() text simply carries on and falls off the page so you lose it. Perhaps the page bbreak machinery is getting out of synch with the tex.write() input. This is purely guesswork and I could easily be wrong of course. I can e-mail you (personally?) a sample PDF which is easier than trying to explain this. Is this something to be expected? A minimal code example may not be that helpful but here is one. Here I have a binding to FreeType (I call it luatype) and in this example it just dumps the glyph names reported by FreeType: \directlua { require "luatype" local result = luatype.fontname("c:/msys/1.0/trunk/build/texk/fonts/type1/lucida/LBD.PFB") tex.print(result.glyphs) %can also say you["glyphs"], as the same thing for k,v in pairs(result) do tex.print(v.." "..k) end } Here is binding to an XML library \directlua 0 { require "lxp" text = [[<note> <to>Tove</to> <from>Jani</from> <heading>Reminder <urgent> yes, very</urgent></heading> <body>Don't forget me this weekend!</body> </note>]] local x=0 callbacks = { StartElement = function (p, s) tex.print("Starting element: ".. "{bf " ..s.. "}" .. " call depth = ".. x.. "par") x=x+1 end, EndElement = function (p, s) x=x-1 tex.sprint("Finishing element: " .. "{it " ..s.. "}" .. " call depth = ".. x.. "par") end, CharacterData = function (p, s) tex.sprint("Character data (may be white space) = " .. "{itbf " ..s.. "}" .. "par") end} p=lxp.new(callbacks) } \directlua{p:parse(text)}!. \directlua{p:close()} If the tex.write() from output of either DLLs goes near a page break it "falls off the page" and gets lost. Apologies if this is not explained very well. Please let me know if you want a PDF sample. All the best Graham
Graham Douglas wrote:
Hi Taco
I have been experimenting with external C libraries by writing Lua bindings under Windows --- DLLs.
I have noticed something I'd like to ask about --- to know if it is expected behavior.
When these DLLs return text I'm doing a lot of tex.write(...) of the data returned from the DLL functions (for testing).
What I've noticed is that when the text data from the DLL calls is output (by tex.write()) and a page break occurs in the middle of the tex.write() outputs calls, then the text is lost across the break.
I don't think this is supposed to happen, but please send me the pdf for a closer look and, if possible, a source to go with it that uses a lua library that is available to me. I am on linux, but lxp should work for me as well. I can even try to run the test under wine. Best wishes, Taco
Hi Graham, Taco Hoekwater wrote:
I don't think this is supposed to happen, but please send me the pdf for a closer look and, if possible, a source to go with it that uses a lua library that is available to me. I am on linux, but lxp should work for me as well. I can even try to run the test under wine.
I've looked at your file, and it is not related to external libraries nor the luatex extensions at all: The problem is the \immediate\pdfximage. You can get the exact same problem in pdftex with this input: \pdfoutput=1 \immediate\pdfximage width 0.5\hsize {./cow.pdf} \pdfrefximage\pdflastximage \input knuth \input knuth \input knuth \input knuth \input knuth \input knuth \input knuth \bye In both cases, the problem can be fixed by adding a \hbox around the \pdfrefximage. But now I have a question for Hartmut: is this supposed to happen? Best wishes, Taco
Hi Taco Taco Hoekwater wrote:
Hi Graham,
Taco Hoekwater wrote:
I don't think this is supposed to happen, but please send me the pdf for a closer look and, if possible, a source to go with it that uses a lua library that is available to me. I am on linux, but lxp should work for me as well. I can even try to run the test under wine.
I've looked at your file, and it is not related to external libraries nor the luatex extensions at all: The problem is the \immediate\pdfximage.
You can get the exact same problem in pdftex with this input:
\pdfoutput=1 \immediate\pdfximage width 0.5\hsize {./cow.pdf} \pdfrefximage\pdflastximage \input knuth \input knuth \input knuth \input knuth \input knuth \input knuth \input knuth \bye
In both cases, the problem can be fixed by adding a \hbox around the \pdfrefximage. But now I have a question for Hartmut: is this supposed to happen?
Best wishes, Taco
Many thanks for your rapid diagnosis of the real issue --- never would that have occurred to me. And apologies for pointing this initially in the wrong direction --- though I am pleased to read that it's nothing to do with libraries since that is the area I'm especially interested in and currently exploring in detail. Best wishes, and much respect! Graham
On 14-4-2010 9:45, Taco Hoekwater wrote:
Hi Graham,
Taco Hoekwater wrote:
I don't think this is supposed to happen, but please send me the pdf for a closer look and, if possible, a source to go with it that uses a lua library that is available to me. I am on linux, but lxp should work for me as well. I can even try to run the test under wine.
I've looked at your file, and it is not related to external libraries nor the luatex extensions at all: The problem is the \immediate\pdfximage.
You can get the exact same problem in pdftex with this input:
\pdfoutput=1 \immediate\pdfximage width 0.5\hsize {./cow.pdf} \pdfrefximage\pdflastximage \input knuth \input knuth \input knuth \input knuth \input knuth \input knuth \input knuth \bye
In both cases, the problem can be fixed by adding a \hbox around the \pdfrefximage. But now I have a question for Hartmut: is this supposed to happen?
definitely looks like a bug to me \setuppapersize[A4][A1] \setuplayout[location=middle,marking=on] \starttext \immediate\pdfximage width 0.5\hsize {./cow.pdf} \dorecurse {3}{\pdfrefximage\pdflastximage} \dorecurse{10}{\the\pagetotal\input knuth \par} \stoptext the ref'd image whatsit is added to the mvl but the height is not added to pagetotal (at least that's what it looks like) due the fact that no paragraph is started i assume that this bug went unnoticed as macropackages put such material normally in a box i'm not sure if it's the parbuilder or the page builder .. another test: \dorecurse {5}{\pdfrefximage\pdflastximage x\par} (i can do some more testing tommorrow) Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
On Wed, 14 Apr 2010, Hans Hagen wrote:
On 14-4-2010 9:45, Taco Hoekwater wrote:
Taco Hoekwater wrote:
I don't think this is supposed to happen, but please send me the pdf for a closer look and, if possible, a source to go with it that uses a lua library that is available to me. I am on linux, but lxp should work for me as well. I can even try to run the test under wine.
I've looked at your file, and it is not related to external libraries nor the luatex extensions at all: The problem is the \immediate\pdfximage.
You can get the exact same problem in pdftex with this input:
\pdfoutput=1 \immediate\pdfximage width 0.5\hsize {./cow.pdf} \pdfrefximage\pdflastximage \input knuth \input knuth \input knuth \input knuth \input knuth \input knuth \input knuth \bye
In both cases, the problem can be fixed by adding a \hbox around the \pdfrefximage. But now I have a question for Hartmut: is this supposed to happen?
definitely looks like a bug to me
\setuppapersize[A4][A1] \setuplayout[location=middle,marking=on]
\starttext
\immediate\pdfximage width 0.5\hsize {./cow.pdf} \dorecurse {3}{\pdfrefximage\pdflastximage} \dorecurse{10}{\the\pagetotal\input knuth \par}
\stoptext
the ref'd image whatsit is added to the mvl but the height is not added to pagetotal (at least that's what it looks like) due the fact that no paragraph is started
but it is so since ancient pdftex times (and i have also stumbled over it, and since then i'm wrapping \pdfrefximage in a \hbox, as you suggest). The \pdfrefximage doesn't do any h or v mode change, it's neutral. And i guess one should keep it like this to avoid incompatibilities. Regards, Hartmut
On 14-4-2010 8:58, Hartmut Henkel wrote:
but it is so since ancient pdftex times (and i have also stumbled over it, and since then i'm wrapping \pdfrefximage in a \hbox, as you suggest). The \pdfrefximage doesn't do any h or v mode change, it's neutral. And i guess one should keep it like this to avoid incompatibilities.
indeed, and i can imagine the why, but i can also imagine that in luatex we wrap it in an hbox as we're not bound to compatibility in luatex; it can even be conditional: if in hmode then no wrapping, else wrap it in an hbox; alternatively when issues in vmode we couldl set a flag that indicates that in the backend the image is lapped up (tlap) so that the effective height in the page stream is zero (actually that would be the cleanest solution as something similar happens with annotations) a comparison with specials is not right as specials are not looked at this way for pdftex i can imagine that a warning is issued, something: "injecting whatsit node with dimensions into a vertical list; this confuses the page builder" or something like that actually, it's a problem with all whatsits that cary dimensions so when we're further along the luatex road we can consider a special kind of hlist subtype for this kind of things Hans ----------------------------------------------------------------- Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com | www.pragma-pod.nl -----------------------------------------------------------------
participants (4)
-
Graham Douglas -
Hans Hagen -
Hartmut Henkel -
Taco Hoekwater