However, your statement seems to contradict your earlier statement (snipped) that one can do Source transformed by LuaTeX to DVI and hence PDF and get the same result as Source transformed directly to PDF by LuaTeX
I didn't write that you would get the same result. But in most cases, it should still work, though. Again, it all depends on what kind of information you want to put it your output file, so you should be more specific.
From what you've said, it seems to me that xdv contains some but not all of the features that LuaTeX would need to add to the DVI format. Are there many such features?
I couldn't say for sure, but if you go back to my vertical text sample, you can see an essential feature that DVI is lacking, as well as xdv: the ability to include arbitrarily modified metrics for any font. In my example, I want to modify the dimensions of glyphs in a font, much like you would do with a virtual font. LuaTeX allows us to emulate this behaviour by using the define_font callback, without going through an actual .vf file. You need, roughly, to produce a Lua hash table in which you put new widths, heights and depths for all the glyphs in that font. This only works to a certain extent in DVI mode, because, then, I would like to modify the vertical placement of some glyphs, and you can't include that information in the DVI. Inside LuaTeX, all that information is contained in your Lua code, but it is lost once you write the DVI file. Needless to say, it would also be lost in the xdv format. In short, this demontrates that DVI and extensions lack an essential feature of LuaTeX, which is, not very surprisingly, Lua itself. Should we go on adding the possibility of including arbitrary Lua code in a DVI file? I shouldn't think so. The way I see it, Lua is part of the typesetting engine of LuaTeX, not the end format (whichever one you use), and whatever you do during the typesetting part should be discarded in the backend, and transformed into the appropriate form (in my example, by writing out the modified metrics in the PDF file, and shifting the glyphs in the embedded font). Incidentally, you can probably still try and include arbitrary Lua code in a DVI file, by using \special's. You can then produce a DVI consumer application that would read and execute that code (for example, by modifying xdvipdfmx). But if you go on producing PDF instead, everything is already there for you. Arthur