# [Dev-luatex] luafontloader virtual access (bug #404, #448)

Taco Hoekwater taco at elvenkind.com
Sun Sep 5 17:08:43 CEST 2010

Hi,

It has been a busy day.

On 09/05/2010 04:39 PM, taco at metatex.org wrote:
> New Revision: 3857
>
> Log:

Here is the relevant new section in the manual:

%%% start manual

As mentioned earlier, the return value of \type{fontloader.open()} is a
userdata object. In \LUATEX\ versions before 0.63, the only way to have
on this object, returning a table structure that is explained in the
following subsections.

However, it turns out that the result from \type{fontloader.to_table()}
sometimes needs very large amounts of memory (depending on the font's
complexity and size) so starting with \LUATEX\ 0.63, it will
incrementally become possible to access the userdata object directly.

In the \LUATEX\ 0.63.0, the following is implemented:

\startitemize
\item all top-level keys that would be returned by \type{to_table()} can
also be accessed directly.
\item the top-level key \quote{glyphs} returns a {\it virtual\/} array
that allows indices from \type{0} to ($\type{f.glyphmax}-1$).
\item the items in that virtual array (the actual glyphs) are themselves
also userdata objects, and each has accessors for all of the keys
explained in the section \quote{Glyph items} below.
\item the top-level key \quote{subfonts} returns an {\it actual} array
of userdata objects, one for each of the subfonts (or nil, if there are
no subfonts).
\stopitemize

A short example may be helpful. This code generates a printout of all
the glyph names in the font \type{PunkNova.kern.otf}:

\starttyping
print (f.fontname);
local i = 0
while (i < f.glyphmax) do
local g = f.glyphs[i]
if g then
print(g.name)
end
i = i + 1
end
\stoptyping

In this case, the \LUATEX\ memory requirement stays below 100MB on the
test computer, while the internal stucture generated by
\type{to_table()} needs more than 2GB of memory (the font itself is
6.9MB in disk size).

In \LUATEX\ 0.63 only the top-level font, the subfont table entries, and
the glyphs are virtual objects, everything else still produces normal
lua values and tables. In future versions, more return values will be
replaced by userdata objects (as much as needed to keep the
memory requirements in check).

%%% end manual

For course the memory savings will be less when more of the font
information is used than just the glyph names, but it does seem to help
quite a lot already (assuming the characters are converted into a
metrics structure one at a time).

The reason for this message is this: what are items that need to be
virtualized, and which ones can easily be left alone?

For example

<glyph>.boundingbox

returns an array of 4 integer numbers. It seems to me that it makes
not that much sense to write a dedicated userdata object for such
boundingboxes: the method call overhead will probably outweigh the
gain from using less memory.

On the other hand,

<glyph>.kerns

can be enormous (as it is in the punknova.kern case) and should probably
be converted.

Does anybody want to think about a shortlist of such items?

Best wishes,
Taco