[Dev-luatex] Memory leak in string.explode()?

[Hans Hagen]

On 11/10/2012 12:02 AM, Reinhard Kotucha wrote:

> I know.  However, I just installed Subversion and copiled the latest
> SVN version of LuaTeX on my Raspberry Pi.  If you or anybody else is
> interested in benchmarks, just send me your test files.

Interesting (i have one laying around). Did you use a 'real disk' or the 
small card?

> When using the cache, I usually clear it first and then run the script
> several times.  I also obey xosview in order to make sure that no
> other processes interfere.  I think that an empty cache is what you
> have after a fresh startup.  And the most important thing is that no
> web-browser is running when doing benchmarks.

Indeed, and no thunderbird either -)

> Hmm, something similar is Taco's string.explode() function.  It's much
> faster than regular expressions, so I prefer it.  What I didn't

Right, The reason for introducing string.explode is simple splitting.

> Sure.  But as far as I can see it doesn't hurt.  The table is
> overwritten whenever a new chunk is processed.  Thus, things don't
> accumulate.  I don't know what happens when I overwrite a table.

But you still store the data twice (and I thought that you wanted to 
limit mem consumption)

> Maybe the new one allocates new memory and the old one is left to the
> garbage collector.  But if this is the case, then the garbage
> collector does a pretty good job.  The function is very fast and
> memory cunsumption isn't even visible in xosview.

it sometimes helps to do a sweep: collectgarbage("collect"), you could 
play with doing that after (say) every 5 buffer loads

> BTW, the f:read(BUFFER, '*line') concept can be less efficient if lines
> are extremely long...

inefficient anyway, just as the 'lines' method

> Well, it splits the file though:
>
>    string.gmatch(buffer,"([^\n\r]-)(\r?\n)")
>
> I suppose that the most promising approach is to use regexps in order
> to determine the linebreak style, abort, and read the file again using
> Taco's function.

yes, that sounds best (you could look at the last few characters of the 
file, assuming that the log files ends each line with a newline)

> Anyway, our discussion is obviously off-topic here.  Hans, I'll inform
> you about the results by private mail.  If anybody else is interested
> in the results, just drop me a line.

Ok.

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------