Re: [NTG-context] ConTeXt unofficial test suite stub

2012-10-17 Marco Pessotto:

luigi scarso writes:

...

It's not a huge difference, but it can integrated better with mkiv; see http://meeting.contextgarden.net/2011/talks/day1_05_luigi_graphicmagick/ (if you like these kind of things)

I think I'll have to run some benchmark to see if it can save some resources and speed up the testing. I don't care too much for the mkiv integration. The other Marco's idea was exactly to make conTeXt run the tests.

I removed that part, it was a stupid idea. At the moment it's a pure Lua solution. It's divided into two parts. The first part is the database update, Lua runs context on the tests and writes the results to a database (a Lua table stored in a file). For the second part context is run on the database to generate the report from the database. With this database you can display information like “What broke in the last beta”.

Looking at the diffs, many of the differences are really tiny, but differences exist nevertheless. It will take me some time to sort out which ones are just tiny space shifting, and which ones give really different output.

That's why I use a header for each test file. You can specify “sensitivity=low|medium|high”, this sets a predefined tolerance for the page difference. I don't see any other option. There is no single value which is sensitive enough, for instance to detect if a footnote is subscripted or not and at the same time tolerant enough not to always mark a fifty page document as “failed”. Marco