On 05/25/11 16:39, Hans Hagen wrote:

On 25-5-2011 2:43, Peter Rolf wrote:

...

Hi,

I just made a one pager (TEXpage) out of a big png graphic (5900x4094). The compressed size of the graphics is normally around 1.37MB on the highest png compress level (9) and 1.32MB after using optipng (only around 3% reduction this time). To my surprise the size of the final PDF was about 2.3MB. After adding '\pdfcompresslevel9' the size went down to 1.48MB. Still not what I wanted...

So I was wondering: is there an option to embed the png graphic as it is (no re-compression)? Otherwise the time consuming usage of optipng would be a complete waste of time. Believe it or not, but size matters :-)

Well, that depends on what optipng does to your image. PDF can do some types of png compression natively (no re-compression), but for that the png has to follow some rules: not everything in the png spec is supported that way. If you see '<png copy>' during inclusion, then the png follows those rules. Otherwise, it is included in recompressed form, where everything is possible that is allowed by png, but it will not be as small as the original. The finer details are in writepng.w from the luatex source and/or the pdf specification, it is much too detailed to repeat here. Best wishes, Taco

On Wed, 25 May 2011, Hans Hagen wrote:

On 25-5-2011 2:43, Peter Rolf wrote:

...

I just made a one pager (TEXpage) out of a big png graphic (5900x4094). The compressed size of the graphics is normally around 1.37MB on the highest png compress level (9) and 1.32MB after using optipng (only around 3% reduction this time). To my surprise the size of the final PDF was about 2.3MB. After adding '\pdfcompresslevel9' the size went down to 1.48MB. Still not what I wanted...

So I was wondering: is there an option to embed the png graphic as it is (no re-compression)?

no. There is a "PNG Copy" function for literal embedding of the PNG file, but that triggers only, if the file simultaneously satisfies quite a few conditions, which are about: non-interlaced, no palette, no transparency, no gamma coming with it, no gamma modification requested, no white adjustment in the PNG, and a few more rare others. Else it's de-compressed and then re-compressed to the \pdfcompresslevel, and additional streams and dicts are added. You see in the log if it finally was "PNG Copy" or not. Preprocessing the PNG, e. g., by convert, sometimes changes it that it gets copyable. Obviously flattening transparency also helps. Anyway direct embedding or not can have positive or negative influence on the PDF file size. E. g. if a PNG is copied verbatim, and it contains lots of meta-data info, the PDF file will probably get larger, since normal PNG embedding removes all these info chunks. Another factor influencing the size is if it's PDF-1.4 or PDF-1.5: If you have a 16 bit PNG, for PDF-1.4 it will be automatically reduced to 8 bit by luatex and pdftex, so suddenly the PDF file gets smaller, but actually also the image quality (silently) went down. These are about the factors affecting the PNG to PDF size. For your big PNG graphic you may find a preprocessing (e. g., pngtopnm | pnmtopng will definitely remove all fat) that makes it compliant with the "PNG copy".

Otherwise the time consuming usage of optipng would be a complete waste of time. Believe it or not, but size matters :-)

yes :-)

This one is for Hartmut to answer. Keep in mind that pdf does support pgn and jpg compression, which is not the same as 'inclusion as-is'.

fwiw, jpg is always embedded literally (no re-compression).

The compresslevel concerns copyright free zip compression of streams (that can happen to gave image data).

Regards, Hartmut

On Thu, May 26, 2011 at 12:52 PM, Peter Rolf wrote:

Am 25.05.2011 21:54, schrieb Hartmut Henkel:

...

On Wed, 25 May 2011, Hans Hagen wrote:

...

On 25-5-2011 2:43, Peter Rolf wrote:

...

I just made a one pager (TEXpage) out of a big png graphic (5900x4094). The compressed size of the graphics is normally around 1.37MB on the highest png compress level (9) and 1.32MB after using optipng (only around 3% reduction this time). To my surprise the size of the final PDF was about 2.3MB. After adding '\pdfcompresslevel9' the size went down to 1.48MB. Still not what I wanted...

So I was wondering: is there an option to embed the png graphic as it is (no re-compression)?

no. There is a "PNG Copy" function for literal embedding of the PNG file, but that triggers only, if the file simultaneously satisfies quite a few conditions, which are about: non-interlaced, no palette, no transparency, no gamma coming with it, no gamma modification requested, no white adjustment in the PNG, and a few more rare others. Else it's de-compressed and then re-compressed to the \pdfcompresslevel, and additional streams and dicts are added. You see in the log if it finally was "PNG Copy" or not.

Sigh, most of my graphics use (and need) transparency. So the only advantage I get from optipng is the smaller file size on my disk. Sad, but good to know. ;-)

...

Preprocessing the PNG, e. g., by convert, sometimes changes it that it gets copyable. Obviously flattening transparency also helps.

Anyway direct embedding or not can have positive or negative influence on the PDF file size. E. g. if a PNG is copied verbatim, and it contains lots of meta-data info, the PDF file will probably get larger, since normal PNG embedding removes all these info chunks.

And what about icc profiles? removed, I suppose. Not really a big problem, and doable in mkiv (see Hacked image color space. in texmf-dist/doc/pdftex/manual/samplepdf of a recent texlive)

-- luigi

Am 26.05.2011 18:17, schrieb Peter Rolf:

Am 26.05.2011 12:52, schrieb Peter Rolf:

...

Am 25.05.2011 21:54, schrieb Hartmut Henkel: [..]

...

no. There is a "PNG Copy" function for literal embedding of the PNG file, but that triggers only, if the file simultaneously satisfies quite a few conditions, which are about: non-interlaced, no palette, no transparency, no gamma coming with it, no gamma modification requested, no white adjustment in the PNG, and a few more rare others. Else it's de-compressed and then re-compressed to the \pdfcompresslevel, and additional streams and dicts are added. You see in the log if it finally was "PNG Copy" or not.

[..]

...

These are about the factors affecting the PNG to PDF size. For your big PNG graphic you may find a preprocessing (e. g., pngtopnm | pnmtopng will definitely remove all fat) that makes it compliant with the "PNG copy".

I will give that a try. But I doubt that there is much 'fat' on that graphic. Anyhow, you never know before you have tried it. :-)

No luck. I used imagemagick to convert to pnm and back. Transparency was removed before by adding a white background, also all not critical chunks (ICC profile, backgroundcolor, resolution, creation and modify date, comment) were removed. The graphic is a valid PNG (TweakPNG) and aside from the size, there is nothing special with this graphic. Still no '(PNG copy)'.

@luigi: an ICC profile definitely breaks the <png copy> rules

The only chunks left are

IHDR PNG image header: 5900x4094, 8bits/sample, truecolor, noninterlaced IDAT PNG image data .. IDAT PNG image data IEND end-of-image marker

Mh, where is the show stopper? The compression method?

Looks like some of ConTeXt PDF/X-related settings is causing this. If I reduce the code to the pure picture, the '(PNG copy)' is triggered. Probably the active color management (default color space) is breaking the copy process here. Sorry for any inconvenience. I should have tested this case before... Best wishes, Peter

Regards, Peter ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : http://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________

Am 27.05.2011 15:09, schrieb Hartmut Henkel:

...

...

@luigi: an ICC profile definitely breaks the <png copy> rules

The only chunks left are

IHDR PNG image header: 5900x4094, 8bits/sample, truecolor, noninterlaced IDAT PNG image data .. IDAT PNG image data IEND end-of-image marker

Mh, where is the show stopper? The compression method?

Looks like some of ConTeXt PDF/X-related settings is causing this. If I reduce the code to the pure picture, the '(PNG copy)' is triggered. Probably the active color management (default color space) is breaking the copy process here.

Sorry Hartmut, my last statement is complete BS. I made a 'blind' run in my lunch break, not inspecting the pdf. And sadly I had forgotten, that I changed the test file yesterday to use a small png test graphic instead of my big png. *brain vs. full stomach: 0:1* So the PDF/X settings have no influence on this. The big png is not 'copied'. Anyhow, this is not a serious problem and honestly I don't have that much time now. When I have some more time I will use gdb to find the failing condition in writepng.w. Will be interesting, the last time I used gdb is more than 10 years ago. Thanks for all answers so far. Regards, Peter

must be some \pdfimageapplygamma > 0, only this and the \pdfimagehicolor primitive can influence this low level stuff.

Btw, just \pdfimageapplygamma > 0 (without setting \pdfgamma and \pdfimagegamma) already changes the PNG image, since the luatex (and pdftex) internal defaults are not gamma-neutral. No idea if (and then to which value) this should be fixed.

Regards, Hartmut

Am 27.05.2011 17:19, schrieb luigi scarso:

On Fri, May 27, 2011 at 5:11 PM, Peter Rolf wrote:

...

Am 27.05.2011 15:09, schrieb Hartmut Henkel:

...

...

...

@luigi: an ICC profile definitely breaks the <png copy> rules

The only chunks left are

IHDR PNG image header: 5900x4094, 8bits/sample, truecolor, noninterlaced IDAT PNG image data .. IDAT PNG image data IEND end-of-image marker

Mh, where is the show stopper? The compression method?

Looks like some of ConTeXt PDF/X-related settings is causing this. If I reduce the code to the pure picture, the '(PNG copy)' is triggered. Probably the active color management (default color space) is breaking the copy process here.

Sorry Hartmut, my last statement is complete BS. I made a 'blind' run in my lunch break, not inspecting the pdf. And sadly I had forgotten, that I changed the test file yesterday to use a small png test graphic instead of my big png. *brain vs. full stomach: 0:1*

So the PDF/X settings have no influence on this. The big png is not 'copied'. Anyhow, this is not a serious problem and honestly I don't have that much time now. When I have some more time I will use gdb to find the failing condition in writepng.w. Will be interesting, the last time I used gdb is more than 10 years ago. if you have an example with a public png I can take a look...

Thanks Luigi. PM is on its way.

On Wed, May 25, 2011 at 09:40, Hans Hagen wrote:

Normally I convert such images to pdf first (using acrobat or gs) simply because inclusion of pdf is much faster.

Wow, that's odd. I've found that SVG -> PDF -> MkIV results in huge document bloat, whereas SVG -> EPS -> MkIV works much better. (Using Inkscape for both SVG conversions.) But I just experimentally preconverted all my PNGs to PDF using ImageMagick, and my document dropped from 411k to 98k. The PNGs had previously been optimized with pngnq, so they were only 99k, and are 177k when converted to PDF, so this result is surprising. Are my experiences with SVG to PDF atypical? Should I try to put together an example for debugging? mathew -- URL:http://www.pobox.com/~meta/

Am 2011-05-26 um 19:43 schrieb mathew:

Some numbers:

SVG to PDF: Two diagrams, 25k. SVG to EPS: Same two diagrams, 54k. Difference: 29k.

Document rendered using the PDFs: 535k. Document rendered using the EPSs: 463k. Difference: 72k in the opposite direction.

Be aware that ConTeXt needs to convert EPS (or SVG) to PDF before including - so providing EPS might elongate processing time. And PDF sizes depend very much on the used tools, e.g. Acrobat Distiller PDFs are often smaller than GhostScript PDFs, even if there's no downgrading of pixel images involved. Greetlings from Lake Constance! Hraban --- http://www.fiee.net/texnique/ http://wiki.contextgarden.net https://www.cacert.org (I'm an assurer)

On Thu, May 26, 2011 at 12:32, mathew wrote:

But I just experimentally preconverted all my PNGs to PDF using ImageMagick, and my document dropped from 411k to 98k. The PNGs had previously been optimized with pngnq, so they were only 99k, and are 177k when converted to PDF, so this result is surprising.

Unfortunately, it looks as if it's not as general a result as the "use EPS rather than PDF for SVG" rule. I just tried another document, and that one grew by 90k when I preconverted the PNGs to PDF (again using ImageMagick). So it seems the answer to whether PDF or PNG gives better final document size is to try both and see what happens. Not really ideal. mathew

On Wed, May 25, 2011 at 11:40 AM, Hans Hagen wrote:

On 25-5-2011 2:43, Peter Rolf wrote:

...

Hi,

I just made a one pager (TEXpage) out of a big png graphic (5900x4094). The compressed size of the graphics is normally around 1.37MB on the highest png compress level (9) and 1.32MB after using optipng (only around 3% reduction this time). To my surprise the size of the final PDF was about 2.3MB. After adding '\pdfcompresslevel9' the size went down to 1.48MB. Still not what I wanted...

Normally I convert such images to pdf first (using acrobat or gs) simply because inclusion of pdf is much faster.

I too convert images to pdf first, but my mainly because I can control the details of the conversion to get the best result for each (type of) image. Some images are well suited to jpeg compression, others are better with reduced color space and lossless compression. The ability to directly include images in <whatever>tex<t> should be seen as a convenience, but not a basis for a workflow where the final product has requirements for minimal size, color rendition, etc. There are many free image to pdf tools that all do the easy cases adequately but don't give the level of control needed for the difficult cases. SVG is different because much of it is based on a graphics model that resembles PDF. Some SVG documents translate directly to PDF, but others, e.g., markers, may "blow up" when translated. -- George N. White III Head of St. Margarets Bay, Nova Scotia