pdfly extract-images is not lossless

[georgthegreat]

Despite being advertised this way, the tool actually recompresses images turning jpegs with estimated quality of 99 into jpegs with estimated quality of 75.

Can be cross-validated using pdfimages tool from xpdf suite.

[lkintact]

I can confirm the issue, tested on Windows 10 Home 22H2, with pdfly 0.5.1.

[Lucas-C]

Hi @georgthegreat, @lkintact

Could you please specify how you measured & tested that?
Is it reproducible using a PDF in this repository, like https://github.com/py-pdf/pdfly/blob/main/resources/jpeg.pdf, or some other image/PDF that you can share?

Is it possible that this compression issue could come from the software used to insert the image in the PDF?

I'm asking because pdfly only dumps the images binary data into files,
it does not use any image processing library like e.g. Pillow.

If there is actually an issue with pdfly regarding this, I'm really interested in fixing it.

Thank you for bringing up this topic 👍

[lkintact]

Could you please specify how you measured & tested that?

I tested as follows:

1. Convert this image (katsura.jpg, 855,668 bytes, quality 94) to PDF with
   pdfly x2pdf -o katsura.pdf katsura.jpg.
2. Extract the image from the PDF with
   pdfly extract-images katsura.pdf.
   The resulting image, 0-I1.jpg, is 384,086 bytes and of quality 75.

On the other hand, if for image extraction I use the pdfimages tool from Xpdf like this: pdfimages -j katsura.pdf output, the resulting image, output-0000.jpg, is identical to the original katsura.jpg.

[georgthegreat]

pdfimages -j katsure.pdf ext gives byte-to-byte identical image (the tool originates from xpdf package).

[Lucas-C]

Thank you for your help, and the precise commands you provided.

I identified the issue and reported it to pdfly there: py-pdf/pypdf#3515

[georgthegreat]

I think that even touching these images with Pillow is an antipattern.
As extract-images --help claims:

Extract images from PDF without resampling or altering.

We should be able to extract images that are bitwise identical to the version we embedded.
i. e. the following test should pass (pseudocode):

subprocess.check_call("pdfly x2pdf -o out.pdf in.jpg")
subprocess.check_call("pdfly extract-images out.pdf")
in_bytes = Path("in.jpg").read_bytes()
out_bytes = Path("01-dfdsfs.jpg").read_bytes()
assert in_bytes == out_bytes

[stefan6419846]

Maybe the description of the command is wrong in this case, but looking at it from the backend library (pypdf), there currently are no plans to replace the current mostly Pillow-based approach for image handling. Retrieving images in the correct manner already is a rather complex task, considering all types of masks etc. When testing different tool combinations for embedding and retrieving images into/from PDF files (especially considering masks and alpha channels), every tool had its own limitations and oddities. Thus, the correct tool of choice might differ depending on your use case, even if all of them somehow extract the images from a PDF file.

With this being said, the quality being changed is something I consider a bug on the pypdf side. You are the first one to report this and it seems like the Pillow documentation is not directly obvious about this. Lucas has started proposing a corresponding PR, which I am open to merge once the necessary aspects are discussed and cleared.

[Lucas-C]

My PR in pypdf has been merged: py-pdf/pypdf#3515

@georgthegreat: you can check that if fixes your original issue by installing the latest version of pypdf from the tip of the repo main branch: pip install git+https://github.com/py-pdf/pypdf.git@main
And then invoke pdfly extract-images, that should now be lossless.

[georgthegreat]

Well, it does not:

(dflt) thegeorg@i112760328:/mnt/c/Temp/books/repro$ ls
0000.jpg
(dflt) thegeorg@i112760328:/mnt/c/Temp/books/repro$ ~/repo/dancebooks/tools/pdf.py convert
Will not change image size during generation
(dflt) thegeorg@i112760328:/mnt/c/Temp/books/repro$ ls
0000.jpg  output.pdf
(dflt) thegeorg@i112760328:/mnt/c/Temp/books/repro$ pdfly extract-images output.pdf
Extracted 1 images:
- 0000-I1.jpg
(dflt) thegeorg@i112760328:/mnt/c/Temp/books/repro$ md5sum *
01b4dd18597cdbb6ca8ebd113a9e87ee  0000-I1.jpg
6412f7c2134216695b063b43403829be  0000.jpg
f215374877e583b530a451ac87da591a  output.pdf
(dflt) thegeorg@i112760328:/mnt/c/Temp/books/repro$ pdfimages -j output.pdf ext
(dflt) thegeorg@i112760328:/mnt/c/Temp/books/repro$ md5sum *
01b4dd18597cdbb6ca8ebd113a9e87ee  0000-I1.jpg
6412f7c2134216695b063b43403829be  0000.jpg
6412f7c2134216695b063b43403829be  ext-000.jpg
f215374877e583b530a451ac87da591a  output.pdf

As you may see, ext-000.jpg (extracted by pdfimages from xpdf suite) is bitwise identical to the image embedded with pdf.py (being a tiny fpdf2 wrapper: https://github.com/hda-technical/dancebooks/blob/main/tools/pdf.py).