doc.save with linear=True PDF fails linearization check with 'overflow reading bit stream'

[EddieOner]

Linearization Error with linear=True Parameters

Problem Description

When saving PDF files using PyMuPDF with linear=True for web optimization, the PDF fails to linearize properly, resulting in integrity errors. These failures were initially observed in PDF proxies generated by AWS Lambda running pymudf (Python):
doc.save(output_file_path, garbage=4, clean=True, deflate=True, linear=True)

Upon further investigation using QPDF, the issue was reproducible on a local machine with pymupdf cli.

Steps to Reproduce

1. Start with a non linearized pdf and run pymupdf command:

PyMuPDF  clean -linear input.pdf output.pdf

2. check

qpdf --check output.pdf
checking output.pdf
PDF Version: 1.7
File is not encrypted
File is linearized
WARNING: output.pdf: error encountered while checking linearization data: overflow reading bit stream: wanted = 32; available = 16
qpdf: operation succeeded with warnings

4. Observe linearization failures

Expected vs Actual Behavior

Expected: PDF should save with deflate compression and linearized structure without errors.
Actual: File produces validation errors indicating invalid linearization structure.

User Impact

If the linearization isn't working correctly:

User Experience: The PDF will still download, but users won't be able to view the first page immediately. Instead, they might have to wait for the entire file to download before they can start reading.

Page Order: The pages might not load in the intended order, which can be confusing and disrupt the reading experience.

Environment Information

• pymupdf version: 1.25.2
• qpdf version: 11.9.1
• OS: Mac 15.3

Additional Context

This seems to happen with almost any pdf file

Twitter 4_linerarized.pdf
Twitter 4.pdf

How to reproduce the bug

pip install pymupdf
brew install qpdf

PyMuPDF version

1.25.2

Operating system

MacOS

Python version

3.11

[JorjMcKie]

This an upstream problem (MuPDF). We will create a report in their issue system.
You can recreate the issue without using PyMuPDF via this MuPDF CLI command:

mutool clean -lggggsz Twitter.4.pdf

Thereafter, running qpdf with the generated output PDF out.pdf shows the problem.

[JorjMcKie]

Here is the link to MuPDF's issue: https://bugs.ghostscript.com/show_bug.cgi?id=708278

[EddieOner]

@JorjMcKie Added additional content how this effects end-users :)
If the linearization isn't working correctly:

User Experience: The PDF will still download, but users won't be able to view the first page immediately. Instead, they might have to wait for the entire file to download before they can start reading.

Page Order: The pages might not load in the intended order, which can be confusing and disrupt the reading experience.