Compressing and Optimizing PDFs with Python

Automated document pipelines are very good at making large PDFs. A scanned purchase order at 600 DPI colour, a report that embeds the same 2 MB logo on every page, a merge of forty statements that each carry a full copy of the same fonts — the output hits an email gateway's 10 MB limit, fills an archive bucket, or loads so slowly in a portal that users give up. The generic advice ("run it through an online compressor") fails in automation for obvious reasons, and the generic Python snippet — re-save with compression on — often saves almost nothing, because it attacks content streams when the bytes are in images, or it destroys text quality by rasterising everything.

Compression is a measurement problem first. A PDF's size breaks down into a handful of object types, and each has a different, safe reduction technique: images are downsampled and re-encoded, fonts are subset, duplicate objects are merged, orphaned objects are dropped, and streams are recompressed. This guide measures the breakdown, applies the matching technique with PyMuPDF and pikepdf, and verifies that the smaller file still looks and behaves like the original.

Prerequisites

python -m venv .venv && source .venv/bin/activate
pip install pymupdf pikepdf pillow
# optional external engines used in the variants
sudo apt-get install -y ghostscript qpdf
mkdir -p in out

PyMuPDF handles analysis, image rewriting, font subsetting and garbage-collected saves. pikepdf (a wrapper around qpdf) adds object streams and linearisation, which PyMuPDF no longer writes. Ghostscript is the heavy option: it re-renders the entire document and can shrink files dramatically, at the cost of control. Keep a before/after test corpus that includes a scan, a chart-heavy report and a fillable form — those are the three document types where careless compression causes visible damage.

Diagnostic: Measure Where the Bytes Are

Walk every object in the file, classify it, and total the raw stream sizes. The result tells you which technique is worth applying.

# pip install pymupdf
from collections import defaultdict
from pathlib import Path
import pymupdf

SOURCE = Path("in/quarterly-pack.pdf")

def size_breakdown(pdf_path: Path) -> dict[str, int]:
    try:
        doc = pymupdf.open(pdf_path)
    except (pymupdf.FileDataError, RuntimeError) as exc:
        raise SystemExit(f"cannot open {pdf_path}: {exc}")
    totals: dict[str, int] = defaultdict(int)
    with doc:
        for xref in range(1, doc.xref_length()):
            try:
                if not doc.xref_is_stream(xref):
                    continue
                raw = doc.xref_stream_raw(xref) or b""
            except RuntimeError:
                continue                                   # free or damaged entry
            subtype = doc.xref_get_key(xref, "Subtype")[1]
            if subtype == "/Image":
                kind = "images"
            elif doc.xref_get_key(xref, "Length1")[0] != "null" or subtype in ("/Type1C", "/CIDFontType0C", "/OpenType"):
                kind = "fonts"
            elif doc.xref_get_key(xref, "Type")[1] == "/Metadata":
                kind = "metadata"
            else:
                kind = "content and other streams"
            totals[kind] += len(raw)
    totals["file on disk"] = pdf_path.stat().st_size
    return dict(totals)

if __name__ == "__main__":
    breakdown = size_breakdown(SOURCE)
    whole = breakdown["file on disk"]
    for kind, n in sorted(breakdown.items(), key=lambda kv: -kv[1]):
        print(f"{kind:>28}: {n / 1e6:7.2f} MB  {100 * n / whole:5.1f}%")

                file on disk:   24.61 MB  100.0%
                      images:   21.94 MB   89.2%
   content and other streams:    1.48 MB    6.0%
                       fonts:    1.02 MB    4.1%
                    metadata:    0.01 MB    0.0%

Almost every oversized business PDF looks like this: images dominate. Font-heavy files are the second pattern, typically merges of many documents that each embed full fonts. If the per-type totals add up to far less than the file size, the file contains orphaned objects from incremental saves — a garbage-collected save alone will fix most of it, as explained in fix PDF size grows after merging.

Core Implementation

Step 1: Downsample and Re-Encode Images

Images placed on the page at 600 DPI are wasted beyond what a screen or office printer shows. Recent PyMuPDF versions can rewrite every image above a resolution threshold in one call:

# pip install "pymupdf>=1.25"
from pathlib import Path
import pymupdf

def downsample_images(doc: pymupdf.Document, target_dpi: int = 150, quality: int = 80) -> None:
    """Downsample images displayed above 1.5x the target DPI and re-encode them as JPEG."""
    if not hasattr(doc, "rewrite_images"):
        raise RuntimeError("PyMuPDF too old for rewrite_images; upgrade or use the Pillow variant")
    doc.rewrite_images(
        dpi_threshold=int(target_dpi * 1.5),   # leave images that are already near the target alone
        dpi_target=target_dpi,
        quality=quality,                        # JPEG quality for lossy re-encoding
        lossy=True,
        lossless=True,                          # also recompress lossless (PNG-like) images
        bitonal=False,                          # keep 1-bit scans as they are (already tiny)
    )

The threshold matters more than the target. Images displayed at 160 DPI re-encoded to 150 DPI save little and cost a generation of JPEG loss; setting the threshold to 1.5 times the target skips them. For documents meant for print, 200 DPI with quality 85 is a safe floor; for screen-only portals, 110–150 DPI works.

If you are on an older PyMuPDF, the Pillow-based loop in reduce PDF file size with Python does the same thing by extracting, resizing and replacing each image.

Step 2: Subset Fonts

A full embedded CJK or Unicode font can weigh several megabytes when a document uses a few dozen glyphs from it. Subsetting keeps only the glyphs actually drawn:

# pip install pymupdf fonttools
import pymupdf

def subset_fonts(doc: pymupdf.Document) -> None:
    try:
        doc.subset_fonts()                       # requires fontTools to be installed
    except Exception as exc:                     # malformed fonts are skipped, not fatal
        print(f"font subsetting skipped: {exc}")

Do not subset fonts in fillable forms. Form fields regenerate their appearance with the embedded font when a user types, and a subset font lacks the glyphs they need; the text then renders as boxes. Detect widgets first with any(page.first_widget for page in doc) and skip this step for those files, or flatten the form before compressing, as in flatten PDF form fields with Python.

Step 3: Merge Duplicates and Drop Orphans

# pip install pymupdf
from pathlib import Path
import pymupdf

def save_compact(doc: pymupdf.Document, dest: Path) -> None:
    dest.parent.mkdir(parents=True, exist_ok=True)
    doc.save(
        dest,
        garbage=4,            # remove unused objects, merge duplicate objects and streams
        deflate=True,         # compress uncompressed streams
        deflate_images=True,
        deflate_fonts=True,
        clean=True,           # sanitise content streams
        use_objstms=1,        # pack small objects into compressed object streams (PyMuPDF 1.24+)
    )

garbage=4 compares stream contents and merges identical ones, which is what reduces the "same logo on every page" file to a single image. use_objstms compresses the many small dictionaries that make up the page tree — a meaningful saving on long, text-only documents. Remove that argument on PyMuPDF versions that reject it.

Step 4: Linearise for Web Delivery

Linearised ("fast web view") PDFs let a browser display page one before the whole file downloads. It does not shrink the file, but for portals it changes perceived speed more than any compression. pikepdf writes it, together with object streams:

# pip install pikepdf
from pathlib import Path
import pikepdf

def linearise(src: Path, dest: Path) -> None:
    try:
        with pikepdf.open(src) as pdf:
            pdf.save(
                dest,
                linearize=True,
                object_stream_mode=pikepdf.ObjectStreamMode.generate,
                compress_streams=True,
                recompress_flate=True,       # re-deflate streams at the best level
            )
    except pikepdf.PdfError as exc:
        raise RuntimeError(f"pikepdf could not rewrite {src}: {exc}") from exc

Run linearisation last. Any later save with another library discards it.

Step 5: Put It Together Conditionally

# pip install "pymupdf>=1.25" pikepdf fonttools
from pathlib import Path
import pymupdf

def compress(src: Path, dest: Path, target_dpi: int = 150) -> tuple[int, int]:
    breakdown = size_breakdown(src)                    # from the diagnostic above
    whole = breakdown["file on disk"]
    tmp = dest.with_suffix(".tmp.pdf")
    with pymupdf.open(src) as doc:
        if breakdown.get("images", 0) > 0.3 * whole:
            downsample_images(doc, target_dpi)
        has_forms = any(page.first_widget for page in doc)
        if breakdown.get("fonts", 0) > 0.1 * whole and not has_forms:
            subset_fonts(doc)
        save_compact(doc, tmp)
    linearise(tmp, dest)
    tmp.unlink(missing_ok=True)
    return whole, dest.stat().st_size

Each technique runs only when its object type is a significant share of the file. That keeps quality loss proportional to the benefit: a text-heavy contract never has its one signature image recompressed for a saving of a few kilobytes.

Edge Cases and Variants

Ghostscript for Maximum Reduction

When the goal is the smallest possible file and exact fidelity matters less — attachments for a mobile approval flow, say — Ghostscript re-renders the document with preset profiles:

# requires: sudo apt-get install ghostscript
import subprocess
from pathlib import Path

def ghostscript_compress(src: Path, dest: Path, preset: str = "/ebook") -> None:
    cmd = [
        "gs", "-sDEVICE=pdfwrite", "-dCompatibilityLevel=1.7",
        f"-dPDFSETTINGS={preset}",          # /screen 72dpi, /ebook 150dpi, /printer 300dpi
        "-dNOPAUSE", "-dQUIET", "-dBATCH", "-dDetectDuplicateImages=true",
        f"-sOutputFile={dest}", str(src),
    ]
    try:
        subprocess.run(cmd, check=True, capture_output=True, timeout=300)
    except FileNotFoundError:
        raise RuntimeError("ghostscript (gs) is not installed")
    except subprocess.CalledProcessError as exc:
        raise RuntimeError(f"gs failed: {exc.stderr.decode(errors='replace')[:400]}") from exc

Ghostscript rewrites everything: it drops form fields unless told otherwise, can change colour spaces, occasionally loses tagged structure, and changes text extraction output. Never use it on files that must stay fillable or accessible.

Scanned Documents

Colour scans of black-and-white paperwork are the largest wins. Converting to grayscale or bitonal before JPEG encoding typically divides size by five to ten. Tools like OCRmyPDF apply this with JBIG2 encoding and add a text layer at the same time — the approach in make scanned PDFs searchable with OCRmyPDF with --optimize 3.

Archival Output

PDF/A forbids some of the techniques above (object streams in PDF/A-1, transparency, unembedded fonts) and requires others. Compress first, then convert, then validate; the order and settings are in convert PDF to PDF/A for archiving.

Setting a Size Budget per Destination

"As small as possible" is not a requirement anyone can test. Each destination for your PDFs has a concrete limit and a quality floor, and the compression settings should follow from those rather than from a single global preset. Keep the budgets in configuration and escalate settings step by step until the file fits, stopping at the quality floor.

# pip install "pymupdf>=1.25"
from pathlib import Path
import pymupdf

LADDER = [(200, 85), (150, 80), (120, 72), (100, 65)]      # (dpi, quality), mildest first

def fit_budget(src: Path, dest: Path, budget_bytes: int) -> tuple[int, int] | None:
    """Try progressively stronger image settings; return the (dpi, quality) that fits, or None."""
    for dpi, quality in LADDER:
        with pymupdf.open(src) as doc:
            doc.rewrite_images(dpi_threshold=int(dpi * 1.5), dpi_target=dpi, quality=quality)
            doc.save(dest, garbage=4, deflate=True)
        if dest.stat().st_size <= budget_bytes:
            return dpi, quality
    dest.unlink(missing_ok=True)
    return None                                             # floor reached: split or send a link

When the ladder runs out, stop compressing. Splitting the document — by section or by customer, as in split a PDF by page ranges — or sending a download link preserves quality that further squeezing would destroy. Log the rung each file needed; a supplier whose scans always need the last rung is a conversation worth having about scanner settings.

Validation

A compressed file must render the same pages, carry the same text, and keep its interactive parts. Compare page count, extracted text, form fields, and a pixel difference of low-resolution renders.

# pip install pymupdf pillow
from pathlib import Path
import pymupdf
from PIL import Image, ImageChops, ImageStat

def render(page: pymupdf.Page, dpi: int = 50) -> Image.Image:
    pix = page.get_pixmap(dpi=dpi)
    return Image.frombytes("RGB", (pix.width, pix.height), pix.samples)

def verify_compression(original: Path, compressed: Path, max_diff: float = 6.0) -> None:
    with pymupdf.open(original) as a, pymupdf.open(compressed) as b:
        assert a.page_count == b.page_count, "page count changed"
        a_fields = sorted(w.field_name for p in a for w in p.widgets())
        b_fields = sorted(w.field_name for p in b for w in p.widgets())
        assert a_fields == b_fields, "form fields changed"
        for pa, pb in zip(a, b):
            ta, tb = " ".join(pa.get_text().split()), " ".join(pb.get_text().split())
            assert ta == tb, f"page {pa.number + 1}: extracted text differs"
            diff = ImageChops.difference(render(pa), render(pb))
            mean = sum(ImageStat.Stat(diff).mean) / 3
            assert mean <= max_diff, f"page {pa.number + 1}: visual difference {mean:.1f}"
    saved = 1 - compressed.stat().st_size / original.stat().st_size
    print(f"verified; {saved:.0%} smaller")

if __name__ == "__main__":
    verify_compression(Path("in/quarterly-pack.pdf"), Path("out/quarterly-pack.pdf"))

Rendering at 50 DPI makes the pixel comparison fast and tolerant of JPEG artefacts while still catching missing images, font substitution and colour shifts. A mean channel difference under about six on a 0–255 scale is visually identical; raise the threshold for aggressive image settings and review the pages that exceed it.

Performance and Scale Notes

Image rewriting is CPU-bound and proportional to image pixels, not page count: a 20-page scan at 600 DPI costs more than a 400-page text report. Run one document per process with ProcessPoolExecutor, and set a per-file timeout — Ghostscript in particular can spend minutes on pathological vector artwork. Memory peaks while decoding the largest image; very large scans (A0 plans, 1200 DPI) may need tens of megabytes each, which is only a problem with many workers on a small container. Write to a temporary name and rename on success so a crashed worker never leaves a truncated PDF where downstream jobs will pick it up. For archives of existing files, record the before and after sizes per file; skip files where the saving is under a few percent and keep the original bytes, because recompressing gains nothing and loses the ability to prove the file is unchanged.

Troubleshooting

Complete Working Script

#!/usr/bin/env python3
# pip install "pymupdf>=1.25" pikepdf fonttools pillow
"""Measure, compress and verify every PDF in a folder."""
import argparse
import logging
import sys
from pathlib import Path

import pikepdf
import pymupdf

log = logging.getLogger("compress")


def image_share(doc: pymupdf.Document, file_size: int) -> float:
    total = 0
    for xref in range(1, doc.xref_length()):
        try:
            if doc.xref_is_stream(xref) and doc.xref_get_key(xref, "Subtype")[1] == "/Image":
                total += len(doc.xref_stream_raw(xref) or b"")
        except RuntimeError:
            continue
    return total / file_size if file_size else 0.0


def compress_one(src: Path, dest: Path, dpi: int, quality: int) -> tuple[int, int]:
    size = src.stat().st_size
    tmp = dest.with_name(dest.stem + ".part.pdf")
    dest.parent.mkdir(parents=True, exist_ok=True)
    with pymupdf.open(src) as doc:
        if doc.needs_pass:
            raise PermissionError("encrypted")
        if image_share(doc, size) > 0.3 and hasattr(doc, "rewrite_images"):
            doc.rewrite_images(dpi_threshold=int(dpi * 1.5), dpi_target=dpi, quality=quality)
        if not any(page.first_widget for page in doc):
            try:
                doc.subset_fonts()
            except Exception as exc:
                log.debug("%s: subsetting skipped (%s)", src.name, exc)
        doc.save(tmp, garbage=4, deflate=True, deflate_images=True, deflate_fonts=True, clean=True)
    with pikepdf.open(tmp) as pdf:
        pdf.save(dest, linearize=True, object_stream_mode=pikepdf.ObjectStreamMode.generate)
    tmp.unlink(missing_ok=True)
    return size, dest.stat().st_size


def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("src", type=Path)
    ap.add_argument("out", type=Path)
    ap.add_argument("--dpi", type=int, default=150)
    ap.add_argument("--quality", type=int, default=80)
    ap.add_argument("--min-saving", type=float, default=0.05, help="keep original below this saving")
    args = ap.parse_args()
    logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")

    failures = 0
    for src in sorted(args.src.glob("*.pdf")):
        dest = args.out / src.name
        try:
            before, after = compress_one(src, dest, args.dpi, args.quality)
            if 1 - after / before < args.min_saving:
                dest.write_bytes(src.read_bytes())          # not worth it: keep original bytes
                log.info("%s: saving under threshold, original kept", src.name)
            else:
                log.info("%s: %.1f MB -> %.1f MB", src.name, before / 1e6, after / 1e6)
        except Exception as exc:
            failures += 1
            log.error("%s: %s", src.name, exc)
    return 1 if failures else 0


if __name__ == "__main__":
    sys.exit(main())

Frequently Asked Questions

What DPI should I target? 150 DPI for on-screen reading, 200–300 DPI for documents that will be printed. Scanned text stays legible down to about 150 DPI in grayscale.

Is JPEG safe for scanned text? At quality 75–85 it is legible but adds halo artefacts around letters. For bitonal paperwork, lossless CCITT or JBIG2 encodings are both smaller and sharper; OCRmyPDF applies them automatically.

Why is a PDF produced by Word so much bigger than the same content from LibreOffice? Differences in image handling and font embedding settings. Word's "minimum size" export option and LibreOffice's image compression settings change the result far more than post-processing does — fix it at the source when you control the export.

Does compression break digital signatures? Yes. Any rewrite invalidates a signature, which covers exact bytes. Compress before signing, never after.

Related

• Reduce PDF File Size with Python — a focused script for image-heavy files, including older PyMuPDF
• Fix PDF Size Grows After Merging — duplicate fonts and images across merged inputs
• Convert PDF to PDF/A for Archiving — long-term storage requirements
• Merging and Splitting PDF Documents — where oversized files usually come from
• Email Generated Reports with Python — attachment limits that make compression necessary

Part of Automating PDF Extraction & Generation.