Python Doc & Data Automation

Make Scanned PDFs Searchable with OCRmyPDF

A shared drive holds eight years of scanned contracts, delivery notes and HR forms. Nobody can search them, and the extraction jobs that work on digital PDFs return empty strings for every one. The first attempt — render each page with PyMuPDF, run Tesseract, write the text to a sidecar file — produces text, but readers still cannot search inside the PDFs, the sidecar drifts from the file when documents are renamed, and the batch dies on the first file that already contains text with ocrmypdf.exceptions.PriorOcrFoundError: page already has text!.

Root Cause

A scanned PDF is a sequence of page images. Making it searchable means adding a text layer: OCR'd words drawn with an invisible rendering mode at the exact positions of the corresponding pixels, so viewers can find, select and copy text while displaying the original scan. Doing that correctly by hand requires mapping every OCR word box into PDF coordinates, embedding a font, handling rotated pages, and preserving existing content. OCRmyPDF does all of this — it is a mature wrapper around Tesseract with page rendering, rotation, deskewing, PDF/A output and image optimisation built in — but its defaults are deliberately cautious: it refuses to process pages that already contain text, so mixed batches of scanned and digital files stop on the first digital one. The job is to choose the right mode for each file type and handle the exceptions it uses to report those situations.

Minimal Diagnostic

Before OCRing a folder, classify files: fully scanned, fully digital, mixed, encrypted, or signed. Each class needs different OCRmyPDF options — or none.

# pip install pymupdf "pandas>=2.2"
from pathlib import Path
import pandas as pd
import pymupdf

ARCHIVE = Path("archive/contracts")

def classify(path: Path) -> dict:
    try:
        doc = pymupdf.open(path)
    except (pymupdf.FileDataError, RuntimeError) as exc:
        return {"file": path.name, "class": f"unreadable: {exc}"}
    with doc:
        if doc.needs_pass:
            return {"file": path.name, "class": "encrypted"}
        signed = any(w.field_type == pymupdf.PDF_WIDGET_TYPE_SIGNATURE for p in doc for w in p.widgets())
        text_pages = sum(1 for p in doc if len(p.get_text().strip()) > 50)
        n = doc.page_count
    kind = "digital" if text_pages == n else "scanned" if text_pages == 0 else "mixed"
    return {"file": path.name, "pages": n, "text_pages": text_pages, "class": kind, "signed": signed}

if __name__ == "__main__":
    report = pd.DataFrame(classify(p) for p in sorted(ARCHIVE.rglob("*.pdf")))
    print(report["class"].value_counts().to_string())
    print("signed:", int(report.get("signed", pd.Series(dtype=bool)).fillna(False).sum()))

class
scanned      1842
mixed         211
digital       590
encrypted      14
signed: 37

Most files are scans, but a fifth are mixed (a scanned contract with a digitally generated cover page) and many are fully digital. Encrypted files cannot be processed without a password, and signed files must not be modified at all.

OCRmyPDF mode per file class The root classifies each file. Fully scanned files run with default OCR plus rotate pages and deskew. Mixed files use skip text so pages that already have text are left untouched and only image pages are OCRed. Fully digital files are skipped entirely. Encrypted files need decryption first. Digitally signed files are copied unchanged because any rewrite invalidates the signature. What kind of PDF is it? classify text pages, encryption, signature scanned Full OCR rotate_pages, deskew mixed skip_text OCR image pages only digital Skip file already searchable signed / encrypted Do not modify copy or decrypt first

Fix: Run OCRmyPDF from Python with the Right Mode per File

Call OCRmyPDF's Python API, choose skip_text for mixed files, rotate and deskew scans, and translate its exceptions into per-file outcomes. Changed lines carry comments.

# pip install ocrmypdf
# system: sudo apt-get install -y tesseract-ocr tesseract-ocr-deu ghostscript
import shutil
from pathlib import Path
import ocrmypdf
from ocrmypdf import exceptions as ox

def make_searchable(src: Path, dest: Path, languages: str = "eng") -> str:
    dest.parent.mkdir(parents=True, exist_ok=True)
    try:
        ocrmypdf.ocr(
            src, dest,
            language=languages,                    # changed: e.g. "eng+deu" for bilingual archives
            skip_text=True,                        # changed: OCR image-only pages, keep digital pages as-is
            rotate_pages=True,                     # changed: fix sideways and upside-down scans
            deskew=True,                           # changed: straighten slightly crooked feeds
            clean=False,                           # clean needs unpaper; enable only if installed
            optimize=1,                            # lossless size optimisation
            output_type="pdf",                     # "pdfa" for archives, see the variant below
            progress_bar=False,
            jobs=2,                                # changed: cores per file; parallelise files separately
        )
        return "ocr"
    except ox.PriorOcrFoundError:
        shutil.copy2(src, dest)                     # changed: already searchable, keep original
        return "already-text"
    except ox.EncryptedPdfError:
        return "encrypted"
    except ox.DigitalSignatureError:
        shutil.copy2(src, dest)                     # changed: never rewrite signed documents
        return "signed-unchanged"
    except ox.MissingDependencyError as exc:
        raise SystemExit(f"install the missing tool first: {exc}")
    except ox.ExitCodeException as exc:
        return f"failed: {type(exc).__name__}: {exc}"

if __name__ == "__main__":
    print(make_searchable(Path("archive/contracts/2019/C-0412.pdf"),
                          Path("searchable/contracts/2019/C-0412.pdf"), "eng+deu"))

skip_text=True is the key option for real archives: it lets one command handle scanned and mixed files without re-OCRing pages that already carry real text. --force-ocr is the opposite — it rasterises every page and replaces all text, which destroys the quality of digital pages and should be reserved for files whose existing text layer is garbage, as in fix CID garbled characters in PDF text. Rotation uses Tesseract's orientation detection, so the output PDF displays upright pages rather than just producing upright text.

What a searchable scan contains The page keeps the original scanned image, optionally optimised. Above it OCRmyPDF adds an invisible text layer drawn in render mode 3 with a glyphless font at the positions of recognised words. The page rotation is corrected when rotate pages is on. Document metadata records the OCR software and, for PDF/A output, an output intent and XMP metadata. Page metadata and rotation /Rotate fixed by rotate_pages what the viewer uses to orient the page Invisible text layer render mode 3, glyphless font search and copy read this Original scan image kept, optionally optimised what readers see Document metadata producer, PDF/A intent if requested archives validate this

Variant Fix 1: Batch Processing with Parallelism and a Report

OCR is CPU-heavy. Parallelise across files with a process pool, give each OCRmyPDF call a small number of threads, and record every outcome so the batch can be resumed and audited.

# pip install ocrmypdf "pandas>=2.2"
import os
import time
from concurrent.futures import ProcessPoolExecutor, as_completed
from pathlib import Path
import pandas as pd

SRC_ROOT, DEST_ROOT = Path("archive"), Path("searchable")

def _one(path_str: str) -> dict:
    src = Path(path_str)
    dest = DEST_ROOT / src.relative_to(SRC_ROOT)
    if dest.exists() and dest.stat().st_mtime >= src.stat().st_mtime:
        return {"file": str(src), "status": "up-to-date", "seconds": 0.0}
    start = time.perf_counter()
    status = make_searchable(src, dest, "eng+deu")
    return {"file": str(src), "status": status, "seconds": round(time.perf_counter() - start, 1)}

def run_batch(workers: int | None = None) -> pd.DataFrame:
    files = [str(p) for p in sorted(SRC_ROOT.rglob("*.pdf"))]
    workers = workers or max(1, (os.cpu_count() or 2) // 2)
    rows = []
    with ProcessPoolExecutor(max_workers=workers) as pool:
        futures = {pool.submit(_one, f): f for f in files}
        for future in as_completed(futures):
            try:
                rows.append(future.result())
            except Exception as exc:
                rows.append({"file": futures[future], "status": f"crashed: {exc}", "seconds": None})
    report = pd.DataFrame(rows).sort_values("file")
    report.to_csv(DEST_ROOT / "ocr-report.csv", index=False)
    return report

Skipping files whose output is newer than the source makes reruns cheap after an interruption. With jobs=2 per file and half the CPU cores as workers, total thread count roughly matches the machine; oversubscribing cores slows everything down because Tesseract and Ghostscript compete for the same CPU.

Variant Fix 2: Archival Output and Smaller Files

For long-term storage, produce PDF/A and optimise images more aggressively when the scans are colour images of black-and-white paperwork:

# pip install ocrmypdf
# system: sudo apt-get install -y pngquant jbig2enc   (optional, improves optimisation)
import ocrmypdf

ocrmypdf.ocr(
    "archive/hr/2017-contract.pdf", "searchable/hr/2017-contract.pdf",
    language="eng",
    skip_text=True, rotate_pages=True, deskew=True,
    output_type="pdfa-2",              # archival profile with embedded fonts and colour intent
    optimize=3,                        # aggressive: JBIG2 for bitonal, pngquant for colour
    jbig2_lossy=False,                 # keep lossless JBIG2; lossy can alter characters
    progress_bar=False,
)

optimize=3 can shrink colour scans of printed pages by an order of magnitude when pngquant and jbig2enc are installed. Keep jbig2_lossy=False for legal and financial records: lossy JBIG2 substitutes similar-looking symbols and has historically changed digits in scanned documents. Validation of the archival output is covered in convert PDF to PDF/A for archiving.

Output size by optimisation level For 100 colour scans of printed contracts totalling 612 megabytes, OCRmyPDF output was 618 megabytes with optimisation off because the text layer adds a little, 541 megabytes with lossless optimisation level 1, 212 megabytes with level 2, and 74 megabytes with level 3 using lossless JBIG2 and pngquant. 100 colour scans of printed contracts, 612 MB input optimize=0 618 MB optimize=1 (lossless) 541 MB optimize=2 212 MB optimize=3 + JBIG2 lossless 74 MB

Running OCRmyPDF in a Container

OCRmyPDF depends on Tesseract, Ghostscript and optionally unpaper, pngquant and jbig2enc, with version requirements between them. Installing those consistently on every server is the main source of batch failures, so run the job from an image that bundles them. The project publishes one, and extending it with your language packs and scripts keeps everything pinned together:

FROM jbarlow83/ocrmypdf:latest
USER root
# extra languages used by the archive
RUN apt-get update \
 && apt-get install -y --no-install-recommends tesseract-ocr-deu tesseract-ocr-fra \
 && rm -rf /var/lib/apt/lists/*
COPY batch_ocr.py /app/batch_ocr.py
ENTRYPOINT ["python3", "/app/batch_ocr.py"]

docker build -t archive-ocr .
docker run --rm \
  -v /srv/archive:/archive:ro \
  -v /srv/searchable:/searchable \
  --cpus 6 \
  archive-ocr

Mounting the source read-only makes it impossible for a bug to overwrite originals, and --cpus stops a large backfill from starving other services on the same host. Pin the image by digest rather than latest once the batch has been validated, so a later rebuild cannot change OCR output silently halfway through an eight-year backfill. MissingDependencyError in the Python code above then only appears in environments that bypass the image — a useful signal that someone is running the job the wrong way.

Verification

Check that each output is searchable, that page count and page sizes match the input, and that digital pages kept their original text rather than being replaced by OCR.

# pip install pymupdf
from pathlib import Path
import pymupdf

def verify_searchable(src: Path, dest: Path, min_chars_per_page: int = 40) -> None:
    with pymupdf.open(src) as a, pymupdf.open(dest) as b:
        assert a.page_count == b.page_count, "page count changed"
        for pa, pb in zip(a, b):
            ra, rb = pa.rect, pb.rect
            same_size = abs(ra.width - rb.width) < 2 and abs(ra.height - rb.height) < 2
            swapped = abs(ra.width - rb.height) < 2 and abs(ra.height - rb.width) < 2   # rotate_pages
            assert same_size or swapped, f"page {pa.number + 1}: size changed"
            original = " ".join(pa.get_text().split())
            ocr = " ".join(pb.get_text().split())
            if len(original) > 50:
                assert original == ocr, f"page {pa.number + 1}: digital text was replaced"
            elif pb.get_images():
                assert len(ocr) >= min_chars_per_page, f"page {pa.number + 1}: no OCR text added"
    print(f"{dest.name}: searchable, digital pages preserved")

Sample a few outputs by eye too: search for a word you can see on a scanned page, and check the highlight lands on it. Misaligned highlights mean rotation or cropping went wrong even though the text itself is correct.

FAQ

Does OCRmyPDF change how the pages look? Not with skip_text and default optimisation — the scan image stays as it was. deskew and clean-final do alter the image; clean alone only affects the OCR input.

Which Tesseract languages are installed? Run tesseract --list-langs. Install more with the system package manager, as described in OCR non-English documents with Tesseract.

Can I get the recognised text as a separate file too? Pass sidecar="out/file.txt" to write the plain text alongside the PDF, useful for indexing without opening PDFs.

Why is OCR slow on some files? High-DPI colour scans and huge page sizes (A0 drawings) dominate. OCRmyPDF renders at the image's native resolution; downsample oversized scans first if accuracy allows.

Part of Scanning and OCR Processing with Python.