Python Doc & Data Automation

Fix pdfplumber Returning Empty Text

The file opens, the page count is right, and every page yields nothing:

>>> import pdfplumber
>>> with pdfplumber.open("data/invoice.pdf") as pdf:
...     print(repr(pdf.pages[0].extract_text()))
None
>>> # or, just as unhelpful:
''

extract_text() returns None when the page contains no character objects at all, and an empty or garbled string when characters exist but their font mapping cannot be resolved to Unicode. Those are different failures with different fixes, and the first job is to tell them apart.

Root Cause

pdfplumber builds text from the page's chars collection — one entry per glyph, each carrying a position, a font reference and a decoded character. Three things can empty that collection or spoil it.

The page may hold only an image, which is the case for anything scanned or exported from a photocopier. There are no character objects to collect, so chars is empty and extract_text() is None.

The page may hold text drawn with a subsetted font that ships no /ToUnicode map. The glyphs render on screen because the font program knows their shapes, but the character codes are arbitrary indices with no Unicode meaning, so extraction produces \x00\x03\x05 or mojibake rather than words.

Finally, the text may be present and decodable but positioned in a way that pdfplumber's default line-grouping tolerances reject — most often when characters are individually placed with sub-point spacing, which makes every glyph look like its own line.

Three ways a page yields no text A page is inspected for character objects. If the character count is zero, the page is an image and needs OCR. If characters exist but the font has no ToUnicode map, the extracted string is garbled and the fix is to re-extract through a font-aware library or OCR the page. If characters exist and decode correctly but are spread beyond the default grouping tolerance, the fix is to raise x_tolerance and y_tolerance. len(page.chars) the first thing to print zero garbled fine but empty Image-only page nothing to extract OCR is the only route No /ToUnicode map codes are font indices try PyMuPDF, else OCR Tolerance too tight glyphs never group raise x_ and y_tolerance Readable text out then parse as usual One printed number — the character count — separates a tooling problem from a document problem

Minimal Diagnostic

# pip install pdfplumber
from pathlib import Path
import pdfplumber

PDF = Path("data/invoice.pdf")

def text_profile(pdf_path: Path, page_number: int = 1) -> dict:
    """Distinguish a scan, a broken font map and a tolerance problem."""
    if not pdf_path.exists():
        raise FileNotFoundError(f"No such file: {pdf_path}")
    try:
        with pdfplumber.open(str(pdf_path)) as pdf:
            page = pdf.pages[page_number - 1]
            chars = page.chars
            sample = "".join(c["text"] for c in chars[:60])
            printable = sum(1 for c in sample if c.isprintable() and c != " ")
            return {
                "characters": len(chars),
                "images": len(page.images),
                "fonts": sorted({c["fontname"] for c in chars})[:5],
                "sample": sample[:60],
                "printable_ratio": round(printable / max(len(sample), 1), 2),
            }
    except Exception as exc:
        raise RuntimeError(f"Could not open {pdf_path}: {exc}") from exc

if __name__ == "__main__":
    print(text_profile(PDF))

{'characters': 0, 'images': 1, ...}                      -> scanned page
{'characters': 2140, 'printable_ratio': 0.12, ...}       -> broken font mapping
{'characters': 2140, 'printable_ratio': 0.98, ...}       -> text is fine; grouping is the issue

Fix: Loosen the Grouping Tolerances

When the characters decode correctly but extract_text() still comes back thin, the layout engine is refusing to join glyphs into words and lines. Both tolerances are measured in PDF points.

# pip install pdfplumber
from pathlib import Path
import pdfplumber

def extract_forgiving(pdf_path: Path, page_number: int = 1) -> str:
    """Extract with tolerances wide enough for loosely tracked type."""
    with pdfplumber.open(str(pdf_path)) as pdf:
        page = pdf.pages[page_number - 1]
        return page.extract_text(
            x_tolerance=2.5,   # default 3 is tight for condensed fonts; 1.5-3 is the useful band
            y_tolerance=3.5,   # raise when lines are being split mid-sentence
            layout=False,      # True preserves column spacing but inserts runs of spaces
        ) or ""

if __name__ == "__main__":
    text = extract_forgiving(Path("data/invoice.pdf"))
    print(f"{len(text)} character(s) recovered")
    print(text[:400])

Lowering x_tolerance splits words that were being glued together; raising it joins words that were being split. Because the right value depends on the font's tracking, tune it against a page whose correct output you know, then keep it fixed for that template — the same "calibrate once per template" discipline that makes extracting tables from PDFs reproducible.

Variant Fix 1: Extract Word Boxes Instead of a Text Blob

extract_words() bypasses line assembly entirely and hands back positioned tokens. When extract_text() is unreliable, words plus coordinates are often the better input anyway, because downstream parsing can use geometry rather than string search.

# pip install pdfplumber pandas
from pathlib import Path
import pandas as pd
import pdfplumber

def words_frame(pdf_path: Path, page_number: int = 1) -> pd.DataFrame:
    """Return every word with its bounding box, sorted in reading order."""
    with pdfplumber.open(str(pdf_path)) as pdf:
        page = pdf.pages[page_number - 1]
        words = page.extract_words(x_tolerance=2, y_tolerance=3, keep_blank_chars=False)
    frame = pd.DataFrame(words)
    if frame.empty:
        raise ValueError("No words on the page — this is a scan or a broken font map")
    # top ascends down the page, x0 across it: this is human reading order
    return frame.sort_values(["top", "x0"]).reset_index(drop=True)

if __name__ == "__main__":
    df = words_frame(Path("data/invoice.pdf"))
    print(df[["text", "x0", "top"]].head(12))

Loading the words into a pandas frame makes column detection a grouping problem rather than a regex problem, which is far more robust when a template shifts by a few points between issues.

Variant Fix 2: A Font with No Unicode Map

If the sample string is unprintable, pdfplumber cannot help — the mapping simply is not in the file. PyMuPDF sometimes recovers it because it consults the embedded font program's glyph names rather than only /ToUnicode.

# pip install pymupdf
from pathlib import Path
import fitz     # PyMuPDF

def extract_with_pymupdf(pdf_path: Path, page_number: int = 1) -> str:
    """Second opinion on a page whose font mapping pdfplumber cannot resolve."""
    try:
        with fitz.open(str(pdf_path)) as doc:
            return doc[page_number - 1].get_text("text")
    except Exception as exc:
        raise RuntimeError(f"PyMuPDF could not read {pdf_path}: {exc}") from exc

if __name__ == "__main__":
    text = extract_with_pymupdf(Path("data/invoice.pdf"))
    print(text[:400] if text.strip() else "PyMuPDF also found nothing — OCR the page")

When both libraries return noise, the text is unrecoverable by parsing and the page must be rasterised and read optically. That path is covered end to end in how to extract tables from scanned PDFs and, for the accuracy work that makes it viable, improve OCR accuracy with image preprocessing.

Variant Fix 3: Text Hidden Behind a Clipping Path or in an Annotation

Some generators place the real text inside form XObjects or annotations that pdfplumber does not traverse by default. Two checks cover it:

# pip install pdfplumber "pypdf>=4.2,<6"
from pathlib import Path
import pdfplumber
from pypdf import PdfReader

def hidden_text_report(pdf_path: Path, page_number: int = 1) -> dict:
    """Look for text living outside the main content stream."""
    with pdfplumber.open(str(pdf_path)) as pdf:
        page = pdf.pages[page_number - 1]
        visible = len(page.chars)
    reader = PdfReader(str(pdf_path))
    raw = reader.pages[page_number - 1].extract_text() or ""
    annots = reader.pages[page_number - 1].get("/Annots") or []
    return {
        "pdfplumber_chars": visible,
        "pypdf_text_length": len(raw.strip()),
        "annotations": len(annots),
    }

A pypdf_text_length well above zero while pdfplumber_chars is zero means the text is reachable but not where pdfplumber looked — usually inside a nested XObject. Extract with pypdf in that case and keep pdfplumber for geometry.

Escalation order for recovering text A four rung ladder ordered by cost. Rung one is tuning the extraction tolerances, which is free and instant. Rung two is switching to word boxes with coordinates, which is still free. Rung three is a second opinion from PyMuPDF, which adds a dependency. Rung four is OCR, which adds Tesseract and roughly one second per page. The advice is to stop at the first rung that returns readable text. 1. tolerances free, instant 2. word boxes free, geometric 3. PyMuPDF one more package 4. OCR ~1 s per page Stop at the first rung that returns readable text Cost rises left to right — never start at OCR for a page that has a working text layer Tuning x_tolerance against a known line The same source line, Invoice Total 1204.50, extracted at three tolerance settings. At a tolerance of one the words are split into fragments such as Inv oice. At a tolerance of two point five the words are grouped correctly. At a tolerance of six the words are glued together into one token. The useful band is between one point five and three point five. Same line, three tolerance settings x_tolerance=1 Inv | oice | To | tal | 1204 | .50 x_tolerance=2.5 Invoice | Total | 1204.50 x_tolerance=6 InvoiceTotal1204.50 Calibrate against a line whose correct output you know, then pin the value per template

Verification

Prove the recovered text is real by asserting on content you know is on the page, not merely on its length.

# pip install pdfplumber
from pathlib import Path
import pdfplumber

def assert_text_recovered(pdf_path: Path, must_contain: list[str], min_chars: int = 200) -> None:
    """Fail unless the page yields enough text, and the right text."""
    with pdfplumber.open(str(pdf_path)) as pdf:
        text = "\n".join(page.extract_text() or "" for page in pdf.pages)

    assert len(text.strip()) >= min_chars, f"only {len(text.strip())} char(s) extracted"
    printable = sum(1 for ch in text if ch.isprintable() or ch in "\n\t")
    assert printable / max(len(text), 1) > 0.95, "output is mostly non-printable — broken font map"
    missing = [needle for needle in must_contain if needle not in text]
    assert not missing, f"expected strings absent: {missing}"
    print(f"{len(text)} character(s) verified in {pdf_path.name}")

if __name__ == "__main__":
    assert_text_recovered(Path("data/invoice.pdf"), ["Invoice", "Total"])

The printable-ratio assertion is what stops a broken font map from passing silently: a page of \x03\x07\x01 has plenty of characters and would sail through a length check alone.

FAQ

Does extract_text() returning None always mean the page is scanned? Almost always, but confirm with the character count rather than assuming — a page whose text sits in an unvisited XObject also reports zero characters through pdfplumber while pypdf finds it.

Which tolerance should I change first?x_tolerance, because word joining fails more visibly than line joining. Move it in steps of 0.5 between 1.5 and 3.5, and only touch y_tolerance if single sentences are being broken across lines.

Is layout=True worth using? It preserves horizontal spacing by padding with spaces, which helps when you plan to slice fixed-width columns and hurts when you plan to split on whitespace. Choose it deliberately per template rather than globally.

Why does the same PDF extract fine on a colleague's machine? Different library versions decode some CID fonts differently. Pin pdfplumber and pdfminer.six in requirements.txt so extraction output is reproducible across machines and CI.

Can I mix pdfplumber and OCR on the same document? Yes, and it is usually the right design for mixed batches: check the character count per page, extract text-layer pages directly, and route only the image pages through OCR. That per-page routing is exactly what the scanned-PDF walkthrough builds.

A Note on Mixed Batches

Most real inboxes contain both born-digital PDFs and scans, often from the same counterparty in the same month, and a pipeline that assumes one or the other will silently mishandle the rest. Profile per page rather than per file: a fifteen-page statement can carry a scanned signature page in the middle, and routing the whole document to OCR because of it costs fourteen unnecessary seconds and some accuracy on the pages that had perfectly good text.

Part of Comparing PDF Table Extraction Libraries.