Python Doc & Data Automation

Flatten PDF Form Fields with Python

A filled PDF form is still a form. The recipient can click into any box, retype the amount, and save a file that looks identical to yours:

>>> from pypdf import PdfReader
>>> PdfReader("out/signed_agreement.pdf").get_fields().keys()
dict_keys(['applicant_name', 'contract_value', 'agreed', 'signature_date'])
# every one of these is still editable in any viewer

Flattening removes that. It merges each widget's appearance stream into the page's content stream — turning the values into ordinary ink — and deletes the /AcroForm dictionary so no field objects remain. The result is visually identical and functionally frozen.

Root Cause of the "Editable After Sending" Problem

Nothing has gone wrong; this is simply what filling does. Filling a form writes /V on the field and an appearance stream on the widget, both of which live in the interactive layer. Viewers keep that layer live because it is, by design, an input control. Read-only flags help but are advisory: they instruct compliant viewers to refuse edits, while a determined user with any PDF library can clear the flag in three lines.

What flattening moves, and what it deletes Before flattening, a page has two layers. The lower layer is the page content stream holding the static artwork. The upper layer is the interactive form layer holding widget annotations with their appearance streams and field values, and it is editable. After flattening, the appearance streams have been drawn into the page content stream and the interactive layer is gone, so the page holds one layer of static ink and no field objects remain. Before: two layers After: one layer Interactive form layer widgets, /V values, /AP editable by the recipient Page content stream rules, labels, artwork already static flatten draw + delete Merged content stream values are now ink no /AcroForm remains Field names are gone extract data before flattening Flattening is one-way Keep the filled, unflattened file if you ever need to re-read the values by name

Minimal Diagnostic: Is This File Actually Flat?

Before and after any flattening step, ask the file three questions. A genuinely flat document has no form dictionary, no widget annotations, and no field names.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader

def flatness_report(pdf_path: Path) -> dict:
    """Report the three markers that distinguish a flat PDF from a live form."""
    if not pdf_path.exists():
        raise FileNotFoundError(f"No such file: {pdf_path}")
    try:
        reader = PdfReader(str(pdf_path))
    except Exception as exc:
        raise RuntimeError(f"Could not parse {pdf_path}: {exc}") from exc

    root = reader.trailer["/Root"]
    widgets = 0
    for page in reader.pages:
        for annot in page.get("/Annots") or []:
            if annot.get_object().get("/Subtype") == "/Widget":
                widgets += 1

    return {
        "has_acroform": "/AcroForm" in root,
        "widget_annotations": widgets,
        "named_fields": len(reader.get_fields() or {}),
    }

if __name__ == "__main__":
    print(flatness_report(Path("out/filled.pdf")))

{'has_acroform': True, 'widget_annotations': 12, 'named_fields': 12}   # still a live form
{'has_acroform': False, 'widget_annotations': 0, 'named_fields': 0}    # genuinely flat

One caveat worth stating early: if the fields render blank before flattening, flattening bakes in the blankness. Confirm the values are visible first — the appearance-stream checks in fix PDF form fields not visible after filling are the prerequisite for this page, not an optional extra.

Fix: Flatten with pypdf

pypdf 5.x can merge a widget's appearance stream onto the page and then drop the annotation. The method below handles the common case — text, checkbox and choice widgets whose appearance stream is a normal XObject.

# pip install "pypdf>=5,<6"
from pathlib import Path
from pypdf import PdfReader, PdfWriter
from pypdf.generic import NameObject

SRC = Path("out/filled.pdf")
DEST = Path("out/flat.pdf")

def flatten(src: Path, dest: Path) -> Path:
    """Merge widget appearances into page content and remove the form layer."""
    reader = PdfReader(str(src))
    writer = PdfWriter(clone_from=reader)

    for page in writer.pages:
        annots = page.get("/Annots") or []
        keep = []
        for annot in annots:
            obj = annot.get_object()
            if obj.get("/Subtype") != "/Widget":
                keep.append(annot)                 # links, notes and stamps survive
                continue
            appearance = obj.get("/AP", {}).get("/N")
            if appearance is None:
                continue                           # nothing drawable — drop the widget
            state = obj.get("/AS")                 # buttons select a sub-state
            if state is not None and hasattr(appearance, "get"):
                appearance = appearance.get(state, appearance)
            try:
                # Draw the widget's own appearance at its own rectangle.
                page.merge_transformed_page(appearance, _widget_matrix(obj))
            except Exception:
                keep.append(annot)                 # unmergeable: leave it rather than lose it
        page[NameObject("/Annots")] = writer._add_object(keep) if keep else writer._add_object([])

    # Remove the form dictionary so no field objects remain.
    writer._root_object.pop(NameObject("/AcroForm"), None)

    dest.parent.mkdir(parents=True, exist_ok=True)
    with dest.open("wb") as fh:
        writer.write(fh)
    return dest

def _widget_matrix(widget) -> tuple:
    """Translation matrix placing an appearance XObject at the widget's /Rect."""
    x0, y0, _x1, _y1 = (float(v) for v in widget["/Rect"])
    return (1, 0, 0, 1, x0, y0)

if __name__ == "__main__":
    print(flatten(SRC, DEST))

The _widget_matrix helper is where hand-rolled flattening usually goes wrong. An appearance stream has its own /BBox and optional /Matrix, and the specification requires mapping the transformed bounding box onto the widget rectangle — a scale as well as a translation. The simplified translation above is correct when /BBox starts at the origin and matches the widget size, which is true of appearance streams that pypdf itself generated. For third-party forms whose streams were produced by Acrobat or LiveCycle, use one of the external tools below rather than debugging matrix arithmetic.

Variant Fix 1: pdftk (Most Reliable)

The pdftk command implements the full specification, including the bounding-box mapping, and handles every form flavour including XFA.

# Debian/Ubuntu
sudo apt-get install -y pdftk-java

# Flatten in one call — values become page content
pdftk out/filled.pdf output out/flat.pdf flatten

# Verify no fields survived
pdftk out/flat.pdf dump_data_fields   # prints nothing on a flat file

Wrapping it keeps the pipeline in Python while delegating the hard part:

# pip install "pypdf>=4.2,<6"   (pdftk must be on PATH)
import shutil
import subprocess
from pathlib import Path

def flatten_with_pdftk(src: Path, dest: Path, timeout: int = 120) -> Path:
    """Flatten via pdftk, failing loudly if the binary is missing or errors."""
    binary = shutil.which("pdftk")
    if binary is None:
        raise RuntimeError("pdftk not found on PATH — install pdftk-java")
    dest.parent.mkdir(parents=True, exist_ok=True)
    try:
        subprocess.run(
            [binary, str(src), "output", str(dest), "flatten"],
            check=True, capture_output=True, timeout=timeout,
        )
    except subprocess.CalledProcessError as exc:
        raise RuntimeError(f"pdftk failed: {exc.stderr.decode(errors='replace')}") from exc
    except subprocess.TimeoutExpired as exc:
        raise RuntimeError(f"pdftk timed out after {timeout}s on {src}") from exc
    return dest

The shutil.which guard is the same defensive pattern that prevents the opaque failures documented in fix tabula java not found error — check for the binary before you shell out, so the error names the missing dependency instead of a return code.

Variant Fix 2: Ghostscript

Ghostscript rewrites the whole document and drops annotations along the way. It is the heaviest option — it re-encodes images and can grow or shrink the file substantially — but it needs no Java and is already installed wherever camelot runs.

gs -sDEVICE=pdfwrite \
   -dPreserveAnnots=false \
   -dNOPAUSE -dBATCH -dQUIET \
   -sOutputFile=out/flat.pdf out/filled.pdf

Ghostscript rasterises nothing by default, so text stays selectable — but it does subset fonts and can alter colour profiles. Compare file sizes and spot-check one page before adopting it for archival output.

Choosing a flattening tool Three tools compared on four criteria. pypdf needs no external binary, is fast, has partial fidelity limited to appearance streams it can map, and does not handle XFA. pdftk needs a Java runtime, is medium speed, has full fidelity and handles XFA. Ghostscript needs the gs binary, is the slowest, has full fidelity but re-encodes the document, and drops XFA along with all other annotations. Criterion pypdf pdftk Ghostscript External binary none Java runtime gs binary Fidelity partial full full, re-encoded XFA forms not supported supported dropped wholesale Speed per file ~40 ms ~250 ms + JVM start ~400 ms Default to pdftk for third-party templates; use pypdf when your own code generated the appearances

Variant Fix 3: Read-Only Instead of Flat

When the recipient must still be able to extract the values — a supplier portal that re-reads the reference number, an internal audit script — flattening is the wrong answer, because it destroys the field names. Lock the fields instead:

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader, PdfWriter
from pypdf.generic import NameObject, NumberObject

READ_ONLY = 1 << 0

def lock(src: Path, dest: Path) -> Path:
    """Set the read-only flag on every widget so viewers refuse edits."""
    writer = PdfWriter(clone_from=PdfReader(str(src)))
    for page in writer.pages:
        for annot in page.get("/Annots") or []:
            obj = annot.get_object()
            if obj.get("/Subtype") == "/Widget":
                obj[NameObject("/Ff")] = NumberObject(int(obj.get("/Ff", 0)) | READ_ONLY)
    dest.parent.mkdir(parents=True, exist_ok=True)
    with dest.open("wb") as fh:
        writer.write(fh)
    return dest

Treat this as a courtesy, not a control. If the requirement is tamper evidence rather than tamper resistance, add an owner password or a certificate signature — see watermarking and securing PDFs and add password protection to PDF files.

Extract before you flatten Three artefacts should be produced before flattening destroys the field layer. A CSV or JSON export of every field name and value preserves the data. A copy of the unflattened file preserves the ability to re-read or amend it. A record of which template version produced it preserves the audit trail. After flattening, only pixels remain, so none of these can be recovered. Produce these three things first — flattening is one-way 1. Values export field name to value CSV or JSON alongside 2. Unflat copy the filled original kept for amendments 3. Template ref which version filled it for the audit trail After flattening the values are ink — no library can read them back by name

Verification

The check that matters is the negative one: after flattening, no field may remain, and the visible text must still be there.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader

def assert_flat(pdf_path: Path, must_contain: list[str]) -> None:
    """Fail if any form object survived, or if the baked text went missing."""
    reader = PdfReader(str(pdf_path))
    root = reader.trailer["/Root"]

    assert "/AcroForm" not in root, "AcroForm dictionary still present"
    assert not (reader.get_fields() or {}), "named fields still present"

    widgets = sum(
        1
        for page in reader.pages
        for annot in (page.get("/Annots") or [])
        if annot.get_object().get("/Subtype") == "/Widget"
    )
    assert widgets == 0, f"{widgets} widget annotation(s) still present"

    text = "\n".join(page.extract_text() or "" for page in reader.pages)
    missing = [needle for needle in must_contain if needle not in text]
    assert not missing, f"values lost during flattening: {missing}"

    print(f"{pdf_path.name} is flat and still contains {len(must_contain)} expected value(s)")

if __name__ == "__main__":
    assert_flat(Path("out/flat.pdf"), ["Dana Whitfield", "APP-2026-00417"])

The extract_text half of that assertion is the important one. A flattening step that silently drops appearance streams produces a file that passes every structural check while showing an empty page — the exact failure that makes flattening feel unreliable. Asserting on the text catches it in CI. For heavier verification, run the extracted text through the same normalisation used when extracting PDF data into pandas so whitespace differences do not cause false alarms.

FAQ

Can I un-flatten a PDF? No. The field objects and their names are gone, and the values are indistinguishable from the rest of the page's ink. Always keep the filled-but-unflattened file if the data may be needed later — or extract the field values to CSV or JSON as part of the flattening job.

Does flattening remove signatures and links? Flattening removes widget annotations. Link annotations, sticky notes and stamps are a different subtype and survive if your code filters on /Subtype == "/Widget", as the pypdf example does. Ghostscript with -dPreserveAnnots=false is blunter and drops them all.

Why is my flattened file larger than the original? Each appearance stream becomes page content, and Ghostscript in particular may embed font subsets per page. Run gs with -dCompressFonts=true, or flatten with pdftk, which reuses the existing font resources.

Will flattening fix fields that render blank? No — it does the opposite. Flattening draws what the appearance stream contains, so a missing or empty stream bakes in an empty box permanently. Fix the appearances first, then flatten.

How do I flatten only some fields? Filter by field name in the pypdf loop: merge and drop the widgets you want frozen, and leave the rest in /Annots. Keep /AcroForm in place in that case, otherwise the surviving widgets lose the resource dictionary their appearance streams reference.

Part of Filling PDF Forms with Python.