Filling PDF Forms with Python

Government tax forms, onboarding packs, insurance claims and purchase orders all arrive as interactive PDFs with named input boxes. The manual loop — open the file, click into each box, type, save, repeat for 400 applicants — is exactly the workload that a script kills in seconds. The catch is that a PDF form is not a document with blanks in it: it is a dictionary of field objects attached to the file's /AcroForm entry, each linked to one or more widget annotations that decide where and how the value is painted. Write the field value without also dealing with the widget's appearance stream and the file will contain the right data while showing a blank box in every viewer — the single most reported symptom on this topic, covered in detail in fix PDF form fields not visible after filling.

Generic approaches fail here for a specific reason. Text-layer tools such as pdfplumber read the page's content stream — the glyphs already painted onto the page — and a form field's value lives outside that stream, in the annotation dictionary. Template engines such as reportlab build a new page from scratch, which is the wrong tool when the counterparty requires their own form back, digitally identical, with only the boxes filled. Form filling is its own operation: locate fields by name, coerce Python values into the string or /Name objects the specification expects, and update the annotation so viewers repaint it.

Prerequisites

Everything below runs on pypdf, which reads and writes AcroForm dictionaries without any system binary. Ghostscript and pdftk appear only in the flattening section as optional alternatives.

# Python 3.9+ recommended
python -m venv .venv
source .venv/bin/activate

# Core: field read/write. pypdf 4.x renamed several form helpers, so pin it.
pip install "pypdf>=4.2,<6"

# Optional: pdfrw is a lighter alternative for read-only field inspection
pip install pdfrw

# Optional: a real fillable PDF to test against. Any AcroForm PDF works —
# the US IRS W-9 and most government forms are AcroForm, not XFA.
mkdir -p data

A requirements.txt for the pipeline that ships this:

pypdf==5.1.0
pandas==2.2.3

Two vocabulary items make the rest of the page readable. A field is the logical input — it owns the name (applicant_name), the type (/Tx for text, /Btn for buttons, /Ch for choice) and the value (/V). A widget is the on-page rectangle that renders it; a field with two widgets appears in two places and shows the same value in both.

Step 1: Diagnose the Form Before Writing Anything

Never fill blind. Dump the field tree first: names are frequently non-obvious (topmostSubform[0].Page1[0].f1_01[0] is a real IRS field name), and the field type dictates what a valid value looks like. Checkboxes in particular do not accept True or "Yes" — they accept one of the /Off and on-state names declared in the widget's appearance dictionary, and that on-state is often /1 or /On rather than /Yes.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader

FORM_PDF = Path("data/application_form.pdf")

def describe_fields(pdf_path: Path) -> list[dict]:
    """List every AcroForm field with its type, current value and legal states."""
    if not pdf_path.exists():
        raise FileNotFoundError(f"Form not found: {pdf_path}")
    try:
        reader = PdfReader(str(pdf_path))
    except Exception as exc:
        raise RuntimeError(f"Could not open {pdf_path}: {exc}") from exc

    fields = reader.get_fields() or {}
    rows = []
    for name, field in fields.items():
        states = field.get("/_States_")           # legal on/off names for buttons
        rows.append({
            "name": name,
            "type": field.get("/FT"),             # /Tx text, /Btn button, /Ch choice
            "value": field.get("/V"),
            "states": list(states) if states else None,
            "flags": field.get("/Ff", 0),         # bit flags: read-only, multiline, radio
        })
    return rows

if __name__ == "__main__":
    for row in describe_fields(FORM_PDF):
        print(f"{row['name']:<40} {row['type']}  states={row['states']}")

Typical output from a real form:

applicant_name                           /Tx  states=None
applicant_email                          /Tx  states=None
agreed                                   /Btn states=['/Off', '/Yes']
delivery_method                          /Btn states=['/Off', '/Post', '/Email']
country                                  /Ch  states=None

Three facts to extract from that dump before writing a line of filling code. agreed is a checkbox whose on-state is /Yes. delivery_method is a radio group — one field, several widgets, and the value must be /Post or /Email, never the label text shown on the page. country is a dropdown whose /Opt array lists the permitted strings; writing a value outside that list produces a form that most viewers will render but that a strict validator will reject.

If reader.get_fields() returns None or an empty dict on a file that clearly has boxes on screen, you are almost certainly looking at an XFA form (Adobe's XML-based successor) or a flattened scan. The variants section below shows how to tell those apart in one line each.

Step 2: Fill Text Fields

Text fields are the easy case: update_page_form_field_values walks the page's annotations, matches on the field name, and writes /V. The two arguments that matter are auto_regenerate, which asks pypdf to rebuild appearance streams itself, and the writer-level /NeedAppearances flag, which asks the viewer to do it. Setting both is the belt-and-braces default that survives Chrome's built-in PDF plugin, Preview on macOS and Acrobat alike.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader, PdfWriter
from pypdf.generic import BooleanObject, NameObject

FORM_PDF = Path("data/application_form.pdf")
OUT_PDF = Path("out/application_filled.pdf")

def fill_text_fields(src: Path, dest: Path, values: dict[str, str]) -> Path:
    """Write text values into an AcroForm and keep them visible in every viewer."""
    reader = PdfReader(str(src))
    writer = PdfWriter(clone_from=reader)          # clone keeps the AcroForm intact

    # Ask viewers to regenerate field appearances on open — the single most
    # important line if fields look blank after filling.
    writer._root_object[NameObject("/AcroForm")][NameObject("/NeedAppearances")] = BooleanObject(True)

    for page in writer.pages:
        try:
            writer.update_page_form_field_values(page, values, auto_regenerate=True)
        except Exception as exc:
            raise RuntimeError(f"Filling failed on a page of {src}: {exc}") from exc

    dest.parent.mkdir(parents=True, exist_ok=True)
    with dest.open("wb") as fh:
        writer.write(fh)
    return dest

if __name__ == "__main__":
    fill_text_fields(FORM_PDF, OUT_PDF, {
        "applicant_name": "Dana Whitfield",
        "applicant_email": "[email protected]",
        "reference": "APP-2026-00417",
    })
    print(f"Wrote {OUT_PDF}")

Two details are worth internalising. PdfWriter(clone_from=reader) copies the whole object graph including /AcroForm; the older writer.append(reader) idiom drops the form dictionary on some files, and the fields then vanish entirely. And the loop runs update_page_form_field_values for every page rather than only page 1, because a field's widget can live on any page — passing writer.pages[0] alone silently skips fields further into the document.

Step 3: Fill Checkboxes and Radio Groups

Buttons need a /Name object, not a Python string, and the name has to be one of the states you dumped in step 1. pypdf accepts a plain "/Yes" string and coerces it, but being explicit makes the failure mode visible: pass "Yes" without the slash and the checkbox stays off with no exception raised.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader, PdfWriter
from pypdf.generic import BooleanObject, NameObject

FORM_PDF = Path("data/application_form.pdf")
OUT_PDF = Path("out/application_checked.pdf")

def on_state_for(reader: PdfReader, field_name: str) -> str:
    """Return the widget's 'ticked' state name, e.g. '/Yes', '/On' or '/1'."""
    field = (reader.get_fields() or {}).get(field_name)
    if field is None:
        raise KeyError(f"No such field: {field_name}")
    states = [s for s in (field.get("/_States_") or []) if s != "/Off"]
    if not states:
        raise ValueError(f"{field_name} is not a button field")
    return states[0]

def fill_buttons(src: Path, dest: Path, ticks: dict[str, bool]) -> Path:
    """Tick or clear checkbox fields using each widget's own declared state name."""
    reader = PdfReader(str(src))
    writer = PdfWriter(clone_from=reader)
    writer._root_object[NameObject("/AcroForm")][NameObject("/NeedAppearances")] = BooleanObject(True)

    values = {}
    for name, ticked in ticks.items():
        # '/Off' is universal; the on-state is per-widget and must be looked up.
        values[name] = NameObject(on_state_for(reader, name) if ticked else "/Off")

    for page in writer.pages:
        writer.update_page_form_field_values(page, values, auto_regenerate=True)

    dest.parent.mkdir(parents=True, exist_ok=True)
    with dest.open("wb") as fh:
        writer.write(fh)
    return dest

if __name__ == "__main__":
    fill_buttons(FORM_PDF, OUT_PDF, {"agreed": True, "marketing_optin": False})
    print(f"Wrote {OUT_PDF}")

Radio groups follow the same rule with one extra wrinkle: all the buttons in a group share one field name, and the value selects which widget lights up. Passing /Post to delivery_method turns on the widget whose appearance dictionary declares /Post and turns every sibling off, which is why you must never iterate the widgets and set them individually — the viewer will show two selected radio buttons, an impossible state that some validators reject outright.

Step 4: Fill Dropdowns and Multi-Line Text

Choice fields (/Ch) hold their permitted values in /Opt. When the option array contains pairs — an export value and a display label — the value you write is the export value, not the label a human sees.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader

def choice_options(pdf_path: Path, field_name: str) -> list[str]:
    """Return the export values a /Ch field will accept."""
    reader = PdfReader(str(pdf_path))
    field = (reader.get_fields() or {}).get(field_name, {})
    opts = field.get("/Opt") or []
    out = []
    for opt in opts:
        # /Opt entries are either a plain string or [export_value, display_label]
        out.append(str(opt[0]) if isinstance(opt, list) else str(opt))
    return out

if __name__ == "__main__":
    print(choice_options(Path("data/application_form.pdf"), "country"))
    # ['GB', 'IE', 'DE', 'FR']  — write 'GB', not 'United Kingdom'

Multi-line text fields carry bit 13 (value 4096) in their /Ff flags. Writing a string with \n into a single-line field is not an error, but the newline is dropped by most renderers, so check the flag before formatting an address block:

# pip install "pypdf>=4.2,<6"
MULTILINE_FLAG = 1 << 12  # bit 13 in the PDF spec's 1-based numbering

def is_multiline(field: dict) -> bool:
    """True when the text field accepts embedded newlines."""
    return bool(int(field.get("/Ff", 0)) & MULTILINE_FLAG)

address = "17 Fenwick Road\nLeeds\nLS8 2QT"
# Collapse to one line when the field cannot render the break:
single_line = " ".join(address.split("\n"))

Edge Cases and Variants

XFA Forms Report No Fields

Adobe LiveCycle forms store their real definition in an XML package under /AcroForm /XFA, leaving the AcroForm field tree empty or stale. pypdf reads the shell, not the XML, so the fields dictionary looks wrong.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader

def is_xfa(pdf_path: Path) -> bool:
    """Detect a LiveCycle/XFA form, whose fields pypdf cannot fill directly."""
    reader = PdfReader(str(pdf_path))
    acro = reader.trailer["/Root"].get("/AcroForm", {})
    return "/XFA" in acro

if __name__ == "__main__":
    print("XFA form — convert before filling" if is_xfa(Path("data/form.pdf")) else "AcroForm — fill directly")

The practical fix is conversion, not code: open the file once in Acrobat and export as a standard PDF, or run pdftk form.pdf output flat.pdf drop_xfa, which keeps the AcroForm layer and discards the XML. After that, the workflow above applies unchanged.

The Same Field Appears on Several Pages

Long forms repeat a reference number in a footer on every page. Because those widgets share one field name, one write fills them all — provided the loop iterates every page, which is exactly why step 2 does not special-case page 1. Confirm the fan-out by counting widgets per name:

# pip install "pypdf>=4.2,<6"
from collections import Counter
from pathlib import Path
from pypdf import PdfReader

def widget_counts(pdf_path: Path) -> Counter:
    """How many on-page widgets each field name owns."""
    reader = PdfReader(str(pdf_path))
    counts = Counter()
    for page in reader.pages:
        for annot in page.get("/Annots") or []:
            obj = annot.get_object()
            if obj.get("/Subtype") != "/Widget":
                continue
            parent = obj.get("/Parent")
            name = obj.get("/T") or (parent.get_object().get("/T") if parent else None)
            if name:
                counts[str(name)] += 1
    return counts

Read-Only and Required Fields

Bit 1 of /Ff marks a field read-only. pypdf writes the value anyway, and compliant viewers then display it — but a form-submission endpoint that re-reads the file may treat the change as tampering. Filter deliberately rather than by accident:

# pip install "pypdf>=4.2,<6"
READ_ONLY = 1 << 0
REQUIRED = 1 << 1

def classify(field: dict) -> str:
    flags = int(field.get("/Ff", 0))
    if flags & READ_ONLY:
        return "read-only — do not write"
    if flags & REQUIRED:
        return "required — must be non-empty before submission"
    return "editable"

Validation: Prove the Values Landed

A filled PDF that looks right in one viewer is not evidence. Re-open the output and assert on the values, and assert on the widget appearance state for buttons — that pair catches both the "value written but invisible" and the "checkbox silently reset to /Off" failures.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader

def verify_fill(pdf_path: Path, expected: dict[str, str]) -> None:
    """Assert every expected field value survived the write."""
    fields = PdfReader(str(pdf_path)).get_fields() or {}
    problems = []
    for name, want in expected.items():
        got = fields.get(name, {}).get("/V")
        if got is None:
            problems.append(f"{name}: field missing or unset")
        elif str(got).lstrip("/") != str(want).lstrip("/"):
            problems.append(f"{name}: expected {want!r}, found {got!r}")
    if problems:
        raise AssertionError("Fill verification failed:\n  " + "\n  ".join(problems))
    print(f"All {len(expected)} field(s) verified in {pdf_path.name}")

if __name__ == "__main__":
    verify_fill(Path("out/application_filled.pdf"), {
        "applicant_name": "Dana Whitfield",
        "agreed": "/Yes",
    })

For a visual check that does not need a human, render page 1 to an image with pdf2image and compare a crop against a known-good baseline — the same rasterise-and-inspect technique used for scanned PDFs. In CI, comparing image hashes for the region around each widget catches font-substitution regressions that a value assertion never sees.

Performance and Scale

Filling is I/O-bound and cheap: a 20-field, 4-page form takes roughly 30–60 ms end to end, so a 5,000-row mail run is a couple of minutes single-threaded. The costs that actually bite at scale are elsewhere.

Cloning the reader per output file re-parses the template every time. When one template produces thousands of documents, parse the template once and clone from the in-memory object graph:

# pip install "pypdf>=4.2,<6" pandas
from io import BytesIO
from pathlib import Path
import pandas as pd
from pypdf import PdfReader, PdfWriter
from pypdf.generic import BooleanObject, NameObject

TEMPLATE = Path("data/application_form.pdf")
ROWS = Path("data/applicants.csv")
OUT_DIR = Path("out/batch")

def batch_fill(template: Path, rows: pd.DataFrame, out_dir: Path) -> int:
    """Fill one template many times, re-reading the source bytes only once."""
    source_bytes = template.read_bytes()          # parse cost paid per writer, not per disk read
    out_dir.mkdir(parents=True, exist_ok=True)
    written = 0
    for row in rows.to_dict("records"):
        reader = PdfReader(BytesIO(source_bytes))
        writer = PdfWriter(clone_from=reader)
        writer._root_object[NameObject("/AcroForm")][NameObject("/NeedAppearances")] = BooleanObject(True)
        values = {k: str(v) for k, v in row.items() if k != "output_name"}
        for page in writer.pages:
            writer.update_page_form_field_values(page, values, auto_regenerate=True)
        dest = out_dir / f"{row['output_name']}.pdf"
        with dest.open("wb") as fh:
            writer.write(fh)
        written += 1
    return written

if __name__ == "__main__":
    df = pd.read_csv(ROWS, dtype=str).fillna("")
    print(f"Wrote {batch_fill(TEMPLATE, df, OUT_DIR)} PDFs")

Reading the driver rows with pandas and dtype=str avoids the classic batch bug where a postcode or reference number is inferred as a float and lands in the PDF as 417.0; the same defensive read is covered in reading Excel files with Python. Memory stays flat because each writer is discarded per row — the failure mode to avoid is accumulating writers in a list "to write later", which holds every page tree in RAM at once.

If output volume is the bottleneck rather than fill time, parallelise across processes, not threads: pypdf's object graph is not thread-safe, and a ProcessPoolExecutor over chunks of the driver frame scales close to linearly up to core count.

Flattening: Making the Values Permanent

A filled form is still editable. For anything sent to a counterparty — a signed agreement, an audit artefact — flatten it so the values become page content and the fields disappear.

The read-only route is a two-line change and needs no extra dependency:

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader, PdfWriter
from pypdf.generic import NameObject, NumberObject

READ_ONLY = 1 << 0

def lock_fields(src: Path, dest: Path) -> Path:
    """Mark every field read-only so viewers refuse further edits."""
    writer = PdfWriter(clone_from=PdfReader(str(src)))
    for page in writer.pages:
        for annot in page.get("/Annots") or []:
            obj = annot.get_object()
            if obj.get("/Subtype") == "/Widget":
                flags = int(obj.get("/Ff", 0)) | READ_ONLY
                obj[NameObject("/Ff")] = NumberObject(flags)
    with dest.open("wb") as fh:
        writer.write(fh)
    return dest

True flattening — merging each widget's appearance stream into the page content and deleting the form — is what pdftk filled.pdf output flat.pdf flatten does in one command, and what Ghostscript does with -dPreserveAnnots=false. Prefer one of those binaries over hand-rolled Python: the appearance stream has to be positioned with the widget's own transformation matrix, and getting that arithmetic wrong shifts every value a few points off its box. The dedicated walkthrough is in flatten PDF form fields with Python.

Troubleshooting

Complete Working Script

One command-line tool that inspects, fills, verifies and optionally locks a form, driven by a JSON value file.

#!/usr/bin/env python3
"""Fill an AcroForm PDF from a JSON mapping and verify the result.

    pip install "pypdf>=4.2,<6"
    python fill_form.py data/form.pdf values.json out/filled.pdf --lock
"""
from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path

from pypdf import PdfReader, PdfWriter
from pypdf.generic import BooleanObject, NameObject, NumberObject

READ_ONLY = 1 << 0

def load_states(reader: PdfReader) -> dict[str, list[str]]:
    """Map each button field to the on-state names it will accept."""
    out = {}
    for name, field in (reader.get_fields() or {}).items():
        states = [s for s in (field.get("/_States_") or []) if s != "/Off"]
        if states:
            out[name] = states
    return out

def coerce(name: str, value, states: dict[str, list[str]]):
    """Turn a JSON value into what the field type expects."""
    if name in states:                                   # button: needs a /Name
        if isinstance(value, bool):
            return NameObject(states[name][0] if value else "/Off")
        text = str(value)
        return NameObject(text if text.startswith("/") else f"/{text}")
    return str(value)                                    # text and choice: plain string

def fill(src: Path, values: dict, dest: Path, lock: bool) -> None:
    if not src.exists():
        raise FileNotFoundError(f"Template not found: {src}")
    reader = PdfReader(str(src))
    if not reader.get_fields():
        raise ValueError(f"{src} exposes no AcroForm fields (scanned or XFA?)")

    states = load_states(reader)
    payload = {k: coerce(k, v, states) for k, v in values.items()}

    writer = PdfWriter(clone_from=reader)
    writer._root_object[NameObject("/AcroForm")][NameObject("/NeedAppearances")] = BooleanObject(True)
    for page in writer.pages:
        writer.update_page_form_field_values(page, payload, auto_regenerate=True)

    if lock:
        for page in writer.pages:
            for annot in page.get("/Annots") or []:
                obj = annot.get_object()
                if obj.get("/Subtype") == "/Widget":
                    obj[NameObject("/Ff")] = NumberObject(int(obj.get("/Ff", 0)) | READ_ONLY)

    dest.parent.mkdir(parents=True, exist_ok=True)
    with dest.open("wb") as fh:
        writer.write(fh)

    written = PdfReader(str(dest)).get_fields() or {}
    missing = [k for k in values if written.get(k, {}).get("/V") in (None, "")]
    if missing:
        raise AssertionError(f"Fields did not persist: {', '.join(missing)}")
    print(f"{dest} — {len(values)} field(s) filled and verified")

def main(argv: list[str] | None = None) -> int:
    parser = argparse.ArgumentParser(description="Fill an AcroForm PDF from JSON.")
    parser.add_argument("template", type=Path)
    parser.add_argument("values", type=Path, help="JSON object of field name to value")
    parser.add_argument("output", type=Path)
    parser.add_argument("--lock", action="store_true", help="mark fields read-only")
    args = parser.parse_args(argv)

    try:
        fill(args.template, json.loads(args.values.read_text()), args.output, args.lock)
    except (FileNotFoundError, ValueError, AssertionError) as exc:
        print(f"error: {exc}", file=sys.stderr)
        return 1
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Point it at a driver spreadsheet instead of a single JSON file and it becomes the document half of a full pipeline — the pattern described in automating document data pipelines, where extraction, transformation and document output run as one scheduled job.

FAQ

Why do my filled values disappear when the file is opened in Chrome? Chrome's PDF viewer renders widget appearance streams and ignores /V when no stream exists. Writing the value alone is therefore invisible there while Acrobat, which honours /NeedAppearances, shows it. Set /NeedAppearances to True and pass auto_regenerate=True so a stream is generated regardless of the viewer.

How do I find the field names in a form nobody documented? Run the describe_fields helper from step 1 — it prints every name, type and legal state. For a visual map, open the form in Acrobat's Prepare Form tool, which overlays each field's name on its box; the names it shows match the strings pypdf reports, including the long topmostSubform[0]... paths used by government forms.

Can I fill a form that has been scanned to PDF? No — a scan is an image with no field objects, so there is nothing to fill. Either obtain the interactive original, or treat it as an image workflow: OCR the page as covered in scanning and OCR processing with Python, then stamp text at fixed coordinates with an overlay generated by reportlab and merged in.

Is writer._root_object a private API I should avoid? It is underscore-prefixed but stable across pypdf 3.x–5.x and is the documented route to /NeedAppearances, which has no public setter. Pin the pypdf version in requirements.txt so an upgrade cannot silently change the behaviour, and cover the flag with the verification step above so a regression fails your tests rather than your recipients' viewers.

Does filling a form invalidate an existing digital signature? Yes, for any signature that covers the whole document. Signed forms use incremental updates so earlier signatures stay valid only if the new content is appended rather than rewritten, and pypdf rewrites. If the template is signed, fill first and sign last — or delegate signing to a service that applies it after your fill step.

Related

• Fix PDF Form Fields Not Visible After Filling — the /NeedAppearances and appearance-stream fix in depth
• Flatten PDF Form Fields with Python — make filled values permanent before sending
• Generating PDF Reports Dynamically — build a document from scratch when no template exists
• Merging and Splitting PDF Documents — assemble filled forms into a single pack
• Watermarking and Securing PDFs — lock or stamp the finished document

Part of Automating PDF Extraction & Generation.