Python Doc & Data Automation

List PDF Form Field Names with Python

Filling a government or supplier form from spreadsheet data starts with a question the PDF does not answer visually: what is each field actually called? The on-page labels say "Company name" and "VAT number", but the field names inside the file are topmostSubform[0].Page1[0].TextField1[3], f1_07[0], or simply Text12. reader.get_form_text_fields() returns only some of them, dropdowns come back without their options, and fields with the same short name on different pages overwrite each other in a dictionary.

Root Cause

A PDF form's fields form a tree. Each node has a partial name (/T), and a field's real, unique name is the dot-joined path from the root — employer.address.postcode. Forms designed in Adobe LiveCycle or converted from XFA produce deep, generated paths with array-style indices; forms made in other tools often use flat, meaningless names. Convenience methods hide part of this: get_form_text_fields() includes only text fields; dictionaries keyed by partial name collide when two branches reuse Name; and a field whose widgets appear on several pages (a reference number repeated in every page header) is one field with several positions. Types are encoded as /FT plus bit flags (/Ff) that distinguish checkboxes from radio groups from push buttons, and combo boxes from list boxes. None of the labels a person sees are stored with the field. To fill a form reliably you need a complete map: full name, type, options, current value, and where on which page each widget sits, so a person can match it to the visible label.

Minimal Diagnostic

Compare what the quick methods return with the full field tree. The gap shows how much a naive approach would miss.

# pip install "pypdf>=4.0"
from pathlib import Path
from pypdf import PdfReader

SOURCE = Path("in/supplier-registration.pdf")

def quick_vs_full(pdf_path: Path) -> None:
    try:
        reader = PdfReader(pdf_path)
    except Exception as exc:
        raise SystemExit(f"cannot read {pdf_path}: {exc}")
    text_only = reader.get_form_text_fields() or {}
    full = reader.get_fields() or {}
    print(f"get_form_text_fields(): {len(text_only)} field(s)")
    print(f"get_fields():           {len(full)} field(s)")
    types = {}
    for name, field in full.items():
        types.setdefault(field.get("/FT", "?"), []).append(name)
    for ft, names in types.items():
        print(f"   {ft}: {len(names)} e.g. {names[:2]}")
    short = [n.split(".")[-1] for n in full]
    dupes = sorted({s for s in short if short.count(s) > 1})
    print("short names used more than once:", dupes[:6])
    if "/XFA" in reader.trailer["/Root"].get("/AcroForm", {}):
        print("form also carries an XFA definition (LiveCycle form)")

if __name__ == "__main__":
    quick_vs_full(SOURCE)

get_form_text_fields(): 31 field(s)
get_fields():           58 field(s)
   /Tx: 31 e.g. ['form1[0].Page1[0].CompanyName[0]', 'form1[0].Page1[0].VATNo[0]']
   /Btn: 19 e.g. ['form1[0].Page1[0].Ltd[0]', 'form1[0].Page2[0].Consent[0]']
   /Ch: 6 e.g. ['form1[0].Page1[0].Country[0]', 'form1[0].Page2[0].Currency[0]']
   /Sig: 2 e.g. ['form1[0].Page3[0].Signature[0]', 'form1[0].Page3[0].Signature2[0]']
short names used more than once: ['Name[0]', 'Phone[0]', 'Email[0]']
form also carries an XFA definition (LiveCycle form)

The text-field helper misses 27 of 58 fields, three short names repeat across sections, and the form is a LiveCycle hybrid — which matters for filling, covered below.

Field names are paths in a tree The root form node form1 has page nodes Page1 and Page2. Page1 holds CompanyName, a Contact subform with Name, Phone and Email, and a Country choice field. Page2 holds a Bank subform with its own Name field and a Consent checkbox. The full names form1.Page1.Contact.Name and form1.Page2.Bank.Name are unique, while the short name Name collides. form1[0] root of the field tree Page1[0] Contact[0].Name[0] unique full path Page1[0] Country[0] /Ch combo box with options Page2[0] Bank[0].Name[0] same short name Page2[0] Consent[0] /Btn checkbox, states /Off /1

Fix: Build a Complete Field Map with Positions

PyMuPDF iterates widgets page by page and reports each one's full field name, type, value, options and rectangle, which is exactly what a field map needs. Export it to CSV so a person can fill in the business meaning next to each field. Changed lines carry comments.

# pip install pymupdf "pandas>=2.2"
from pathlib import Path
import pandas as pd
import pymupdf

SOURCE = Path("in/supplier-registration.pdf")
DEST = Path("out/supplier-registration-fields.csv")

def nearby_label(page: pymupdf.Page, rect: pymupdf.Rect, words: list) -> str:
    """Best-effort visible label: words just left of, or just above, the widget."""
    left = [w for w in words if abs((w[1] + w[3]) / 2 - (rect.y0 + rect.y1) / 2) < max(rect.height, 8)
            and rect.x0 - 220 <= w[2] <= rect.x0 + 2]
    above = [w for w in words if rect.y0 - 16 <= w[3] <= rect.y0 + 1 and w[0] < rect.x1 and w[2] > rect.x0]
    chosen = sorted(left, key=lambda w: w[0])[-6:] or sorted(above, key=lambda w: w[0])[:6]
    return " ".join(w[4] for w in chosen)

def field_map(pdf_path: Path) -> pd.DataFrame:
    rows = []
    with pymupdf.open(pdf_path) as doc:
        if not doc.is_form_pdf:
            raise ValueError(f"{pdf_path.name} has no fillable fields")
        for page in doc:
            words = page.get_text("words")
            for w in page.widgets():                                         # changed: every widget, every page
                rows.append({
                    "page": page.number + 1,
                    "field_name": w.field_name,                              # changed: full dotted name
                    "type": w.field_type_string,                             # changed: Text, CheckBox, RadioButton...
                    "value": w.field_value,
                    "options": "|".join(map(str, w.choice_values or [])),    # changed: dropdown and list options
                    "on_state": w.on_state() if w.field_type in (pymupdf.PDF_WIDGET_TYPE_CHECKBOX,
                                                                  pymupdf.PDF_WIDGET_TYPE_RADIOBUTTON) else "",
                    "max_len": w.text_maxlen or "",
                    "read_only": bool(w.field_flags & 1),
                    "label_guess": nearby_label(page, w.rect, words),        # changed: help a human map fields
                    "x0": round(w.rect.x0, 1), "y0": round(w.rect.y0, 1),
                    "maps_to": "",                                           # filled in by a person
                })
    frame = pd.DataFrame(rows).sort_values(["page", "y0", "x0"], kind="stable")
    frame["occurrences"] = frame.groupby("field_name")["field_name"].transform("size")
    return frame

if __name__ == "__main__":
    try:
        fmap = field_map(SOURCE)
    except (ValueError, RuntimeError) as exc:
        raise SystemExit(str(exc))
    DEST.parent.mkdir(parents=True, exist_ok=True)
    fmap.to_csv(DEST, index=False, encoding="utf-8-sig")
    print(f"{len(fmap)} widgets, {fmap['field_name'].nunique()} fields -> {DEST}")

Sorting by page, then vertical, then horizontal position puts fields in reading order, so the CSV lines up with the printed form. The label_guess column is only a hint — labels on forms are placed in every conceivable position — but it typically gets most fields right and turns mapping a 60-field form from an hour of clicking into a few minutes of checking. occurrences above one flags fields shown in several places, which must be filled once, not per widget.

The exported field map The exported map lists one row per widget in reading order. CompanyName is a text field on page 1 with label guess Company name, mapped to supplier.name. Country is a combo box with options United Kingdom, Ireland and Germany, mapped to supplier.country. Ltd is a checkbox with on-state 1. Reference appears on three pages with occurrences 3. The maps_to column is filled in by a person and drives filling. field_name type label_guess maps_to ...Page1[0].CompanyName[0] Text Company name supplier.name ...Page1[0].Country[0] ComboBox Country supplier.country ...Page1[0].Ltd[0] CheckBox Limited company supplier.is_ltd ...Header[0].Reference[0] Text (x3) Our reference fill once

Variant Fix 1: pypdf Only, Including Choice Options

Where PyMuPDF is not available, pypdf's get_fields() returns the full tree with fully qualified names. Options for choice fields are in /Opt, and states for buttons in the computed /_States_:

# pip install "pypdf>=4.0"
from pathlib import Path
from pypdf import PdfReader

TYPE_NAMES = {"/Tx": "text", "/Ch": "choice", "/Sig": "signature"}

def pypdf_field_map(pdf_path: Path) -> list[dict]:
    reader = PdfReader(pdf_path)
    rows = []
    for name, field in (reader.get_fields() or {}).items():
        ft = field.get("/FT")
        flags = int(field.get("/Ff", 0))
        if ft == "/Btn":
            kind = "radio" if flags & (1 << 15) else "pushbutton" if flags & (1 << 16) else "checkbox"
        else:
            kind = TYPE_NAMES.get(ft, "group")
        options = field.get("/Opt") or []
        options = [o[1] if isinstance(o, list) and len(o) == 2 else o for o in options]  # [export, display] pairs
        rows.append({"field_name": name, "type": kind, "value": field.get("/V"),
                     "options": "|".join(map(str, options)),
                     "states": "|".join(field.get("/_States_", []))})
    return rows

pypdf does not report positions directly from get_fields(); to add page numbers, iterate each page's /Annots and match widgets to fields by their /Parent chain, as in the diagnostic of fill PDF checkboxes and radio buttons with Python. Non-terminal nodes (groups with children) appear in the result too; kind == "group" marks them so they can be filtered out.

Variant Fix 2: LiveCycle (XFA) Hybrid Forms

The diagnostic's XFA message matters. Hybrid forms store their definition twice: as standard AcroForm fields, which Python libraries fill, and as an XFA XML package that Adobe Reader prefers when present. Filling only the AcroForm fields can produce a file that looks empty in Acrobat, because Acrobat renders from the unchanged XFA data. Remove the XFA package so every viewer uses the AcroForm fields:

# pip install "pypdf>=4.0"
from pathlib import Path
from pypdf import PdfReader, PdfWriter
from pypdf.generic import NameObject

def drop_xfa(src: Path, dest: Path) -> bool:
    writer = PdfWriter(clone_from=PdfReader(src))
    acroform = writer._root_object.get("/AcroForm")
    if acroform is None:
        return False
    acroform = acroform.get_object()
    had_xfa = NameObject("/XFA") in acroform
    if had_xfa:
        del acroform[NameObject("/XFA")]
    with dest.open("wb") as fh:
        writer.write(fh)
    return had_xfa

Dynamic XFA forms (which grow sections at run time) have no meaningful AcroForm fallback and cannot be filled this way at all; they need Adobe tooling or a re-created form. Static hybrids — the common case for government and banking forms — work well after the XFA package is removed.

From unknown form to repeatable filling Export every widget with full names, types, options and label guesses to CSV. A person fills the maps_to column with data field names. If the form carries XFA, remove it so viewers use the AcroForm fields. Filling code reads the map and writes values by full field name, choosing export values for buttons and valid options for choices. Verification reads the filled file back and compares every mapped field with the source data. Export map CSV, reading order Human mapping maps_to column Drop XFA if hybrid form Fill by full name from the map Verify read back and compare

Filling from the Map

With the mapped CSV committed next to the job, filling becomes data-driven and reviewable. Values for choice fields are checked against their options before writing:

# pip install pymupdf "pandas>=2.2"
from pathlib import Path
import pandas as pd
import pymupdf

def fill_from_map(template: Path, fmap_csv: Path, record: dict, dest: Path) -> list[str]:
    fmap = pd.read_csv(fmap_csv, dtype=str).fillna("")
    wanted = {r.field_name: r for r in fmap.itertuples() if r.maps_to}
    problems = []
    with pymupdf.open(template) as doc:
        for page in doc:
            for w in page.widgets():
                row = wanted.get(w.field_name)
                if row is None:
                    continue
                value = record.get(row.maps_to)
                if value is None:
                    problems.append(f"no data for {row.maps_to}")
                    continue
                if row.options and str(value) not in row.options.split("|"):
                    problems.append(f"{row.maps_to}={value!r} not an option of {w.field_name}")
                    continue
                if w.field_type == pymupdf.PDF_WIDGET_TYPE_CHECKBOX:
                    w.field_value = w.on_state() if value in (True, "true", "1", "yes") else "Off"
                else:
                    w.field_value = str(value)
                w.update()
        dest.parent.mkdir(parents=True, exist_ok=True)
        doc.save(dest, garbage=3, deflate=True)
    return problems

Returning problems instead of raising lets a batch fill hundreds of forms and report every mismatch at the end — typically a country spelled differently from the form's dropdown option.

Verification

Check that the map is complete and still matches the form — forms get revised, and a renamed field silently stops being filled.

# pip install pymupdf "pandas>=2.2"
from pathlib import Path
import pandas as pd
import pymupdf

def verify_map(template: Path, fmap_csv: Path) -> None:
    fmap = pd.read_csv(fmap_csv, dtype=str).fillna("")
    with pymupdf.open(template) as doc:
        live = {w.field_name for page in doc for w in page.widgets()}
    mapped = set(fmap.loc[fmap["maps_to"] != "", "field_name"])
    stale = mapped - live
    assert not stale, f"mapped fields no longer in the form: {sorted(stale)[:5]}"
    required = fmap[(fmap["type"] == "Text") & (fmap["read_only"] == "False") & (fmap["maps_to"] == "")]
    print(f"{len(mapped)} mapped field(s) all present; {required['field_name'].nunique()} editable text field(s) unmapped")

Run it whenever a new version of the form is downloaded. Stale mappings mean the supplier renamed fields; a jump in unmapped fields means new questions were added.

FAQ

Why do field names end in [0]? They come from XFA/LiveCycle naming, where every node is an array element. Use the names exactly as reported; the indices are part of the name.

Can I rename fields to something readable? Yes, by editing each field's /T, but it breaks compatibility with anyone else filling the same form. Keep the original names and map them instead.

Why does a field report no widgets? Hidden or orphaned fields exist in the tree without an on-page appearance. They cannot be seen or filled meaningfully; ignore them.

How do I find which fields are required? Bit 2 of /Ff marks a field as required. PyMuPDF exposes it through field_flags & 2; add it to the map so the filling job can refuse incomplete records.

Part of Filling PDF Forms with Python.