Python Doc & Data Automation

Fix PDF Form Fields Not Visible After Filling

The script reports success, the file size grows, and re-reading the PDF confirms every value is present — but the form opens with all boxes blank:

>>> PdfReader("out/filled.pdf").get_fields()["applicant_name"]["/V"]
'Dana Whitfield'
# ...yet the same file in Chrome, Firefox or macOS Preview shows an empty box.

The data is in the file. What is missing is the picture of the data. A PDF form field stores its value in the field dictionary (/V) and stores what the reader draws in a separate appearance stream (/AP) attached to the widget annotation. Writing /V without regenerating /AP leaves the old, empty appearance in place, and every viewer that renders appearance streams verbatim — Chrome's PDFium, Firefox's pdf.js, macOS Preview, most mobile readers — faithfully draws the blank box it was handed.

Root Cause

Acrobat is the outlier that hides the bug. It honours the document-level /NeedAppearances flag, which means "the appearance streams are stale, rebuild them on open", so a file filled without appearances looks correct there and broken everywhere else. When that flag is absent and no fresh /AP was written, there is nothing to draw and nothing instructing the viewer to draw it.

Why the value is in the file but not on the page A fill script writes the field value slash V. On the upper path nothing else changes, so the widget keeps its original empty appearance stream and viewers that render appearance streams verbatim, such as Chrome PDFium, Firefox pdf.js and macOS Preview, show an empty box. On the lower path the script also regenerates the appearance stream and sets the document NeedAppearances flag, so both stream-rendering viewers and Acrobat draw the value. A footnote records that Acrobat alone can mask the bug. Fill script runs writes /V into the field /AP left untouched stale, empty appearance /AP regenerated + /NeedAppearances true Chrome: empty box Preview: empty box Chrome: value shown Acrobat: value shown Acrobat rebuilds stale appearances on open — testing only there hides the defect

A second, less common cause produces the same symptom: the field's default appearance string (/DA) names a font that is not present in the form's resource dictionary (/AcroForm /DR). The viewer then has a stream to draw but no font to draw it with, and renders nothing. Both causes are separated by the diagnostic below.

Minimal Diagnostic

Run this against the output file. It reports, per field, whether a value exists, whether an appearance stream exists, and whether the document asks viewers to rebuild appearances.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader

FILLED_PDF = Path("out/filled.pdf")

def appearance_report(pdf_path: Path) -> None:
    """Show whether each filled field also carries a drawable appearance stream."""
    if not pdf_path.exists():
        raise FileNotFoundError(f"No such file: {pdf_path}")
    try:
        reader = PdfReader(str(pdf_path))
    except Exception as exc:
        raise RuntimeError(f"Could not parse {pdf_path}: {exc}") from exc

    acro = reader.trailer["/Root"].get("/AcroForm", {})
    print(f"/NeedAppearances = {acro.get('/NeedAppearances', False)}")

    for page_no, page in enumerate(reader.pages, start=1):
        for annot in page.get("/Annots") or []:
            obj = annot.get_object()
            if obj.get("/Subtype") != "/Widget":
                continue
            parent = obj.get("/Parent")
            name = obj.get("/T") or (parent.get_object().get("/T") if parent else "?")
            value = obj.get("/V") or (parent.get_object().get("/V") if parent else None)
            has_ap = "/AP" in obj
            print(f"p{page_no} {str(name):<28} value={str(value)!r:<20} /AP={'yes' if has_ap else 'NO'}")

if __name__ == "__main__":
    appearance_report(FILLED_PDF)

The broken file looks like this:

/NeedAppearances = False
p1 applicant_name               value='Dana Whitfield'    /AP=NO
p1 applicant_email              value='[email protected]'  /AP=NO

A value with /AP=NO and /NeedAppearances = False is the confirmed root cause: nothing in the file tells any viewer what to paint. The four combinations of those two markers map cleanly onto what a reader will show:

Appearance stream against NeedAppearances: what renders A two by two matrix. Rows are the NeedAppearances flag, false on the top row and true on the bottom row. Columns are the widget appearance stream, absent on the left and present on the right. Flag false with no stream renders nothing, which is the reported bug. Flag false with a stream renders in all viewers. Flag true with no stream renders only in Acrobat and other viewers that rebuild appearances. Flag true with a stream renders everywhere and is the recommended combination. Widget appearance stream /AP absent present flag false flag true /NeedAppearances nothing renders the reported bug value is in the file only renders everywhere stream is drawn as-is no rebuild needed Acrobat only rebuilt on open blank in most readers ship this one stream plus rebuild hint covers every viewer The fix below moves a file from the top-left cell to the bottom-right cell

The Fix

Do both things — generate the appearance streams and set the flag. They cover different viewers, and the pair costs one extra line.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader, PdfWriter
from pypdf.generic import BooleanObject, NameObject

SRC = Path("data/application_form.pdf")
DEST = Path("out/filled.pdf")

def fill_visibly(src: Path, dest: Path, values: dict[str, str]) -> Path:
    """Fill an AcroForm so the values render in every viewer, not just Acrobat."""
    reader = PdfReader(str(src))
    # clone_from copies the /AcroForm dictionary; append() can drop it entirely
    writer = PdfWriter(clone_from=reader)

    # 1. Tell viewers the appearances are stale and must be rebuilt on open.
    writer._root_object[NameObject("/AcroForm")][NameObject("/NeedAppearances")] = BooleanObject(True)

    # 2. Write the values AND let pypdf build an appearance stream for each one,
    #    which is what auto_regenerate=True does. Every page, not just page 1.
    for page in writer.pages:
        writer.update_page_form_field_values(page, values, auto_regenerate=True)

    dest.parent.mkdir(parents=True, exist_ok=True)
    with dest.open("wb") as fh:
        writer.write(fh)
    return dest

if __name__ == "__main__":
    fill_visibly(SRC, DEST, {"applicant_name": "Dana Whitfield", "applicant_email": "[email protected]"})
    print(f"Wrote {DEST}")

Re-run the diagnostic on the new output and the two markers flip:

/NeedAppearances = True
p1 applicant_name               value='Dana Whitfield'    /AP=yes

Variant Fix 1: Values Render as Empty Boxes or Wrong Glyphs

If the text appears as hollow rectangles, or in a font nothing on the page uses, the appearance stream exists but the /DA string names a font missing from the form's resources. Inspect what the field asks for:

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader

def font_report(pdf_path: Path) -> None:
    """Compare each field's requested font against the form's resource dictionary."""
    reader = PdfReader(str(pdf_path))
    acro = reader.trailer["/Root"].get("/AcroForm", {})
    available = list((acro.get("/DR", {}).get("/Font", {}) or {}).keys())
    print("fonts available in /DR:", available)
    print("document default /DA:", acro.get("/DA"))
    for name, field in (reader.get_fields() or {}).items():
        da = field.get("/DA")
        if da:
            print(f"{name:<28} /DA={da}")

if __name__ == "__main__":
    font_report(Path("data/application_form.pdf"))

Output on a form built by a tool that did not embed its font:

fonts available in /DR: ['/Helv', '/ZaDb']
document default /DA: /Helv 0 Tf 0 g
applicant_name               /DA=/Arial 10 Tf 0 g

/Arial is not in /DR, so nothing renders. Point the field at a font that is present:

# pip install "pypdf>=4.2,<6"
from pypdf.generic import NameObject, TextStringObject

def repoint_font(writer, field_name: str, da: str = "/Helv 10 Tf 0 g") -> None:
    """Rewrite a field's default appearance to a font present in /DR."""
    for page in writer.pages:
        for annot in page.get("/Annots") or []:
            obj = annot.get_object()
            target = obj.get("/T") or (obj.get("/Parent").get_object().get("/T") if obj.get("/Parent") else None)
            if str(target) == field_name:
                # /Helv and /ZaDb are the two fonts every AcroForm viewer provides
                obj[NameObject("/DA")] = TextStringObject(da)

Setting the size to 0 (/Helv 0 Tf 0 g) turns on auto-sizing, which is the safe choice for fields whose content length varies — a 40-character company name in a fixed 12 pt field is clipped rather than shrunk.

Variant Fix 2: Only Checkboxes Stay Blank

If text fields render but buttons do not, the cause is different: the widget's appearance state (/AS) was not moved to match the new value. pypdf's auto_regenerate handles text but not the button state on some files, so set both explicitly.

# pip install "pypdf>=4.2,<6"
from pypdf.generic import NameObject

def tick(writer, field_name: str, on_state: str = "/Yes") -> None:
    """Set both the field value and the widget's drawn state for a checkbox."""
    for page in writer.pages:
        for annot in page.get("/Annots") or []:
            obj = annot.get_object()
            target = obj.get("/T") or (obj.get("/Parent").get_object().get("/T") if obj.get("/Parent") else None)
            if str(target) != field_name:
                continue
            obj[NameObject("/V")] = NameObject(on_state)   # the data
            obj[NameObject("/AS")] = NameObject(on_state)  # what gets drawn

Confirm on_state against the widget's own dictionary rather than assuming /Yes — the full lookup is in filling PDF forms with Python, whose diagnostic step prints the legal states for every button field.

Variant Fix 3: Fields Vanish Entirely After the Write

If the output has no fields at all — get_fields() returns None — the form dictionary was lost during the copy, not during the fill. This happens when the writer is built by appending pages instead of cloning the document:

# pip install "pypdf>=4.2,<6"
from pypdf import PdfReader, PdfWriter

reader = PdfReader("data/form.pdf")

# WRONG: copies pages, drops /AcroForm on many files
broken = PdfWriter()
broken.append(reader)

# RIGHT: copies the whole object graph, form dictionary included
fixed = PdfWriter(clone_from=reader)
print("fields preserved:", bool(fixed._root_object.get("/AcroForm")))

The same trap catches page-level operations: splitting or merging a form with the techniques in merging and splitting PDF documents keeps the pages but not necessarily the form, so fill before you assemble.

Verification

Assert on all three markers — value, appearance stream and flag — so a regression fails in CI rather than in a recipient's browser.

# pip install "pypdf>=4.2,<6"
from pathlib import Path
from pypdf import PdfReader

def assert_renderable(pdf_path: Path, expected: dict[str, str]) -> None:
    """Fail unless each value is present AND drawable."""
    reader = PdfReader(str(pdf_path))
    acro = reader.trailer["/Root"].get("/AcroForm", {})
    need_appearances = bool(acro.get("/NeedAppearances", False))

    has_ap = {}
    for page in reader.pages:
        for annot in page.get("/Annots") or []:
            obj = annot.get_object()
            parent = obj.get("/Parent")
            name = obj.get("/T") or (parent.get_object().get("/T") if parent else None)
            if name:
                has_ap[str(name)] = "/AP" in obj

    fields = reader.get_fields() or {}
    for name, want in expected.items():
        got = fields.get(name, {}).get("/V")
        assert got is not None, f"{name}: no value written"
        assert str(got).lstrip("/") == str(want).lstrip("/"), f"{name}: {got!r} != {want!r}"
        assert need_appearances or has_ap.get(name), (
            f"{name}: value present but nothing to draw — set /NeedAppearances or regenerate /AP"
        )
    print(f"{len(expected)} field(s) verified renderable in {pdf_path.name}")

if __name__ == "__main__":
    assert_renderable(Path("out/filled.pdf"), {"applicant_name": "Dana Whitfield"})

For an end-to-end proof that does not trust any PDF library, rasterise page 1 and check that ink appears inside the widget rectangle:

# pip install pdf2image "pypdf>=4.2,<6"   (needs poppler-utils on the system)
from pathlib import Path
from pdf2image import convert_from_path
from pypdf import PdfReader

def ink_in_widget(pdf_path: Path, field_name: str, dpi: int = 150) -> bool:
    """Rasterise page 1 and report whether the field's box contains any dark pixels."""
    reader = PdfReader(str(pdf_path))
    page = reader.pages[0]
    rect = None
    for annot in page.get("/Annots") or []:
        obj = annot.get_object()
        parent = obj.get("/Parent")
        name = obj.get("/T") or (parent.get_object().get("/T") if parent else None)
        if str(name) == field_name:
            rect = [float(v) for v in obj["/Rect"]]
            break
    if rect is None:
        raise KeyError(f"No widget for {field_name}")

    scale = dpi / 72.0
    image = convert_from_path(str(pdf_path), dpi=dpi, first_page=1, last_page=1)[0]
    height = image.size[1]
    x0, y0, x1, y1 = rect
    # PDF origin is bottom-left; image origin is top-left, so flip the y axis.
    crop = image.crop((int(x0 * scale), int(height - y1 * scale),
                       int(x1 * scale), int(height - y0 * scale))).convert("L")
    return min(crop.getdata()) < 200        # any pixel darker than light grey

A True result is the strongest available evidence: the value is not merely stored, it is painted where a human will look for it. Wire the three checks into the order below and a broken fill can never reach a recipient:

The order to verify a filled form in CI A four stage pipeline. Stage one asserts the field value was written. Stage two asserts a widget appearance stream exists. Stage three asserts the NeedAppearances flag is set. Stage four rasterises the page and asserts dark pixels appear inside the widget rectangle. Each stage is cheaper than the next, so the pipeline fails fast, and only the last stage requires poppler. 1. value present get_fields()["/V"] < 10 ms 2. /AP exists per widget annot < 20 ms 3. flag set /NeedAppearances < 5 ms 4. ink visible raster crop check ~400 ms Fail fast: cheap assertions first, rasterisation last Stages 1 to 3 need only pypdf; stage 4 adds poppler, so run it on one sample per batch

FAQ

Why does the file look correct in Acrobat but blank everywhere else? Acrobat honours /NeedAppearances and rebuilds stale appearance streams when it opens a file. Chrome, Firefox and Preview render the stream exactly as stored. Testing only in Acrobat therefore hides the defect — always open the output in a stream-rendering viewer before shipping.

Is setting /NeedAppearances alone enough? Often, but not universally: some mobile and embedded viewers ignore the flag. Generating the appearance stream with auto_regenerate=True covers those, and the flag covers viewers whose own layout differs from pypdf's. Ship both.

Does regenerating appearances change the visual design of the form? It redraws only the field contents, using the font and size from /DA inside the widget's existing rectangle. Borders, background shading and page artwork come from the widget's /MK dictionary and the page content stream, neither of which is touched.

The values show but are clipped at the right edge — what now? The field is fixed-size and the string is longer than the box. Switch the /DA size to 0 for auto-sizing, or shorten the value. Auto-sizing is generally preferable on names and addresses; see the font section above for the exact edit.

Can I flatten instead of fixing appearances? Flattening merges the appearance stream into the page — but if no appearance exists, flattening produces a page with nothing on it. Fix the appearances first, then flatten, as described in flatten PDF form fields with Python.

Part of Filling PDF Forms with Python.