Python Doc & Data Automation

Convert PDF to PDF/A for Archiving

The records management system rejects uploads with Document is not PDF/A compliant, or an auditor's validator lists errors such as The font program is not embedded, DeviceRGB colour space used without an output intent, and Metadata in the Info dictionary does not match XMP. The files open fine everywhere; they simply do not meet the archival profile the system requires.

Root Cause

PDF/A is a restricted subset of PDF designed so a file renders identically decades from now without external resources. It forbids anything that depends on the viewing environment or can change: fonts must be embedded, device-dependent colour needs an embedded ICC output intent, encryption and JavaScript are banned, PDF/A-1 disallows transparency, and the Info dictionary must agree with an XMP packet that declares the PDF/A part and conformance level. Ordinary generators — Word exports, ReportLab reports, scanners, merges — violate one or more of these rules by default. Simply adding a PDF/A identifier to the metadata does not make a file compliant; it makes a file that claims compliance and fails validation, which is worse. Conversion has to rewrite the content that breaks the rules, and only a validator can confirm it worked.

Minimal Diagnostic

Run veraPDF, the reference open-source validator, from Python and summarise the failed rules. That list decides which conversion settings matter.

# requires veraPDF CLI on PATH: https://docs.verapdf.org/install/
import subprocess
import xml.etree.ElementTree as ET
from collections import Counter
from pathlib import Path

SOURCE = Path("in/contract-2026-0412.pdf")

def verapdf_failures(pdf_path: Path, flavour: str = "2b") -> tuple[bool, Counter]:
    try:
        proc = subprocess.run(
            ["verapdf", "--flavour", flavour, "--format", "xml", str(pdf_path)],
            capture_output=True, text=True, timeout=300,
        )
    except FileNotFoundError:
        raise SystemExit("verapdf not installed or not on PATH")
    root = ET.fromstring(proc.stdout)
    report = root.find(".//validationReport")
    if report is None:
        raise RuntimeError(f"no validation report: {proc.stderr[:300]}")
    compliant = report.get("isCompliant") == "true"
    failures = Counter()
    for rule in root.iter("rule"):
        if rule.get("status") == "failed":
            desc = (rule.findtext("description") or "").strip()
            failures[f"{rule.get('clause')}: {desc[:90]}"] += int(rule.get("failedChecks", "1"))
    return compliant, failures

if __name__ == "__main__":
    ok, fails = verapdf_failures(SOURCE)
    print("compliant" if ok else "NOT compliant")
    for rule, count in fails.most_common(10):
        print(f"  {count:>4} x {rule}")

NOT compliant
    38 x 6.2.11.4.1: The font programs for all fonts used for rendering within a conforming file shall be embedded
    12 x 6.2.4.3: DeviceRGB shall only be used if a device independent DefaultRGB colour space has been set
     1 x 6.6.4: The PDF/A version and conformance level of a file shall be specified using the PDF/A Identification extension schema

Three rule families: unembedded fonts, uncalibrated colour, missing identification. All three are fixed by a proper conversion rather than by editing metadata.

What PDF/A requires and what usually breaks it Six cards. All fonts embedded, broken by Word exports that reference standard fonts, fixed by converter font embedding. Device colour needs an output intent, broken by generators using DeviceRGB, fixed by embedding an sRGB ICC profile. Info and XMP must agree and declare the part, broken by generic metadata, fixed by the converter writing both. No encryption or JavaScript, broken by protected files, fixed by decrypting first. No transparency in PDF/A-1, broken by modern charts, fixed by choosing PDF/A-2. No external content or embedded files in PDF/A-2, broken by attachments, fixed by PDF/A-3 or removing them. Fonts embedded Word exports often reference Arial. Converter embeds it. Colour output intent DeviceRGB needs an sRGB ICC profile in the file. Info matches XMP Both stores agree and declare part and level. No encryption or JS Decrypt before converting; scripts are stripped. Transparency Banned in PDF/A-1. Target PDF/A-2b instead. Attachments Only PDF/A-3 allows arbitrary embedded files.

Fix: Convert with OCRmyPDF, Then Validate

OCRmyPDF wraps Ghostscript with sensible PDF/A settings, embeds an sRGB output intent, writes consistent metadata, and can skip OCR for pages that already have text. It is the most reliable converter to drive from Python for mixed business documents.

# pip install ocrmypdf
# system: sudo apt-get install -y ghostscript tesseract-ocr
from pathlib import Path
import ocrmypdf

SOURCE = Path("in/contract-2026-0412.pdf")
DEST = Path("archive/contract-2026-0412.pdf")

def to_pdfa(src: Path, dest: Path) -> None:
    dest.parent.mkdir(parents=True, exist_ok=True)
    try:
        ocrmypdf.ocr(
            src,
            dest,
            output_type="pdfa-2",          # changed: PDF/A-2b allows transparency, unlike PDF/A-1
            skip_text=True,                # changed: do not re-OCR pages that already have text
            optimize=1,                    # changed: lossless optimisation only, safe for archives
            pdfa_image_compression="lossless",
            progress_bar=False,
        )
    except ocrmypdf.exceptions.EncryptedPdfError:
        raise RuntimeError(f"{src.name} is encrypted: decrypt before archiving")
    except ocrmypdf.exceptions.DigitalSignatureError:
        raise RuntimeError(f"{src.name} is signed: conversion would invalidate the signature")
    except ocrmypdf.exceptions.ExitCodeException as exc:
        raise RuntimeError(f"ocrmypdf failed on {src.name}: {exc}") from exc

if __name__ == "__main__":
    to_pdfa(SOURCE, DEST)
    ok, fails = verapdf_failures(DEST)             # from the diagnostic above
    print("compliant" if ok else fails.most_common(5))

Target PDF/A-2b unless a regulation names another part. The b (basic) conformance level guarantees visual reproduction; a (accessible) additionally requires a complete tag structure that most generated documents do not have, and conversion cannot invent it. PDF/A-1 forbids transparency, which rules out many charts and signature images, and has no real advantage for new archives.

The signed-document exception is deliberate. Converting a digitally signed PDF rewrites it and invalidates the signature; the legal record is the signed original. Archive signed files as they are — PDF/A-conformant signing must happen at signing time, not afterwards.

Archival conversion pipeline Five steps. Intake rejects encrypted and signed files for separate handling. Conversion runs OCRmyPDF with output type pdfa-2 and skip text. Validation runs veraPDF with flavour 2b. Only compliant files are stored in the archive, together with the veraPDF report. Non-compliant files go to a review queue with their failed rules. Intake checks reject encrypted files; route signed files to archive as-is Convert ocrmypdf output_type=pdfa-2, skip_text, lossless images Validate verapdf --flavour 2b; parse isCompliant and failed rules Store with report archive the PDF and its XML report side by side Review queue non-compliant files with the rule list, never silently stored

Variant Fix 1: Ghostscript Directly

When OCRmyPDF is not an option, Ghostscript produces PDF/A itself. It needs an ICC profile and a small PostScript definition file that sets the output intent; the fonts are embedded automatically.

# requires: sudo apt-get install ghostscript icc-profiles-free
import subprocess
from pathlib import Path

ICC = Path("/usr/share/color/icc/sRGB.icc")          # adjust to where your sRGB profile lives
PDFA_DEF = Path("pdfa_def.ps")

PDFA_DEF_TEMPLATE = """%!
[/_objdef {icc_PDFA} /type /stream /OBJ pdfmark
[{icc_PDFA} << /N 3 >> /PUT pdfmark
[{icc_PDFA} (%s) (r) file /PUT pdfmark
[/_objdef {OutputIntent_PDFA} /type /dict /OBJ pdfmark
[{OutputIntent_PDFA} << /Type /OutputIntent /S /GTS_PDFA1
  /DestOutputProfile {icc_PDFA} /OutputConditionIdentifier (sRGB) >> /PUT pdfmark
[{Catalog} << /OutputIntents [ {OutputIntent_PDFA} ] >> /PUT pdfmark
"""

def gs_pdfa(src: Path, dest: Path) -> None:
    if not ICC.exists():
        raise FileNotFoundError(f"sRGB ICC profile not found at {ICC}")
    PDFA_DEF.write_text(PDFA_DEF_TEMPLATE % ICC, encoding="ascii")
    cmd = [
        "gs", "-dPDFA=2", "-dBATCH", "-dNOPAUSE", "-dQUIET", "-dNOOUTERSAVE",
        "-sColorConversionStrategy=RGB", "-sDEVICE=pdfwrite",
        "-dPDFACompatibilityPolicy=1",                # drop non-conforming features instead of failing
        f"-sOutputFile={dest}", str(PDFA_DEF), str(src),
    ]
    try:
        subprocess.run(cmd, check=True, capture_output=True, timeout=600)
    except subprocess.CalledProcessError as exc:
        raise RuntimeError(exc.stderr.decode(errors="replace")[:500]) from exc

PDFACompatibilityPolicy=1 tells Ghostscript to remove features it cannot make conformant — annotations with unsupported types, for example — rather than aborting. That is usually what an archive wants, but it means the output can lose interactive elements; compare annotation counts before and after if they matter.

Variant Fix 2: Validator Still Reports Metadata Mismatch

Files edited after conversion — a title fixed with another tool, pages stamped with a received date — often break the Info/XMP agreement even though the content still conforms. Update both stores together with pikepdf, which keeps them in sync:

# pip install pikepdf
from pathlib import Path
import pikepdf

def set_title_pdfa_safe(path: Path, title: str) -> None:
    tmp = path.with_suffix(".tmp.pdf")
    try:
        with pikepdf.open(path) as pdf:
            with pdf.open_metadata(set_pikepdf_as_editor=False) as meta:
                meta["dc:title"] = title                 # writes XMP and mirrors to the Info dict
            pdf.save(tmp)
    except pikepdf.PdfError as exc:
        raise RuntimeError(f"{path.name}: {exc}") from exc
    tmp.replace(path)

open_metadata updates the XMP packet and synchronises the corresponding Info entries on save, so validators see one consistent value. Revalidate after any post-conversion edit; stamping content onto pages can reintroduce unembedded fonts.

Choosing a PDF/A part PDF/A-1b does not allow transparency or JPEG2000 and does not allow embedded files, and is only suitable when a regulation requires it. PDF/A-2b allows transparency and JPEG2000, allows only other PDF/A files as attachments, and is the recommended default for new archives. PDF/A-3b allows transparency, JPEG2000 and arbitrary embedded files such as source XML, and suits e-invoicing formats. Part Transparency Embedded files Use it for PDF/A-1b not allowed none legacy mandates PDF/A-2b allowed PDF/A only default choice PDF/A-3b allowed any file e-invoices with XML

Verification

Store compliance evidence, not just the converted file. Validate every output, keep the veraPDF report next to the PDF, and check that visible content did not change during conversion.

# pip install pymupdf
import subprocess
from pathlib import Path
import pymupdf

def verify_archive(original: Path, archived: Path) -> Path:
    ok, fails = verapdf_failures(archived)                   # from the diagnostic
    assert ok, f"{archived.name} failed PDF/A-2b: {fails.most_common(3)}"
    with pymupdf.open(original) as a, pymupdf.open(archived) as b:
        assert a.page_count == b.page_count, "page count changed"
        for pa, pb in zip(a, b):
            ta = " ".join(pa.get_text().split())
            tb = " ".join(pb.get_text().split())
            if ta:                                             # scanned pages may gain OCR text
                assert ta == tb, f"page {pa.number + 1}: text changed during conversion"
    report = archived.with_suffix(".verapdf.xml")
    proc = subprocess.run(["verapdf", "--flavour", "2b", "--format", "xml", str(archived)],
                          capture_output=True, text=True, check=False)
    report.write_text(proc.stdout, encoding="utf-8")
    print(f"{archived.name}: compliant, report saved to {report.name}")
    return report

if __name__ == "__main__":
    verify_archive(Path("in/contract-2026-0412.pdf"), Path("archive/contract-2026-0412.pdf"))

Text comparison skips pages that had no text in the original, because skip_text leaves digital pages alone but scanned pages legitimately gain an OCR layer. Keep the reports for the retention period of the documents themselves — when an auditor asks how you know a 2026 contract was archived compliantly, the report is the answer.

Batch Archiving a Folder

Archival runs are usually batches: last month's signed-off contracts, a department's closed case files. The script below applies the intake rules, converts, validates, and sorts every file into exactly one of three destinations — archived with its report, kept as an original because it is signed, or queued for review — so nothing is silently skipped.

#!/usr/bin/env python3
# pip install ocrmypdf pymupdf
"""Convert a folder of PDFs to PDF/A-2b, validate each, and sort results."""
import argparse
import logging
import shutil
import sys
from pathlib import Path

import ocrmypdf
import pymupdf

log = logging.getLogger("archive")


def is_signed(path: Path) -> bool:
    with pymupdf.open(path) as doc:
        return any(w.field_type == pymupdf.PDF_WIDGET_TYPE_SIGNATURE
                   for page in doc for w in page.widgets())


def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("src", type=Path)
    ap.add_argument("archive", type=Path)
    args = ap.parse_args()
    logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
    signed_dir, review_dir = args.archive / "_signed-originals", args.archive / "_review"
    for d in (args.archive, signed_dir, review_dir):
        d.mkdir(parents=True, exist_ok=True)

    counts = {"archived": 0, "signed": 0, "review": 0}
    for src in sorted(args.src.glob("*.pdf")):
        try:
            if is_signed(src):
                shutil.copy2(src, signed_dir / src.name)
                counts["signed"] += 1
                continue
            dest = args.archive / src.name
            ocrmypdf.ocr(src, dest, output_type="pdfa-2", skip_text=True,
                         optimize=1, progress_bar=False)
            verify_archive(src, dest)          # from the verification section
            counts["archived"] += 1
        except Exception as exc:
            log.warning("%s -> review: %s", src.name, exc)
            shutil.copy2(src, review_dir / src.name)
            (args.archive / src.name).unlink(missing_ok=True)
            counts["review"] += 1
    log.info("done: %s", counts)
    return 0 if counts["review"] == 0 else 2


if __name__ == "__main__":
    sys.exit(main())

The exit code distinguishes "everything archived" (0) from "some files need a human" (2), which lets a scheduler alert on the second case without treating it as a crash. Signed files are detected through signature form fields, which covers signatures applied by common signing tools; files signed without a visible field are rare in business archives but worth spot-checking if your signing platform produces them.

FAQ

Can I just set the PDF/A flag in the metadata? No. A file that declares PDF/A but violates the rules fails validation and may be rejected more harshly than an undeclared file. Always convert and validate.

Does PDF/A conversion make files larger? Usually slightly, because fonts get embedded and an ICC profile is added (about 3 KB). Lossy image optimisation is inappropriate for archives, so do size reduction before archival conversion if needed — see reduce PDF file size with Python.

What happens to form fields? PDF/A-2b allows form fields but requires appearance streams; converters flatten or regenerate them. Flatten filled forms explicitly before archiving so the recorded values are fixed.

Can Python validate PDF/A without veraPDF? Not authoritatively. pikepdf's pdfa_status only reads the declaration. veraPDF implements the actual rule set and is what auditors and records systems use.

Part of Compressing and Optimizing PDFs with Python.