Python Doc & Data Automation

Fix Merged PDF Bookmarks Lost

The board pack is built by merging twelve PDFs — agenda, minutes, finance report, risk register — each with its own bookmarks. The merged file opens with an empty bookmark panel. A second version of the script keeps bookmarks, but clicking "Cash flow" jumps to page 3 of the agenda instead of page 41 of the finance section, and all 180 bookmarks from all twelve files sit in one flat list with no indication of which document they belong to.

from pypdf import PdfWriter
writer = PdfWriter()
for path in sources:
    for page in PdfReader(path).pages:
        writer.add_page(page)            # pages copied, outline not
writer.write("board-pack.pdf")

Root Cause

Bookmarks are not attached to pages. They live in the document catalogue's /Outlines tree, and each entry points to a page object through a destination. Copying pages one by one with add_page brings the page objects across but not the source catalogue, so the outline is simply never copied. When a merge tool does import outlines, destinations must be remapped from the source page objects to the new page objects; tools that copy destinations as raw page numbers, or that resolve named destinations against the wrong document, produce bookmarks pointing to the right page number in the wrong file. And importing every source outline at the top level loses the grouping readers need, because the merged document's structure — one section per source file — was never represented in the outline at all.

Minimal Diagnostic

Compare each source's outline with the merged file's outline: count entries, and check where each bookmark points in the output relative to where its source section starts.

# pip install pymupdf
from pathlib import Path
import pymupdf

SOURCES = sorted(Path("in/board-pack").glob("*.pdf"))
MERGED = Path("out/board-pack.pdf")

def outline_report(sources: list[Path], merged: Path) -> None:
    offset = 0
    expected = []
    for path in sources:
        with pymupdf.open(path) as doc:
            toc = doc.get_toc()
            print(f"{path.name:<32} pages {doc.page_count:>3}  bookmarks {len(toc):>3}")
            expected += [(title, page + offset) for _lvl, title, page in toc]
            offset += doc.page_count
    with pymupdf.open(merged) as doc:
        merged_toc = doc.get_toc()
    print(f"merged: {len(merged_toc)} bookmarks, expected {len(expected)}")
    actual = {title: page for _lvl, title, page in merged_toc}
    wrong = [(t, p, actual[t]) for t, p in expected if t in actual and actual[t] != p]
    for title, want, got in wrong[:5]:
        print(f"   {title!r}: points to page {got}, should be {want}")

if __name__ == "__main__":
    outline_report(SOURCES, MERGED)

01-agenda.pdf                    pages   3  bookmarks   4
02-minutes.pdf                   pages   9  bookmarks  11
03-finance-report.pdf            pages  38  bookmarks  46
...
merged: 0 bookmarks, expected 181

Zero bookmarks confirms the outline was not copied. After a partial fix, the same report lists bookmarks whose target page lacks the offset of the preceding files.

Outline after naive and structured merges The left panel shows the naive merge results, first with no bookmarks at all and then with every source bookmark in one flat list where Cash flow points to page 3 because the page offset of earlier files was not added. The right panel shows the structured outline, with one top-level bookmark per source document and that document's own bookmarks nested beneath, each pointing to the correct page in the merged file. Naive merge (no bookmarks) -- or, flat -- Introduction p.1 Cash flow p.3 Risk heat map p.2 Structured merge 1 Agenda p.1 2 Minutes p.4 3 Finance report p.13 Cash flow p.41 4 Risk register p.51

Fix: Rebuild the Outline with Offsets and One Parent per File

PyMuPDF's insert_pdf copies pages, and get_toc/set_toc read and write the outline as a simple list of [level, title, page]. Build the merged outline explicitly: one level-1 entry per source file pointing to its first page, and the file's own bookmarks shifted down one level and offset by the pages already merged. Changed lines carry comments.

# pip install pymupdf
from pathlib import Path
import pymupdf

SOURCES = sorted(Path("in/board-pack").glob("*.pdf"))
DEST = Path("out/board-pack.pdf")

def section_title(path: Path) -> str:
    stem = path.stem.split("-", 1)[-1] if path.stem[:2].isdigit() else path.stem
    return stem.replace("-", " ").strip().capitalize()

def merge_with_outline(sources: list[Path], dest: Path) -> int:
    merged = pymupdf.open()
    toc: list[list] = []
    for path in sources:
        try:
            src = pymupdf.open(path)
        except (pymupdf.FileDataError, RuntimeError) as exc:
            raise RuntimeError(f"cannot open {path.name}: {exc}") from exc
        with src:
            offset = merged.page_count                                    # changed: pages already merged
            merged.insert_pdf(src, links=True, annots=True)
            toc.append([1, section_title(path), offset + 1])              # changed: one parent per file
            for level, title, page, *_ in src.get_toc(simple=False):
                if page < 1:
                    continue                                              # unresolvable destination
                toc.append([level + 1, title, page + offset])             # changed: nest and offset
    merged.set_toc(toc)                                                   # changed: write the outline
    dest.parent.mkdir(parents=True, exist_ok=True)
    merged.save(dest, garbage=4, deflate=True)
    merged.close()
    return len(toc)

if __name__ == "__main__":
    print(f"{merge_with_outline(SOURCES, DEST)} bookmarks written")

set_toc requires that levels never jump by more than one between consecutive entries — a source whose outline starts at level 2, or jumps from 1 to 3, raises ValueError: bad hierarchy level. Nesting under a per-file parent absorbs the most common case; for malformed source outlines, normalise levels before appending, as shown in the verification section. Using simple=False returns destination details, which lets the loop skip bookmarks whose targets were never resolvable in the source.

Building the merged outline For each source file, record the current page count of the merged document as the offset, insert the source pages, add a level one bookmark titled after the file pointing to offset plus one, and add each of the source's bookmarks with its level increased by one and its page increased by the offset. After all files, write the whole list with set_toc and save. Record offset offset = merged.page_count before inserting the next file Insert pages insert_pdf(src, links=True, annots=True) Parent bookmark [1, 'Finance report', offset + 1] Nested bookmarks [level + 1, title, page + offset] for each source entry Write outline set_toc(toc) once, then save with garbage collection

Variant Fix 1: pypdf with Imported Outlines

In pypdf, PdfWriter.append imports each file's outline and remaps destinations when import_outline=True, and outline_item lets you place each file's bookmarks under a new parent:

# pip install "pypdf>=4.0"
from pathlib import Path
from pypdf import PdfReader, PdfWriter

def merge_pypdf(sources: list[Path], dest: Path) -> None:
    writer = PdfWriter()
    for path in sources:
        try:
            reader = PdfReader(path)
        except Exception as exc:
            raise RuntimeError(f"cannot read {path.name}: {exc}") from exc
        writer.append(
            reader,
            outline_item=section_title(path),     # parent bookmark for this file, pointing to its first page
            import_outline=True,                  # bring the source outline, nested under the parent
        )
    writer.page_mode = "/UseOutlines"             # open with the bookmark panel visible
    dest.parent.mkdir(parents=True, exist_ok=True)
    with dest.open("wb") as fh:
        writer.write(fh)

append remaps destinations to the newly added page objects, which avoids the offset arithmetic. Setting page_mode to /UseOutlines makes viewers open the bookmark panel, which is often the difference between readers noticing the outline and not. The general merge job, including ordering and error handling, is in batch merge PDFs with a Python script.

Variant Fix 2: Sources Without Bookmarks

Some inputs — scans, exports from systems that never create outlines — have no bookmarks at all, so the merged outline has only the parent entry for them. Generate useful child bookmarks from headings detected by font size:

# pip install pymupdf
import pymupdf

def headings_as_toc(doc: pymupdf.Document, min_size_ratio: float = 1.35, max_per_page: int = 3) -> list[list]:
    sizes = [s["size"] for page in doc for b in page.get_text("dict")["blocks"]
             for l in b.get("lines", []) for s in l["spans"] if s["text"].strip()]
    if not sizes:
        return []
    body = sorted(sizes)[len(sizes) // 2]                      # median span size ~ body text
    toc = []
    for page in doc:
        found = 0
        for b in page.get_text("dict")["blocks"]:
            for l in b.get("lines", []):
                text = "".join(s["text"] for s in l["spans"]).strip()
                size = max((s["size"] for s in l["spans"]), default=0)
                if text and size >= body * min_size_ratio and len(text) < 90 and found < max_per_page:
                    toc.append([1, text, page.number + 1])
                    found += 1
    return toc

Use it only when get_toc() is empty, and keep the generated entries at one level under the file's parent. Large-font text catches most section headings in reports and slide exports; it also catches the occasional big number on a dashboard page, which is why the per-page cap exists.

Outline source per input file For each input the tree asks whether get_toc returns entries. Files with a valid outline have it nested under their parent bookmark with offsets. Files whose bookmarks use named destinations are imported with pypdf append, which resolves and remaps them. Files without any outline get child bookmarks generated from large-font headings. Scanned files without text get only the parent bookmark. Does the input have bookmarks? get_toc(simple=False) valid outline Nest with offset level + 1, page + offset named destinations pypdf append remaps destinations no outline, has text From headings font size heuristic scanned Parent only one bookmark per file

Verification

Check the outline's structure and every destination: levels form a valid hierarchy, every bookmark points inside the document, and each file's parent bookmark lands on a page whose text belongs to that file.

# pip install pymupdf
from pathlib import Path
import pymupdf

def normalise_levels(toc: list[list]) -> list[list]:
    """Clamp level jumps so set_toc accepts the outline."""
    fixed, previous = [], 0
    for level, title, page, *rest in toc:
        level = max(1, min(level, previous + 1))
        fixed.append([level, title, page, *rest])
        previous = level
    return fixed

def verify_outline(merged: Path, sources: list[Path]) -> None:
    with pymupdf.open(merged) as doc:
        toc = doc.get_toc()
        assert toc, "merged file has no bookmarks"
        assert toc == normalise_levels(toc), "outline has invalid level jumps"
        bad = [(t, p) for _l, t, p in toc if not 1 <= p <= doc.page_count]
        assert not bad, f"bookmarks outside the document: {bad[:3]}"
        parents = [(t, p) for l, t, p in toc if l == 1]
        assert len(parents) == len(sources), f"{len(parents)} parent bookmarks for {len(sources)} files"
        offset = 0
        for (title, page), path in zip(parents, sources):
            with pymupdf.open(path) as src:
                first_words = " ".join(src[0].get_text().split()[:8])
                assert page == offset + 1, f"{title!r} points to {page}, expected {offset + 1}"
                merged_words = " ".join(doc[page - 1].get_text().split()[:8])
                assert first_words == merged_words, f"{title!r} lands on a page from another file"
                offset += src.page_count
    print(f"{merged.name}: {len(toc)} bookmarks verified across {len(sources)} files")

Comparing the first words of each source's first page with the page its parent bookmark targets catches offset errors that a range check cannot. Run the check on every generated pack; readers depend on the bookmarks to navigate a 200-page document and rarely report that they are wrong — they just stop trusting the pack.

FAQ

Why does PdfMerger work differently from PdfWriter?PdfMerger was the older merge API and is deprecated in current pypdf; PdfWriter.append provides the same outline handling. See fix pypdf PdfFileReader deprecation error for the renamed APIs.

Can bookmarks keep their colours and bold styles? With PyMuPDF, get_toc(simple=False) returns a dictionary per entry including color and bold; pass the same dictionaries back as the fourth element to set_toc.

Do internal links inside documents survive merging? Links to pages within the same source survive with insert_pdf(..., links=True), which remaps them. Links to named destinations may need the pypdf path.

Why are bookmark titles garbled? The source stored titles in PDFDocEncoding or UTF-16 with a byte-order mark that one library decoded wrongly. Decode explicitly or re-title bookmarks from filenames.

Part of Merging and Splitting PDF Documents.

/html>