Python Doc & Data Automation

Fix LibreOffice Headless Conversion Timeout

The first conversion works. The second hangs forever, or the batch dies part way through:

subprocess.TimeoutExpired: Command '['soffice', '--headless', '--convert-to', 'pdf', ...]'
timed out after 60 seconds

javaldx: Could not find a Java Runtime Environment!
Error: source file could not be loaded

And the tell-tale one, which explains most of the others:

soffice: Application error — the user profile is locked

A headless LibreOffice is still a full office suite with a single-instance model and a user profile on disk. Run two conversions at once against the same profile and the second waits for a lock the first will never release, because the first is also headless and has no window to close.

Root Cause

soffice starts, checks for a running instance that owns the user profile directory, and if one exists it forwards the request to it and exits. In an interactive session that is what makes a second document open in the same application. In a batch job it means the second subprocess.run call returns immediately while the first process does the work — or blocks indefinitely if the first has already exited without releasing .~lock files.

Three related causes produce the same hang. A crashed previous run leaves lock files behind, so every subsequent start waits. A container without a writable HOME cannot create the profile at all and stalls on the first run. And a document containing an OLE object or a font LibreOffice must substitute can genuinely take minutes, which is a slow conversion rather than a deadlock and needs a longer timeout, not a different fix.

One profile, one conversion at a time On the left, three conversion processes all point at the default user profile directory. The first acquires the lock and converts; the second and third wait on the lock and eventually time out. On the right, each process is given its own temporary profile directory with the user installation switch, so all three convert concurrently with no contention. Shared profile: serialised, then stuck Isolated profiles: parallel convert A convert B convert C ~/.config one lock B and C wait then time out convert A convert B convert C profile-a profile-b profile-c no contention -env:UserInstallation gives each process its own profile — the single change that fixes most batch hangs Each profile costs a few megabytes and about a second of first-run setup

Minimal Diagnostic

Confirm which of the three causes applies before changing the code.

# 1. Is an instance already running and holding the profile?
pgrep -a soffice

# 2. Are stale locks left over from a crash?
find "$HOME/.config/libreoffice" -name ".~lock*" -o -name "*.lock" 2>/dev/null

# 3. Does a single conversion work at all, with a generous timeout?
time timeout 180 soffice --headless --convert-to pdf --outdir /tmp/out data/report.docx

# 4. Is HOME writable in this environment? (containers often set it to /)
python3 -c "import os, tempfile; print(os.environ.get('HOME'), os.access(os.environ.get('HOME','/'), os.W_OK))"

A single conversion that succeeds in step 3 while the batch hangs points squarely at profile contention. A single conversion that also hangs points at the environment — no writable HOME, or a missing font cache.

# Standard library only
import shutil
import subprocess
from pathlib import Path

def soffice_binary() -> str:
    """Locate the LibreOffice binary, failing with a useful message if absent."""
    for name in ("soffice", "libreoffice"):
        found = shutil.which(name)
        if found:
            return found
    raise RuntimeError("LibreOffice not found on PATH — install libreoffice-writer")

def version() -> str:
    result = subprocess.run([soffice_binary(), "--version"], capture_output=True,
                            text=True, timeout=30)
    return result.stdout.strip() or result.stderr.strip()

if __name__ == "__main__":
    print(version())

The Fix: Isolated Profile, Hard Timeout, Cleanup

# Standard library only (LibreOffice must be installed)
import shutil
import subprocess
import tempfile
from pathlib import Path

def convert_to_pdf(src: Path, out_dir: Path, timeout: int = 180) -> Path:
    """Convert one DOCX to PDF in an isolated profile, with a hard timeout."""
    if not src.exists():
        raise FileNotFoundError(f"No such document: {src}")
    binary = shutil.which("soffice") or shutil.which("libreoffice")
    if binary is None:
        raise RuntimeError("LibreOffice not found on PATH")

    out_dir.mkdir(parents=True, exist_ok=True)
    profile = Path(tempfile.mkdtemp(prefix="lo-profile-"))
    try:
        result = subprocess.run(
            [
                binary,
                "--headless",
                "--norestore",                       # do not try to recover a crashed session
                "--nolockcheck",
                "--nodefault",
                f"-env:UserInstallation=file://{profile}",   # the key argument
                "--convert-to", "pdf:writer_pdf_Export",
                "--outdir", str(out_dir),
                str(src),
            ],
            capture_output=True,
            text=True,
            timeout=timeout,
            check=False,
        )
    except subprocess.TimeoutExpired as exc:
        raise RuntimeError(f"{src.name}: conversion exceeded {timeout}s") from exc
    finally:
        shutil.rmtree(profile, ignore_errors=True)   # always clean up the profile

    dest = out_dir / f"{src.stem}.pdf"
    if result.returncode != 0 or not dest.exists():
        raise RuntimeError(
            f"{src.name}: conversion failed (exit {result.returncode})\n"
            f"stdout: {result.stdout.strip()[:300]}\nstderr: {result.stderr.strip()[:300]}"
        )
    return dest

if __name__ == "__main__":
    print(convert_to_pdf(Path("data/report.docx"), Path("out")))

Four of those arguments earn their place. -env:UserInstallation gives the process a private profile, which removes the lock contention entirely. --norestore stops LibreOffice attempting session recovery, which is itself a common source of hangs after a killed run. Checking that the output file exists matters because soffice returns exit code 0 on several failures. And removing the profile in a finally block stops a long batch filling the disk with abandoned profiles.

Variant Fix 1: Converting a Batch in Parallel

With isolated profiles, parallelism is safe. Bound it by core count — each LibreOffice process is memory-hungry.

# Standard library only
from concurrent.futures import ProcessPoolExecutor, as_completed
from pathlib import Path

def convert_many(sources: list[Path], out_dir: Path, workers: int = 4,
                 timeout: int = 180) -> dict[str, str]:
    """Convert many documents concurrently, returning a per-file outcome."""
    outcomes: dict[str, str] = {}
    with ProcessPoolExecutor(max_workers=workers) as pool:
        futures = {pool.submit(convert_to_pdf, src, out_dir, timeout): src for src in sources}
        for future in as_completed(futures):
            src = futures[future]
            try:
                outcomes[src.name] = str(future.result())
            except Exception as exc:               # one bad document must not stop the batch
                outcomes[src.name] = f"FAILED: {exc}"
    return outcomes

if __name__ == "__main__":
    files = sorted(Path("data/letters").glob("*.docx"))
    for name, outcome in convert_many(files, Path("out/pdf")).items():
        print(f"{name}: {outcome}")

Four workers is a sensible default: each process peaks around 300–500 MB, so eight workers on a 4 GB container will be killed by the OOM reaper rather than finishing faster. Collecting failures instead of raising keeps a 5,000-letter run alive when one document is corrupt — the batch discipline described in scheduling and logging automation jobs.

Variant Fix 2: Retry With Escalating Timeouts

Some documents are genuinely slow: embedded objects, large images, or fonts that must be substituted. Retrying with a longer budget separates "slow" from "stuck".

# Standard library only
import time
from pathlib import Path

def convert_with_retry(src: Path, out_dir: Path,
                       timeouts: tuple[int, ...] = (60, 180, 420)) -> Path:
    """Try progressively longer budgets before declaring a document unconvertible."""
    last_error: Exception | None = None
    for attempt, timeout in enumerate(timeouts, start=1):
        try:
            return convert_to_pdf(src, out_dir, timeout=timeout)
        except RuntimeError as exc:
            last_error = exc
            print(f"{src.name}: attempt {attempt} failed within {timeout}s")
            time.sleep(2 * attempt)          # let any orphaned process exit
    raise RuntimeError(f"{src.name}: unconvertible after {len(timeouts)} attempts") from last_error

Variant Fix 3: Container and CI Environments

Minimal images need three things beyond the package itself: a writable HOME, the font cache, and no Java expectation for simple Writer conversions.

FROM python:3.12-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
      libreoffice-writer \
      fonts-dejavu-core fonts-liberation \
    && fc-cache -f \
    && rm -rf /var/lib/apt/lists/*

# LibreOffice needs a writable HOME to create its profile
ENV HOME=/tmp

Installing libreoffice-writer rather than the full libreoffice metapackage keeps the image roughly a gigabyte smaller and removes the Java dependency that produces the javaldx warning. Missing fonts do not fail the conversion — they silently substitute, which changes the page layout and can shift a table onto a second page, so install the fonts your templates use. The wider engine comparison, including the Windows-only docx2pdf route, is in converting DOCX to PDF with Python and fix docx2pdf error on Linux.

Triaging a headless conversion hang Three checks in order. If another soffice process is running, the cause is profile contention and the fix is an isolated user installation per conversion. If stale lock files exist from a crash, the fix is to remove them and adopt disposable profiles so they cannot accumulate. If a single conversion also hangs, the cause is environmental, such as a read only home directory or a missing font cache, and the fix is in the image rather than the code. pgrep soffice instance running? Profile contention give every conversion its own -env:UserInstallation find .~lock* stale locks? Crash residue remove them; disposable profiles stop them accumulating single convert also hangs? Environment writable HOME, font cache, longer timeout — fix the image

Verification

Assert that the PDF exists, has pages, and contains text from the source — an empty or one-page PDF from a ten-page document is a silent failure the exit code will not report.

# pip install "pypdf>=4.2,<6" "python-docx>=1.1,<2"
from pathlib import Path
from docx import Document
from pypdf import PdfReader

def assert_converted(docx_path: Path, pdf_path: Path, min_pages: int = 1) -> None:
    """Fail unless the PDF exists, has pages, and shares text with the source."""
    assert pdf_path.exists(), f"{pdf_path} was not produced"
    assert pdf_path.stat().st_size > 1000, f"{pdf_path} is suspiciously small"

    reader = PdfReader(str(pdf_path))
    assert len(reader.pages) >= min_pages, (
        f"{pdf_path.name} has {len(reader.pages)} page(s), expected at least {min_pages}"
    )

    pdf_text = " ".join((page.extract_text() or "") for page in reader.pages)
    source_words = [p.text.strip() for p in Document(str(docx_path)).paragraphs if p.text.strip()]
    probe = source_words[0][:40] if source_words else ""
    if probe:
        assert probe.split()[0] in pdf_text, (
            f"{pdf_path.name} does not contain text from {docx_path.name} — blank render?"
        )
    print(f"{pdf_path.name}: {len(reader.pages)} page(s) verified")

if __name__ == "__main__":
    assert_converted(Path("data/report.docx"), Path("out/report.pdf"))

Checking for shared text is what catches a font-substitution disaster or a conversion that produced a blank page set. Run it on every converted file in a batch; it costs milliseconds and it is the only automated way to know the PDF is not empty.

How many conversion workers are useful Documents converted per minute against worker count on a four core container with four gigabytes of memory. One worker manages about eighteen. Two managed about thirty four. Four manage about sixty. Eight do not improve on four and risk being killed by the out of memory reaper, because each LibreOffice process peaks near half a gigabyte. Documents per minute by worker count (4 cores, 4 GB) 1 worker 18 2 workers 34 4 workers 60 8 workers 57, OOM risk Each process peaks near 500 MB, so memory caps the useful worker count before CPU does

FAQ

Why does the second conversion return instantly but produce nothing?soffice detected a running instance, forwarded the request and exited. The work either happens in the other process or not at all. Isolated profiles stop the detection and give each call its own process.

Is unoconv a better option? It solves the same problem by keeping one listener process alive, which is efficient but adds a daemon to supervise. For batch jobs, disposable profiles plus a process pool are simpler to reason about and fail more predictably.

How long should the timeout be? Long enough for your slowest real document and short enough to fail fast on a deadlock. 180 seconds suits most reports; escalate through retries rather than setting one very long budget, so a genuine hang is detected quickly.

Does --convert-to pdf need the writer_pdf_Export filter? Not strictly, but naming it removes ambiguity when the input could be opened by more than one LibreOffice component, which is a real source of odd output for .doc and .rtf inputs.

Why do the PDFs look different from Word's own export? LibreOffice re-lays out the document with its own engine and substitutes any font it does not have. Install the template's fonts in the image, and check pagination-sensitive documents after any image change.

Part of Converting DOCX to PDF with Python.