Python Doc & Data Automation

Fix OCR Garbage on Rotated Scans

Most pages in the scanned batch OCR cleanly, and a handful produce output like this:

>>> pytesseract.image_to_string(Image.open("scans/page-017.png"))
'\n\nIl ! | . \'\n\n= —= = \n\nWV 3 : aa\n\n'

or strings of plausible-looking but meaningless letters. Opening the images shows the pattern immediately: the bad pages are landscape spreadsheets scanned in portrait, delivery notes fed upside down, or forms turned 90 degrees on the glass. Confidence scores for those pages sit near zero, but nothing raises an error, so the garbage flows into search indexes and extraction jobs.

Root Cause

Tesseract's recognition models are trained on upright text. Its default page segmentation mode (--psm 3, fully automatic layout without orientation detection) assumes lines run left to right and top to bottom; it does not rotate the image to find text. A page rotated 90 or 270 degrees presents vertical strokes where the model expects horizontal lines, and a page rotated 180 degrees presents mirrored glyph shapes that the model maps to whatever characters they most resemble — producing confident nonsense rather than empty output. Orientation information exists in two places the default pipeline ignores: Tesseract's own orientation and script detection (OSD, --psm 0), and the scan metadata (TIFF or EXIF orientation tags) that some scanners set instead of rotating pixels. Small skew — a few degrees from a crooked feed — is a related but separate problem: it degrades accuracy gradually rather than producing garbage, and 90-degree rotation cannot fix it.

Minimal Diagnostic

For each page, compare OCR confidence at the four right-angle rotations. A page whose best rotation is not 0 degrees is the cause; the gap between best and worst confidence shows how decisive the signal is.

# pip install pytesseract pillow
from pathlib import Path
import pytesseract
from PIL import Image, ImageOps

SCANS = Path("scans")

def mean_confidence(image: Image.Image) -> float:
    data = pytesseract.image_to_data(image, config="--psm 6", output_type=pytesseract.Output.DICT)
    confs = [float(c) for c, t in zip(data["conf"], data["text"]) if t.strip() and float(c) >= 0]
    return sum(confs) / len(confs) if confs else 0.0

def rotation_profile(path: Path) -> dict[int, float]:
    try:
        base = ImageOps.exif_transpose(Image.open(path)).convert("L")
    except OSError as exc:
        raise SystemExit(f"cannot open {path}: {exc}")
    small = base.copy()
    small.thumbnail((1400, 1400))                          # fast probe; full size is not needed
    return {angle: round(mean_confidence(small.rotate(-angle, expand=True)), 1)
            for angle in (0, 90, 180, 270)}

if __name__ == "__main__":
    for path in sorted(SCANS.glob("page-01[5-9].png")):
        profile = rotation_profile(path)
        best = max(profile, key=profile.get)
        print(f"{path.name}: {profile}  best={best}")

page-015.png: {0: 91.3, 90: 12.0, 180: 8.7, 270: 10.4}  best=0
page-016.png: {0: 88.9, 90: 9.1, 180: 11.2, 270: 7.8}   best=0
page-017.png: {0: 14.2, 90: 86.5, 180: 9.9, 270: 12.1}  best=90
page-018.png: {0: 21.3, 90: 7.7, 180: 90.1, 270: 8.4}   best=180

Pages 17 and 18 are rotated; the correct orientation scores in the high 80s, every wrong one below 25. That clear separation is typical and is what makes automatic correction safe.

OCR confidence by rotation for page 17 For page 17, a landscape spreadsheet scanned in portrait, mean word confidence was 14.2 without rotation, 86.5 when rotated 90 degrees clockwise, 9.9 at 180 degrees and 12.1 at 270 degrees. Only the correct orientation produces confident recognition, and the gap is large enough to choose it automatically. page-017.png, mean word confidence (0-100) 0 degrees (as scanned) 14.2 90 degrees clockwise 86.5 180 degrees 9.9 270 degrees 12.1

Fix: Detect Orientation with OSD, Rotate, Then OCR

Tesseract's OSD mode reports the rotation needed and its confidence in one fast call. Apply EXIF orientation first, run OSD, rotate the pixels, and only then run recognition. Changed lines carry comments.

# pip install pytesseract pillow
import re
from pathlib import Path
import pytesseract
from PIL import Image, ImageOps

SCANS = Path("scans")
OSD_MIN_CONFIDENCE = 2.0

def upright(image: Image.Image) -> tuple[Image.Image, int, str]:
    """Return the image rotated upright, the applied clockwise angle, and how it was decided."""
    try:
        osd = pytesseract.image_to_osd(image, config="--psm 0")          # changed: orientation detection
    except pytesseract.TesseractError:
        return image, 0, "osd-failed"                                     # too little text on the page
    angle = int(re.search(r"Rotate: (\d+)", osd).group(1))
    confidence = float(re.search(r"Orientation confidence: ([\d.]+)", osd).group(1))
    if angle and confidence >= OSD_MIN_CONFIDENCE:
        return image.rotate(-angle, expand=True), angle, "osd"            # changed: PIL rotates counter-clockwise
    return image, 0, "osd-low-confidence" if angle else "upright"

def ocr_page(path: Path) -> dict:
    try:
        image = ImageOps.exif_transpose(Image.open(path))                 # changed: honour scanner orientation tags
    except OSError as exc:
        raise RuntimeError(f"cannot open {path}: {exc}") from exc
    gray = image.convert("L")
    fixed, angle, how = upright(gray)                                     # changed: rotate before recognition
    text = pytesseract.image_to_string(fixed, config="--psm 3")
    return {"file": path.name, "rotated": angle, "decided_by": how, "text": text}

if __name__ == "__main__":
    for result in map(ocr_page, sorted(SCANS.glob("*.png"))):
        print(f"{result['file']}: rotated {result['rotated']} ({result['decided_by']}), "
              f"{len(result['text'].split())} words")

Tesseract's Rotate value is the clockwise rotation that makes the image upright. PIL's Image.rotate turns counter-clockwise for positive angles, which is why the code passes -angle; getting that sign wrong turns a 90-degree fix into a 180-degree error that looks exactly like the original problem. expand=True enlarges the canvas so a rotated landscape page is not cropped to portrait dimensions.

Orientation-safe OCR per page Each page image first has its EXIF orientation tag applied. Tesseract OSD reports the clockwise rotation needed and a confidence. If confidence is at least 2 the image is rotated by the negative angle in PIL with canvas expansion. If OSD fails or is not confident, a probe compares recognition confidence at four rotations on a downscaled copy. Recognition then runs on the upright image and the chosen angle is recorded. EXIF transpose scanner tags OSD psm 0 angle + confidence Rotate pixels rotate(-angle, expand) OCR psm 3 upright image Record angle for audit Low confidence 4-way confidence probe

Variant Fix 1: When OSD Cannot Decide

OSD needs enough text — roughly a few lines of body text — and fails on sparse pages such as forms with a few filled fields, drawings with labels, or pages dominated by a table grid. Fall back to the confidence probe from the diagnostic, which works with less text because it runs full recognition:

# pip install pytesseract pillow
from PIL import Image

def upright_with_fallback(image: Image.Image, min_gap: float = 25.0) -> tuple[Image.Image, int, str]:
    fixed, angle, how = upright(image)
    if how in ("osd", "upright"):
        return fixed, angle, how
    probe = image.copy()
    probe.thumbnail((1400, 1400))
    scores = {a: mean_confidence(probe.rotate(-a, expand=True)) for a in (0, 90, 180, 270)}
    best = max(scores, key=scores.get)
    runner_up = max(v for a, v in scores.items() if a != best)
    if scores[best] - runner_up >= min_gap:                     # only act on a clear winner
        return image.rotate(-best, expand=True), best, "confidence-probe"
    return image, 0, "undecided"

The probe costs four recognition passes on a reduced image — noticeably slower than OSD — so use it only as the fallback. Pages that stay undecided are usually blank, photographic or genuinely unreadable; list them for review instead of guessing.

Variant Fix 2: Small Skew After Rotation

A page can be upright to within 90 degrees and still tilted by two or three degrees, which silently lowers accuracy on long lines and breaks table column detection. Estimate the skew from text line angles and correct it after the right-angle rotation:

# pip install opencv-python-headless numpy pillow
import cv2
import numpy as np
from PIL import Image

def deskew(image: Image.Image, max_angle: float = 10.0) -> tuple[Image.Image, float]:
    gray = np.array(image.convert("L"))
    binary = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
    coords = np.column_stack(np.where(binary > 0)).astype(np.float32)
    if len(coords) < 500:
        return image, 0.0                                       # nearly blank page
    angle = cv2.minAreaRect(coords)[-1]
    angle = angle - 90 if angle > 45 else angle                 # normalise OpenCV's angle convention
    if abs(angle) < 0.3 or abs(angle) > max_angle:
        return image, 0.0
    return image.rotate(angle, expand=True, fillcolor=255, resample=Image.Resampling.BICUBIC), angle

Deskewing is covered alongside binarisation and denoising in improve OCR accuracy with image preprocessing. Apply it only after the coarse rotation is fixed; skew estimation on a sideways page measures the wrong axis.

Orientation methods compared EXIF or TIFF orientation tags fix rotations the scanner recorded, cost nothing and fail when the scanner does not set them. Tesseract OSD fixes 90 degree steps, is fast and fails on pages with little text. The four-rotation confidence probe fixes 90 degree steps, is slow and fails on blank or photo pages. Deskewing fixes small angles, is fast, and must run after coarse rotation is correct. Method Fixes Cost Fails on EXIF / TIFF tag 90 degree steps free untagged scans Tesseract OSD 90 degree steps fast sparse text Confidence probe 90 degree steps 4 OCR passes blank pages Deskew small angles fast before coarse fix

Fixing Orientation in the PDF, Not Just for OCR

When the scans arrive as PDFs, rotating images in memory produces correct OCR text but leaves the PDF itself sideways for readers. Write the detected angle back as the page's /Rotate so the document and the OCR agree:

# pip install pymupdf pytesseract pillow
import io
from pathlib import Path
import pymupdf
from PIL import Image

def fix_pdf_orientation(src: Path, dest: Path) -> list[tuple[int, int]]:
    changed = []
    with pymupdf.open(src) as doc:
        for page in doc:
            pix = page.get_pixmap(dpi=150)                   # render as currently displayed
            image = Image.open(io.BytesIO(pix.tobytes("png"))).convert("L")
            _fixed, angle, how = upright_with_fallback(image)
            if angle:
                page.set_rotation((page.rotation + angle) % 360)
                changed.append((page.number + 1, angle))
        dest.parent.mkdir(parents=True, exist_ok=True)
        doc.save(dest, garbage=3, deflate=True)
    return changed

Once pages carry the right /Rotate, tools that render pages for OCR — including OCRmyPDF — receive upright images automatically. Page-level rotation logic, including reordering, is covered in rotate and reorder PDF pages with Python.

Verification

Measure the improvement on a labelled sample: pages you know the correct orientation for, and OCR quality before and after.

# pip install pytesseract pillow
import re
from pathlib import Path

COMMON = {"the", "and", "of", "to", "total", "date", "invoice", "delivery", "qty", "page"}

def word_quality(text: str) -> float:
    words = re.findall(r"[a-z]{2,}", text.lower())
    return sum(w in COMMON for w in words) / (len(words) or 1)

def verify(sample_dir: Path, truth: dict[str, int]) -> None:
    wrong, gains = [], []
    for name, expected_angle in truth.items():
        path = sample_dir / name
        result = ocr_page(path)
        if result["rotated"] != expected_angle:
            wrong.append((name, result["rotated"], expected_angle))
        naive = pytesseract.image_to_string(Image.open(path).convert("L"))
        gains.append(word_quality(result["text"]) - word_quality(naive))
    assert not wrong, f"wrong rotation decisions: {wrong}"
    print(f"{len(truth)} pages: all rotations correct; mean quality gain {sum(gains) / len(gains):+.3f}")

if __name__ == "__main__":
    verify(Path("scans/labelled"), {"page-015.png": 0, "page-017.png": 90, "page-018.png": 180})

Label twenty or thirty pages covering every orientation your scanners produce, including a few sparse forms. The assertion guards the rotation logic; the quality gain shows whether the change actually helps the downstream text, which is what matters.

FAQ

Does OCRmyPDF handle rotation? Yes, with --rotate-pages, which uses the same OSD and corrects page rotation in the output PDF. See make scanned PDFs searchable with OCRmyPDF.

Why does OSD fail with "Too few characters"? The page has too little text for a reliable estimate. Use the confidence probe fallback or accept the page as-is.

Can the same page contain text in two orientations? Yes — vertical labels on a landscape chart. OSD reports the dominant orientation; OCR of the minority text needs cropping and a separate pass.

Is --psm 1 (automatic with OSD) enough? It detects orientation during layout analysis, but results vary by version. Explicit OSD plus rotation is easier to log, test and override.

Part of Scanning and OCR Processing with Python.