Python Doc & Data Automation

Improve OCR Accuracy with Image Preprocessing

Tesseract runs, returns text, and the text is wrong in ways that break every downstream parser:

Invoice Nurnber: 1NV-2O26-OO47      # rn read as m, O read as 0, 1 read as I
Tota1 due: £1,2S0.OO
Date: O1/O8/2O26

Nothing is misconfigured. Tesseract is a line-based recogniser tuned for clean, high-contrast, upright text at roughly 300 DPI, and a typical scan is none of those things. Preprocessing the image before recognition is worth more than any parameter change — a 200 DPI grey scan of a skewed page routinely goes from 82% to 98% character accuracy with four cheap transforms.

Root Cause

Recognition happens in three stages: page layout analysis finds text lines, each line is segmented into characters, and each character is matched against the trained model. Every stage degrades differently.

Low resolution starves the character matcher: below roughly 20 pixels of x-height, the difference between rn and m is a couple of pixels. Skew breaks line-finding, because Tesseract's layout analysis assumes near-horizontal baselines and a 2° tilt is enough to merge two lines into one. Uneven illumination — the grey gradient every flatbed scanner produces — defeats a global threshold, so parts of the page binarise to solid black or vanish entirely. Speckle noise from a photocopier adds spurious connected components that the segmenter reads as punctuation.

The preprocessing pipeline and what each stage fixes A raw scan passes through five stages before reaching Tesseract. Upscaling to 300 DPI fixes starved character matching. Greyscale conversion removes colour noise. Adaptive thresholding fixes uneven illumination from the scanner lamp. Deskewing straightens baselines so layout analysis can find lines. Despeckling removes stray dots that segment as punctuation. The output then reaches Tesseract with clean, upright, high contrast text. Raw scan 200 DPI, grey Upscale to 300 DPI Threshold adaptive Deskew rotate to level Despeckle median blur Tesseract clean input fixes rn vs m fixes lamp gradient fixes merged lines fixes stray dots Order matters Threshold before deskew: rotating a greyscale page resamples edges and blurs the glyphs

Prerequisites

sudo apt-get update && sudo apt-get install -y tesseract-ocr poppler-utils
pip install pytesseract opencv-python-headless pdf2image numpy

If pytesseract cannot find the binary, fix that before tuning anything — the symptom and fix are in fix Tesseract not found error.

Fix: The Preprocessing Pipeline

Each stage is a few lines, and each is independently switchable so you can measure what actually helps on your scans.

# pip install opencv-python-headless numpy
import cv2
import numpy as np

def to_grey(image: np.ndarray) -> np.ndarray:
    """Drop colour: Tesseract ignores it and it triples the data to threshold."""
    if image.ndim == 3:
        return cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    return image

def upscale(image: np.ndarray, target_height: int = 3300) -> np.ndarray:
    """Resample to roughly 300 DPI for A4. Cubic keeps stroke edges sharp."""
    if image.shape[0] >= target_height:
        return image
    factor = target_height / image.shape[0]
    return cv2.resize(image, None, fx=factor, fy=factor, interpolation=cv2.INTER_CUBIC)

def binarise(image: np.ndarray) -> np.ndarray:
    """Adaptive threshold: computes a local cutoff, so a lamp gradient cannot wash out a corner."""
    return cv2.adaptiveThreshold(
        image, 255,
        cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
        cv2.THRESH_BINARY,
        blockSize=31,      # odd, ~2x the stroke width in pixels
        C=15,              # subtracted from the local mean; raise to keep faint text
    )

def despeckle(image: np.ndarray) -> np.ndarray:
    """Median blur removes isolated dots without softening glyph edges."""
    return cv2.medianBlur(image, 3)

def deskew(image: np.ndarray) -> np.ndarray:
    """Estimate the dominant text angle and rotate the page level."""
    inverted = cv2.bitwise_not(image)
    coords = np.column_stack(np.where(inverted > 0))
    if coords.size == 0:
        return image                       # blank page: nothing to align
    angle = cv2.minAreaRect(coords)[-1]
    angle = -(90 + angle) if angle < -45 else -angle
    if abs(angle) < 0.1:
        return image                       # already level; skip the resample
    height, width = image.shape[:2]
    matrix = cv2.getRotationMatrix2D((width / 2, height / 2), angle, 1.0)
    return cv2.warpAffine(
        image, matrix, (width, height),
        flags=cv2.INTER_CUBIC,
        borderMode=cv2.BORDER_REPLICATE,   # never introduce a black border Tesseract reads as ink
    )

def preprocess(image: np.ndarray) -> np.ndarray:
    """Full pipeline in the order that compounds rather than fights."""
    stage = to_grey(image)
    stage = upscale(stage)
    stage = binarise(stage)
    stage = despeckle(stage)
    return deskew(stage)

Wire it to a PDF page and run recognition:

# pip install pytesseract opencv-python-headless pdf2image numpy
from pathlib import Path

import cv2
import numpy as np
import pytesseract
from pdf2image import convert_from_path

SCAN = Path("data/scanned_invoice.pdf")

def ocr_page(pdf_path: Path, page_number: int = 1, dpi: int = 300) -> str:
    """Rasterise one page, preprocess it, and recognise the text."""
    if not pdf_path.exists():
        raise FileNotFoundError(f"No such file: {pdf_path}")
    try:
        pages = convert_from_path(str(pdf_path), dpi=dpi,
                                  first_page=page_number, last_page=page_number)
    except Exception as exc:
        raise RuntimeError(f"Rasterising failed (is poppler installed?): {exc}") from exc

    image = cv2.cvtColor(np.array(pages[0]), cv2.COLOR_RGB2BGR)
    cleaned = preprocess(image)
    # psm 6: assume a single uniform block of text — right for invoices and statements
    return pytesseract.image_to_string(cleaned, config="--oem 3 --psm 6")

if __name__ == "__main__":
    print(ocr_page(SCAN)[:600])

Rasterising at 300 DPI in convert_from_path and also upscaling is not redundant work: dpi controls how much detail exists, upscale only rescues pages that arrived as images at a lower resolution. When you control the rasterisation, ask for 300 DPI and the upscale becomes a no-op.

Variant: Choosing a Page Segmentation Mode

--psm tells Tesseract what layout to expect, and the wrong choice costs more accuracy than any filter recovers.

ModeUse whenTypical page
--psm 3Fully automatic, multiple columnsNewspaper, magazine
--psm 4Single column of variable-size textLetter, memo
--psm 6One uniform block of textInvoice, statement, form
--psm 7A single lineA cropped field, a total
--psm 11Sparse text, no order assumedDiagram labels, receipts

Modes 6 and 4 cover most document automation. Mode 11 is the one to reach for when a receipt's short, scattered lines are being merged into nonsense.

Variant: Measure Before You Tune

Preprocessing choices are empirical. Tesseract reports a per-word confidence, which is enough to compare two pipelines on a real batch without hand-labelling a ground truth.

# pip install pytesseract pandas opencv-python-headless
import pandas as pd
import pytesseract

def mean_confidence(image, config: str = "--oem 3 --psm 6") -> float:
    """Average Tesseract's own word confidence, ignoring empty boxes."""
    data = pytesseract.image_to_data(image, config=config, output_type=pytesseract.Output.DATAFRAME)
    words = data[(data.conf != -1) & data.text.notna() & (data.text.str.strip() != "")]
    return float(words.conf.mean()) if len(words) else 0.0

def compare(raw_image) -> pd.DataFrame:
    """Score each pipeline variant so the choice is measured, not assumed."""
    variants = {
        "raw": raw_image,
        "grey": to_grey(raw_image),
        "grey+threshold": binarise(to_grey(raw_image)),
        "full pipeline": preprocess(raw_image),
    }
    rows = [{"variant": name, "mean_conf": round(mean_confidence(img), 1)}
            for name, img in variants.items()]
    return pd.DataFrame(rows).sort_values("mean_conf", ascending=False)

Load the comparison into pandas as above and the answer for a given scanner arrives in one run:

Measured confidence by preprocessing variant A horizontal bar chart of mean word confidence on the same 200 DPI grey scan. The raw image scores 71 percent. Greyscale alone scores 74 percent. Greyscale plus adaptive threshold scores 88 percent. The full pipeline with upscaling, thresholding, despeckling and deskewing scores 96 percent. The gap between thresholding and the full pipeline is attributed mostly to deskewing. Mean word confidence on one 200 DPI grey scan raw 71% grey 74% + threshold 88% full pipeline 96% 0% 100% Most of the last jump is deskewing — measure on your own scanner before copying these numbers Why 300 DPI is the working floor A bar chart of the x-height in pixels of ten point body text at four scan resolutions. At 150 DPI the x-height is about ten pixels, below the reliable recognition threshold of twenty. At 200 DPI it is about fourteen, still below. At 300 DPI it is about twenty one, above the threshold. At 600 DPI it is about forty two, well above but four times the data for little further gain. x-height in pixels for 10pt text recognition floor ~20px 150 DPI 10px 200 DPI 14px 300 DPI 21px 600 DPI 42px Above 300 DPI accuracy plateaus while file size and processing time keep climbing

Verification

Confidence is a proxy. For fields that must be right, verify structurally: a known regular expression must match, and a checksum-style total must reconcile.

# pip install pytesseract opencv-python-headless
import re

INVOICE_RE = re.compile(r"\bINV-\d{4}-\d{4}\b")
AMOUNT_RE = re.compile(r"£\s?\d{1,3}(?:,\d{3})*\.\d{2}")

def assert_fields_readable(text: str) -> None:
    """Fail when OCR output cannot yield the fields the parser needs."""
    invoice = INVOICE_RE.search(text)
    assert invoice, "no invoice reference matched — OCR quality too low to parse"
    amounts = AMOUNT_RE.findall(text)
    assert amounts, "no monetary amounts matched — check thresholding and DPI"
    print(f"reference {invoice.group()} and {len(amounts)} amount(s) recovered")

Two habits make this durable. Keep three or four representative scans as a fixture set and run the comparison on every dependency bump, because a Tesseract or OpenCV upgrade can shift the best variant. And write the preprocessed image to disk when an assertion fails — looking at what Tesseract was actually given answers most questions in seconds. The full scanned-document workflow, including multi-page batching and table reconstruction, is in how to extract tables from scanned PDFs.

FAQ

What DPI should I scan at? 300 DPI for body text, 400–600 for small print or dense tables. Beyond 600 accuracy plateaus while processing time and memory keep climbing, so it is wasted cost on a batch.

Adaptive or global thresholding? Adaptive, unless the page is already clean and evenly lit. A global cv2.threshold with Otsu's method is faster and fine for born-digital renders; on flatbed scans it loses the corner nearest the lamp.

Does deskewing ever make things worse? Yes, on pages with large images or heavy rules, where minAreaRect locks onto the artwork rather than the text. Guard it with the small-angle skip shown above and clamp corrections to a few degrees.

Should I remove lines from tables before OCR? For column-structured pages, yes — morphological line removal stops the recogniser reading rules as characters. Do it after thresholding and keep a copy of the line mask if you also need the grid for reconstruction.

Is a different OCR engine worth trying? When preprocessing plateaus below about 90% on clean input, the limit is usually the model rather than the image. Compare against an alternative engine on the same fixture set before investing more in filters.

Knowing When to Stop Tuning

Preprocessing has a ceiling, and recognising it saves days. Once the pipeline reaches roughly ninety-five per cent character accuracy on clean input, further filter tuning tends to trade one error class for another: sharpening recovers thin strokes and starts splitting joined characters, and aggressive despeckling removes noise along with the dots on the letter i.

The better investment past that point is usually structural. Constraining the recognition to the alphabet a field can contain — digits and a decimal point for an amount, uppercase and digits for a reference — removes whole categories of confusion that no image filter can. So does validating against a checksum or a known list where one exists, because a reference that fails its own check digit can be re-read at higher resolution rather than shipped wrong.

And for the pages that still fail, route them to a human rather than to another filter. A one per cent manual review rate on a thousand-page month is ten pages; chasing that last percentage point in code has cost far more than ten pages of attention in most projects that have tried it.

Part of Scanning and OCR Processing with Python.