pdf-inspector

Library

firecrawl/pdf-inspector

Fast Rust library for PDF classification, text extraction, and Markdown conversion.

Overview

A high-performance Rust library that detects whether PDFs are text-based or scanned, extracts positioned text, and converts it to clean Markdown without OCR. Includes Python, Node.js, and WebAssembly bindings, plus CLI tools. Designed for smart routing to skip OCR for the majority of native-text PDFs.

README Preview

# pdf-inspector\n\n[](https://crates.io/crates/pdf-inspector)\n[](https://www.npmjs.com/package/@firecrawl/pdf-inspector)\n[](https://pypi.org/project/pdf-inspector/)\n[](LICENSE)\n\nFast Rust library for PDF classification and text extraction. Detects whether a PDF is text-based or scanned, extracts text with position awareness, and converts to clean Markdown — all without OCR. Includes bindings for [Python](docs/python.md), [Node.js](napi/README.md), and [browser WebAssembly](wasm/README.md).\n\nBuilt by [Firecrawl](https://firecrawl.dev) to handle text-based PDFs locally in under 200ms, skipping expensive OCR services for the ~54% of PDFs that don't need them.\n\n## Features\n\n- **Smart classification** — Detect TextBased, Scanned, ImageBased, or Mixed PDFs in ~10-50ms by sampling content streams. Returns a confidence score (0.0-1.0) and per-page OCR routing.\n- **Text extraction** — Position-aware extraction with font info, X/Y coordinates, and automatic multi-column reading order.\n- **Markdown conversion** — Headings (H1-H4 via font size ratios), bullet/numbered/letter lists, code blocks (monospace font detection), tables (rectangle-based and heuristic), bold/italic formatting, URL linking, and page breaks.\n- **Table detection** — Dual-mode: rectangle-based detection from PDF drawing ops, plus heuristic detection from text alignment. Handles financial tables, footnotes, and continuation tables across pages.\n- **CID font support** — ToUnicode CMap decoding for Type0/Identity-H fonts, UTF-16BE, UTF-8, and Latin-1 encodings.\n- **Multi-column layout** — Automatic detection of newspaper-style columns, sequential reading order, and RTL text support.\n- **Encoding issue detection** — Automatically flags broken font encodings so callers can fall back to OCR.\n- **Single document load** — The document is parsed once and shared between detection and extraction, avoiding redundant I/O.\n- **Browser WebAssembly** — Run the same Rust parser locally in browsers and Web 

FAQ (5)

Troubleshooting
How do I fix “GLIBC_2.35 not found” when loading @firecrawl/pdf-inspector on AWS Lambda or Amazon Linux 2023?

The x64 GNU binary in versions 1.13.0–1.17.0 was built without --use-napi-cross, so it requires GLIBC_2.35 and fails on glibc 2.34 hosts such as Amazon Linux 2023, RHEL 9, and Rocky Linux 9. The misleading Cannot find module error can hide the real GLIBC_2.35 not found failure in loadErrors.

Permanent release fix: build the x86_64-unknown-linux-gnu target with napi-cross in .github/workflows/publish.yml:

- runner: blacksmith-8vcpu-ubuntu-2404
  target: x86_64-unknown-linux-gnu
  build-flags: --use-napi-cross

Then rebuild:

bunx napi build --platform --release \
  --target x86_64-unknown-linux-gnu --use-napi-cross

Verify the resulting binary:

objdump -T pdf-inspector.linux-x64-gnu.node \
  | grep -o 'GLIBC_[0-9.]*' | sort -Vu | tail -1
# expected: GLIBC_2.17

Temporary workarounds on glibc 2.34 hosts: pin the x64 platform package to version 1.12.0, the last release with a GLIBC_2.34 requirement, or use a locally rebuilt binary with the command above. An npm override can be used if the package is pulled in transitively:

{
  "overrides": {
    "@firecrawl/pdf-inspector-linux-x64-gnu": "1.12.0"
  }
}

Version 1.12.0 still requires GLIBC_2.34, so it does not help older systems such as Debian 11; those require the rebuilt GLIBC_2.17 binary or a fixed release. Do not switch to a musl build solely to work around this.

GitHub Issue #505
Troubleshooting
How do I fix Japanese CFF PDF extraction showing repeated unrelated kanji, such as "針針針", in pdf-inspector?

This happens because try_remap_subset_cmap applies a TrueType subset-GID repair to CIDFontType0 (CFF) fonts. In CFF fonts, CIDs are resolved through the CFF charset and are not renumbered glyph IDs, so the original /ToUnicode CMap must not be remapped.

Fix it by upgrading to a release containing the PR fix(tounicode): skip subset GID remap for CIDFontType0 (CFF) descendants, or build pdf2md/pdf-inspector from the patched source.

If patching src/tounicode.rs yourself, add a guard in try_remap_subset_cmap before the W-array/sequential-remap checks:

// Keep the existing ToUnicode CMap unchanged for CFF fonts.
if descendant_subtype == "CIDFontType0" {
    return None;
}

Leave the existing remap_to_sequential path enabled only for CIDFontType2 (TrueType) descendants. Then rebuild:

cargo build --release
cargo run --bin pdf2md -- input.pdf > out.md

Affected versions include @firecrawl/pdf-inspector 1.11.2 and @firecrawl/pdf-inspector-wasm 0.1.3. As a temporary workaround, extract the PDF with pdfjs-dist 6.0.227 using getTextContent(). Note that this corruption may produce valid but wrong kanji, so counting U+FFFD replacement characters will not detect it.

GitHub Issue #208
Troubleshooting
How do I fix the PDF parsing error "invalid file trailer" caused by classic xref entries using bare LF (19-byte stride)?

Upgrade pdf-inspector to a release or build containing PR #508. The fix detects classic cross-reference tables whose entries end with a single LF byte—19 bytes instead of the required 20-byte SP LF ending—and rebuilds the table in place with spec-compliant SP LF entries while preserving the rest of the file. This is separate from startxref-pointer recovery. As a temporary workaround, rewrite the PDF with a tolerant reader such as pypdf so the xref table is regenerated:

from pypdf import PdfReader, PdfWriter

reader = PdfReader("input.pdf")
writer = PdfWriter(clone_from=reader)

with open("fixed.pdf", "wb") as f:
    writer.write(f)

GitHub Issue #473
Troubleshooting
How to fix pdf-inspector dropping or mistranslating Spanish accented letters, such as Cámara → CÆmara and código → cdigo, from a PDF?

This happens in pdf-inspector 1.20.0 when a standard-14 Type1 font uses an encoding dictionary such as << /BaseEncoding /WinAnsiEncoding >> without a /Differences array. pdf-inspector ignores /BaseEncoding and decodes the text as StandardEncoding, so bytes in the 0xC0–0xFF range become wrong glyphs or are dropped. has_encoding_issues remaining false is also a bug.

Permanent fix: upgrade pdf-inspector to a release containing the font-encoding fix "decode simple fonts through their base and built-in encodings and glyph names", then re-extract the file.

Temporary workarounds:

1. Extract with pypdf 6.10 or newer, which handles this encoding correctly:

pip install -U "pypdf>=6.10"

from pypdf import PdfReader

reader = PdfReader("repro.pdf")
text = "\n".join(page.extract_text() for page in reader.pages)
print(text)

2. Or normalize the PDF by adding an empty /Differences [] array to encoding dictionaries that already contain /BaseEncoding:

import pikepdf

with pikepdf.open("repro.pdf") as pdf:
    for page in pdf.pages:
        resources = page.get("/Resources")
        if not isinstance(resources, pikepdf.Dictionary):
            continue
        fonts = resources.get("/Font")
        if not isinstance(fonts, pikepdf.Dictionary):
            continue
        for font in fonts.values():
            encoding = font.get("/Encoding")
            if (
                isinstance(encoding, pikepdf.Dictionary)
                and "/Differences" not in encoding
            ):
                encoding.Differences = pikepdf.Array()
    pdf.save("repro-fixed.pdf")

Then run pdf-inspector against repro-fixed.pdf. The empty /Differences array is valid and forces the affected version to consult /BaseEncoding /WinAnsiEncoding.

GitHub Issue #544
Troubleshooting
Why does extract_pages_markdown() report needs_ocr=False for a scanned PDF page that has a small native-text header?

This happens when a page is mostly a full-page raster image but contains a small amount of valid native text, such as a header, stamp, or masthead. Older extract_pages_markdown() versions calculated needs_ocr only from text-quality signals, not image coverage, so the decodable header caused the page to be incorrectly marked as not needing OCR. classify_pdf() correctly detects the page as image_based.

Fix: upgrade pdf-inspector to a release containing the fixes for issue #227, where OCR routing uses the shared classification logic and masthead-over-scan pages are sent to OCR:

pip install --upgrade pdf-inspector

Workaround on affected versions: treat classify_pdf_bytes().pages_needing_ocr as authoritative instead of the per-page needs_ocr field:

import pdf_inspector

cls = pdf_inspector.classify_pdf_bytes(data)
ext = pdf_inspector.extract_pages_markdown_bytes(data)

pages_needing_ocr = set(cls.pages_needing_ocr)

for page_number, page in enumerate(ext.pages):
    if page_number in pages_needing_ocr:
        # Run OCR even if page.markdown is non-empty or page.needs_ocr is False.
        ocr_text = run_ocr_for_page(data, page_number)

Do not use non-empty extracted Markdown as proof that a page does not need OCR; it may contain only the native header while the scanned table or body is missing.

GitHub Issue #227