Extracting data from Bill of Lading PDFs in Python

Direct answer: Bill of Lading PDFs are extraction-hostile because every carrier and every forwarder uses its own layout — there is no fixed-position template to anchor on. Use pdfmux.extract_json() to get per-page Markdown plus a confidence score, locate the header fields (B/L number, shipper, consignee, vessel/voyage) from the first page’s text, and parse the cargo/container table from the Markdown table pdfmux emits for that page. Gate anything headed for a TMS, customs filing, or freight-payment system on the confidence score — a BoL with a truncated container table still looks like a valid document when you only check that extraction “succeeded.”

Why Bill of Lading extraction is harder than invoice extraction

An invoice has a fairly narrow set of common templates — most accounting software exports to a handful of recognizable layouts. A Bill of Lading doesn’t have that convergence. Every ocean carrier (Maersk, MSC, CMA CGM, Hapag-Lloyd, COSCO…) issues its own template for a Master B/L, and every freight forwarder issuing a House B/L on top of that adds another layer of layout variation. On top of carrier variation, you’re dealing with:

• Multiple B/L types — straight (non-negotiable, named consignee), order/negotiable (“to order of” a bank, common in letter-of-credit trade finance), and seaway bills, each with slightly different required fields and legal language in the terms-and-conditions block.
• Master vs. House B/L — an MBL covers the carrier’s contract with the forwarder; an HBL covers the forwarder’s contract with the actual shipper. A single container can have one MBL and several HBLs consolidated inside it, each needing separate extraction and separate downstream routing.
• Scanned and faxed originals — “telex release” workflows and older trade-finance processes still move a lot of BoLs as scans, sometimes multiple generations removed from the original (scan of a fax of a print).
• Dense, inconsistent cargo tables — container number, seal number, number of packages, gross weight, and marks-and-numbers often sit in a table with merged cells, multi-line cargo descriptions, and no consistent column order between carriers.

None of this is specific to one region. If your corpus is bilingual Arabic/English GCC trade with ZATCA or UAE e-invoicing compliance on top, that combination of problems is distinct enough to have its own post — read that one instead if that’s your situation. This post covers the general case: English-language (or any single-script) BoLs from any carrier or lane.

Basic extraction

pip install pdfmux

import pdfmux

data = pdfmux.extract_json("bl-msku7654321.pdf", quality="standard")

print(f"{data['page_count']} pages, confidence {data['confidence']:.0%}")
print(f"Pages that needed OCR: {data['ocr_pages']}")

for page in data["pages"]:
    print(f"page {page['page']}: ocr={page['ocr']}")

extract_json returns the per-page Markdown plus document-level metadata — page count, overall confidence, and which pages needed OCR. A digital, carrier-generated BoL should extract at high confidence on the free, CPU-only path. A faxed or re-scanned original routes through OCR automatically and will report a lower confidence — that’s the signal to use for routing, not a guess.

Locating header fields

There’s no schema preset for Bill of Lading field extraction built into pdfmux (the built-in presets cover invoices, receipts, contracts, résumés, and papers — not shipping documents), so header fields come from the page-one Markdown. BoL headers are reasonably consistent about labeling their fields even when layout varies, which makes a label-anchored regex more robust than a fixed-position one:

import re

HEADER_PATTERNS = {
    "bl_number": r"(?:B/?L\s*No\.?|Bill\s+of\s+Lading\s+No\.?)[:\s]+([A-Z0-9\-/]+)",
    "vessel": r"Vessel[:\s]+([A-Za-z0-9\s\.\-]+?)(?:\n|Voyage)",
    "voyage": r"Voy(?:age)?\.?\s*(?:No\.?)?[:\s]+([A-Z0-9]+)",
    "port_of_loading": r"Port\s+of\s+Loading[:\s]+([A-Za-z\s,]+?)(?:\n|Port)",
    "port_of_discharge": r"Port\s+of\s+Discharge[:\s]+([A-Za-z\s,]+?)(?:\n)",
}

def extract_header(page_text: str) -> dict:
    fields = {}
    for name, pattern in HEADER_PATTERNS.items():
        m = re.search(pattern, page_text, re.IGNORECASE)
        fields[name] = m.group(1).strip() if m else None
    return fields

first_page_text = data["pages"][0]["text"]
header = extract_header(first_page_text)

This is deliberately a starting point, not a finished parser. Every carrier phrases these labels slightly differently (“Bill of Lading No.” vs “B/L No.” vs just “No.”), and you’ll extend the pattern dict as you encounter your actual carrier mix — which is faster than it sounds, since most forwarders only deal with a handful of carriers repeatedly. Shipper, consignee, and notify-party blocks are harder to anchor on a label reliably across carriers; for those, treat the top-left/top-right boxes on the page as position-based zones using pdfmux.extract_text()’s page-ordered Markdown, or fall back to a human review step for anything the regex pass doesn’t resolve with confidence.

Parsing the container and cargo table

The cargo table is the highest-risk part of the document — it’s also the part most likely to be truncated silently by an extractor that doesn’t audit its own output (see the silent_drops check below). Parse it from the Markdown table pdfmux emits for that page, the same pattern used for invoice line items:

def parse_cargo_table(markdown_text: str) -> list[dict]:
    lines = markdown_text.splitlines()
    in_table, headers, rows = False, [], []

    for line in lines:
        if re.match(r"^\|.*\|$", line.strip()):
            cells = [c.strip() for c in line.strip().strip("|").split("|")]
            if not in_table:
                in_table = True
                headers = [h.lower().replace(" ", "_") for h in cells]
            elif all(c.replace("-", "") == "" for c in cells):
                continue  # markdown separator row
            else:
                rows.append(dict(zip(headers, cells)))
        elif in_table:
            break

    return rows

cargo_rows = parse_cargo_table(data["pages"][1]["text"])  # cargo table is often page 2

Don’t trust row count alone as a correctness signal — a table clipped from 12 containers to 3 still parses cleanly and still looks successful. Compare the row count you extracted against the “Total containers” or “Total packages” summary line most BoLs print at the bottom of the cargo block, and flag a mismatch for human review before it reaches a TMS or customs filing.

Auditing the extraction before you trust it

Before feeding an extracted BoL into anything automated, audit it against the source PDF:

from pdfmux import verify_extraction

manifest = verify_extraction("bl-msku7654321.pdf", "extracted.json", engine="pdfmux")

print(manifest.verdict)       # "PASS" | "REVIEW" | "FAIL"
print(manifest.confidence)
print(manifest.silent_drops)  # pages that vanished without a warning
print(manifest.review_pages)

silent_drops is the check that catches a page that extracted to plausible-looking text in the wrong place — a cargo table that silently lost its last few rows reads as a successful extraction unless something is specifically checking page-level coverage against the source.

Batch processing a folder of BoLs

from pathlib import Path
import pdfmux

paths = list(Path("inbox/").glob("*.pdf"))

for result in pdfmux.batch_extract(paths, output_format="json", workers=4):
    path, outcome = result
    if isinstance(outcome, Exception):
        print(f"FAILED {path.name}: {outcome}")
        continue
    # route by confidence, as in the gating pattern above

batch_extract yields results as they complete rather than waiting for the whole batch, and handles non-ASCII filenames correctly — relevant if your inbox mixes Latin, Arabic, or CJK carrier-assigned filenames.

Gating by confidence before downstream submission

CONFIDENCE_FLOOR = 0.85  # tune against your own corpus, not this number

if data["confidence"] >= CONFIDENCE_FLOOR and not data["warnings"]:
    route_to_tms(header, cargo_rows)
else:
    queue_for_human(header, cargo_rows, data["confidence"], data["warnings"])

There’s no universal correct value for the confidence floor — it depends on your carrier mix and how costly a downstream error is (a misrouted TMS entry is cheaper to fix than a rejected customs filing). Set it empirically: run a batch you’ve already checked by hand, and find the threshold that separates the documents you’d have accepted from the ones you’d have caught.

What’s measured and what isn’t

Being direct about this, because an invented number is worse than no number: there is no published Bill-of-Lading-specific accuracy benchmark for pdfmux, or for any PDF extractor we’re aware of. What’s measured is pdfmux’s score on the public opendataloader-bench suite — 200 real-world PDFs spanning financial filings, academic papers, legal contracts, and government documents: 0.903 overall, #2 of 8 engines. That corpus doesn’t include shipping documents, so it says nothing directly about BoL extraction accuracy.

The measurement that actually tells you something: take 50 representative BoLs from your own carrier mix (digital and scanned, MBL and HBL), extract them at quality="standard", hand-check the header and cargo-table fields you actually route on, and find the confidence threshold that separates the ones you’d have trusted from the ones you’d have caught. That’s the number that applies to your pipeline — a vendor-quoted accuracy percentage for “Bill of Lading extraction” with no stated corpus isn’t one you should build a customs or freight-payment workflow on.

For scanned and low-quality originals specifically, see OCR PDF extraction in Python. For bilingual Arabic/English shipping documents under ZATCA or UAE e-invoicing compliance, see Arabic PDF extraction for GCC logistics. For the invoice side of a shipment’s paperwork, see extracting invoice data from PDFs in Python.