PDF to ChromaDB: a local-first embeddings pipeline in Python

Direct answer: To get PDFs into Chroma, extract clean Markdown with pdfmux (pip install pdfmux), chunk it by heading boundary, embed each chunk, and add it to a Chroma collection with the extraction confidence stored as metadata. Chroma’s advantage over pgvector, Pinecone, and Qdrant is that nothing has to run anywhere — PersistentClient writes straight to a local directory, so you can go from a folder of PDFs to a queryable index without starting a server or signing up for anything. The extraction and chunking steps are identical to every other pdfmux ingestion pipeline; what’s specific to Chroma is the collection API and its metadata filter syntax.

Why Chroma over a managed vector database

Every other pipeline in this series assumes you’re willing to run or pay for infrastructure: the pgvector post assumes a Postgres instance, Pinecone assumes a serverless billing account, Qdrant assumes a server process. Chroma’s embedded mode removes that requirement entirely. PersistentClient(path="./chroma_data") is a local SQLite-backed store that lives inside your process — there’s no port to open, no connection string, no separate service to keep alive in another terminal tab. For a prototype, a notebook, a CLI tool, or a single-machine batch job, that’s the whole appeal.

The tradeoff is the one you’d expect: embedded mode doesn’t give you the operational guarantees a managed or clustered store does — no built-in replication, no horizontal sharding, and concurrent writes from multiple processes aren’t the design target. Chroma does offer a client-server mode (HttpClient, backed by a long-running Chroma server) and a hosted Chroma Cloud for when you outgrow embedded, and migrating between them changes only the client constructor, not your collection schema or query code. Below is roughly where it fits:

If you start in Chroma and later need Qdrant or Pinecone’s operational guarantees, the extraction and chunking code below doesn’t change — only the four lines that talk to the vector store do.

Setup

pip install pdfmux chromadb openai

import chromadb

client = chromadb.PersistentClient(path="./chroma_data")

collection = client.get_or_create_collection(
    name="document_chunks",
    metadata={"hnsw:space": "cosine"},
)

PersistentClient is the one to reach for by default — it persists to disk across process restarts. chromadb.Client() with no path gives you an ephemeral in-memory store that’s useful for tests but disappears when the process exits. get_or_create_collection makes the setup idempotent: rerunning the script doesn’t fail on a collection that already exists.

Step 1: extract with confidence

from pdfmux import process

result = process("annual-report.pdf", quality="standard")

print(f"Confidence: {result.confidence:.0%}")
print(f"Extractor used: {result.extractor_used}")

This is the same self-healing pipeline behind every other pdfmux ingestion path: fast-extract with PyMuPDF, audit each page, re-extract anything that fails the audit with OCR or a table-aware backend, and return one confidence score for the document. On the opendataloader-bench of 200 real-world PDFs, this pipeline scores 0.903 overall — full numbers in the benchmark writeup.

Step 2: chunk by heading

Reuse the same heading-based chunker as the pgvector, Pinecone, and Qdrant pipelines — the chunking decision doesn’t depend on which vector store receives the output:

import re

def chunk_by_heading(markdown: str, source_file: str) -> list[dict]:
    sections = re.split(r"(?m)^(#{1,3}\s+.+)$", markdown)
    chunks = []
    current_heading = "Introduction"
    for part in sections:
        if re.match(r"^#{1,3}\s+", part):
            current_heading = part.lstrip("#").strip()
            continue
        text = part.strip()
        if len(text) < 40:
            continue
        chunks.append({
            "heading": current_heading,
            "text": text,
            "source_file": source_file,
        })
    return chunks

Splitting on Markdown headings keeps each chunk inside one logical section instead of cutting mid-paragraph at a fixed character count — a direct benefit of pdfmux emitting structured Markdown instead of a flat text blob.

Step 3: embed and add, gated on confidence

A document extracted at low confidence still produces chunks — they just contain garbled or partial text where the extraction failed. If a low-confidence chunk gets embedded and added alongside clean ones, Chroma’s similarity search has no way to tell them apart at query time; it ranks on vector distance, not on whether the source text was trustworthy. Gate the insert, not the search:

from openai import OpenAI

openai_client = OpenAI()
MIN_CONFIDENCE = 0.6

def embed(text: str) -> list[float]:
    response = openai_client.embeddings.create(
        model="text-embedding-3-small",
        input=text,
    )
    return response.data[0].embedding

def ingest(pdf_path: str, collection) -> None:
    result = process(pdf_path, quality="standard")

    if result.confidence < MIN_CONFIDENCE:
        print(f"SKIPPED (confidence {result.confidence:.0%}): {pdf_path}")
        return

    chunks = chunk_by_heading(result.text, pdf_path)
    if not chunks:
        print(f"No chunks extracted from {pdf_path}")
        return

    collection.add(
        ids=[f"{pdf_path}::{i}" for i in range(len(chunks))],
        embeddings=[embed(c["text"]) for c in chunks],
        documents=[c["text"] for c in chunks],
        metadatas=[
            {
                "heading": c["heading"],
                "source_file": c["source_file"],
                "confidence": result.confidence,
                "extractor_used": result.extractor_used,
            }
            for c in chunks
        ],
    )
    print(f"Ingested {len(chunks)} chunks from {pdf_path} (confidence {result.confidence:.0%})")

confidence and extractor_used ride along as ordinary metadata fields on every chunk — not a side table you have to join against later. Chroma stores metadata as part of the record itself, so it’s available at query time with zero extra lookups.

Step 4: query with confidence filtering

def search(query: str, collection, min_confidence: float = 0.7, n_results: int = 5):
    results = collection.query(
        query_embeddings=[embed(query)],
        n_results=n_results,
        where={"confidence": {"$gte": min_confidence}},
    )
    for doc, meta, distance in zip(
        results["documents"][0], results["metadatas"][0], results["distances"][0]
    ):
        similarity = 1 - distance
        print(f"[{similarity:.2f}] {meta['source_file']} — {meta['heading']} "
              f"(extraction confidence {meta['confidence']:.0%})")


search("what was Q3 revenue?", collection, min_confidence=0.7)

The where clause runs the metadata filter and the vector search together in one call — Chroma doesn’t require a separate post-filter pass. $gte is one of Chroma’s comparison operators ($eq, $ne, $gt, $gte, $lt, $lte, $in, $nin); combine multiple conditions with $and / $or if you need to filter on both confidence and, say, a specific source_file prefix in the same query.

results = collection.query(
    query_embeddings=[embed(query)],
    n_results=5,
    where={
        "$and": [
            {"confidence": {"$gte": 0.7}},
            {"source_file": {"$eq": "annual-report.pdf"}},
        ]
    },
)

When embedded mode stops being enough

Chroma’s PersistentClient is a single-process store — it’s not built for several processes writing to the same path concurrently, and it doesn’t shard across machines. The signal that you’ve outgrown it is usually one of: multiple services need to query the same index at once, the index has grown past what fits comfortably on one machine’s disk and memory, or you need the index available to a team rather than one local script. At that point, switching to Chroma’s HttpClient against a running Chroma server is the smallest step up; moving to Qdrant or Pinecone is the bigger one, with the extraction and chunking code above staying exactly as written either way.