pdfParse

At a glance

• Outcome: Extracts facts from large PDF collections while preserving each source page and region. Enables downstream retrieval and chat to cite their evidence.
• Status: Tests working ingest, verification, revision supersession and read-only queries. Job-run verbs remain placeholders; corpus service, web UI and MCP belong to Qurite and remain out of scope.
• Role: Solo. This is the PDF extraction component of Qurite.
• Stack & libraries: Uses Rust for an engine library, thin CLI and contract crate versioned independently from the engine. Produces a portable vector dataset containing values, locations, a manifest and the action log.
• Source: MIT-licensed and release-prepped, but not published yet.
• Limitations: Demonstrates results against one golden corpus with a partly unverified reference answer. The corpus cannot be redistributed, so dependent tests skip in public checkouts; broad-document generality remains unproven.

Document structure

Preserves provenance that chunk embedding and retrieval usually remove from contracts, specifications and structural drawings. Classifies each document before extraction as hierarchical outline nodes or spatial page regions. Treats the proving drawing set as spatial despite its text layer because reading order cannot identify beams.

Golden corpus

Finds, counts and locates every W18X40 beam callout. Reads each callout as W18X40 (44) <1 3/4">: 44 is the repeated stud count and the angle brackets contain camber. Defines one callout instance as one member-label occurrence with its own page and box; a correct highlight draws one box per callout.

Records 294 text-layer callouts with boxes across ten of eleven pages. Does not claim a hand-verified physical-beam count. Treats all 294 as provisionally current because the set merges 22 change orders without revision-cloud analysis. Records a handful of callouts with no text-layer camber and normalizes and flags a handful of stud tokens that disagree with the design constant. Keeps these open items in the fixture.

Extraction identity

Hashes source, page, whole-number-quantized region, extraction kind, config version and model versions. Quantization keeps identities reproducible across languages with different floating-point bytes.

Lets the model select a neighborhood but uses deterministic code to snap geometry to content. Updates unchanged reruns in place and prevents model jitter from creating facts. Falls back to a grid cell on scanned pages, marks records AI-suggested and exempts them from that guarantee.

Model controls

Constrains model output with a grammar, separates data from instructions, masks personal information before embedding and records replayable model interactions.

Uses an append-only action log as the source of truth. Folds active, superseded and suppressed state without a mutable status column. Implements undo as a forward event so reversal history survives. Rebuilds extraction and evidence projections without another model call; the same replay resumes crashed jobs.

Marks “How many physical W18X40 beams are in the project?” unsupported in the evaluation set. Requires not-enough-evidence and forbids both callout-count and stud-count answers because no verified beam count exists. Enforces the refusal in a test.

Storage costs

Flushes every model decision before applying its effect, adding an ordered durable write to the hot path. Retains superseded rows unindexed and grows the log by default. Reclaims space only through logged compaction and rebuilds reads from history instead of trusting extraction tables directly.

Validation

Reads actual PDF bytes and matches all 294 text-layer designations page by page, including one rotated ninety degrees. Separately builds reference records without opening the PDF, replays a recorded model session, folds the projection and verifies a second run writes nothing. Covers extraction, orchestration, idempotence and replay without a GPU.

Omits the unredistributable drawing set from public releases. Skips dependent tests when absent and re-enables them from a private copy at the original path.