Trustworthy AI
Turning 1,000-page freight rate sheets into trusted data
Pratyank’s work trial for Starboard.
Zero fabricated values across real carrier documents, including a 628-page contract (about 25,000 records) processed in about 4.5 minutes; Starboard’s team judged it about 4x more efficient than their existing system.
- Fabricated values across real carrier documents
- 0
- Contract processed in about 4.5 minutes
- 628 pages
- More efficient than the existing system, in Starboard’s judgment
- ~4x
The situation
Carriers send ocean-freight rate sheets as huge, inconsistent Excel and PDF files, some over 1,000 pages. People compare them to pick the best rate, so one wrong number costs real money.
What was hard
Every carrier uses a different layout. Meaning hides in formatting: white hidden text, struck-through withdrawn rates, coloured fills. And an LLM that reads numbers directly can invent them.
What Pratyank did
- Parsed Excel and PDF into one structural format that keeps formatting signals (hidden text, fills, strikethrough) and a source location for every cell.
- Limited the LLM to writing a mapping spec that says which column means what; deterministic code reads every number from a real cell.
- Added validation, confidence bands and flags, so low-confidence or untraceable values are withheld, never guessed.
- Built template memory so a repeat layout is reused with zero LLM calls.
- Let the LLM write new extraction rules for unfamiliar conventions, run in a sandbox behind static checks, an AI reviewer and human approval.
- Streamed huge PDFs in chunks, so the 628-page contract processes in about 4.5 minutes with bounded memory.
- Built a read-only operator console showing each value’s source, flags and withheld fields, backed by 260 tests and a hard gate against fabrication.
What changed
All four real test documents passed with zero fabricated values and full precision and recall against a hand-checked answer key.
The 628-page contract ran end to end with full provenance, and the system was also tested on documents over 1,000 pages in very different formats. Starboard judged it about 4x more efficient than what they had.
Bring us in when you need AI to pull data out of messy documents and every number must trace back to its source.