Some bug reports arrive as stack traces. The best ones arrive as feelings.
"I have a feeling not enough knowledge is being pulled in when distilling — maybe truncated content — the knowledge seems half-baked. Could be my assumption, but validate it."
That's the whole report. No error, no repro steps. A person who lives inside his own workspace, noticing that the system's grasp of his world felt thinner than it should.
The query that ended the argument
Margin's Mind runs on a distill-then-embed pipeline. Instead of indexing raw text, a model first compresses each item into a compact artifact (what it's about, its themes, who's involved) and that becomes the item's meaning in the system. It's the technique that fixed our noisy suggestions, and we were proud of it.
The distiller read the first 4,000 characters of each item.
One query against production settled it. Eight notes ran past that window. The largest was 238,467 characters, an imported document of which the distiller had read four thousand. The system's entire understanding of that note was built from 1.7% of it.
A second, quieter truncation made it worse. Embedding providers silently cut off long inputs, so even the raw fallback vectors represented only the openings of documents. Two truncations, both invisible. One saving grace, though. We never truncate stored text, so exact-keyword search still saw whole documents, which is why the failure felt like thinness rather than breakage. Nothing was broken. Everything was just shallow.
Reading the whole book without paying for the whole book
The naive fix, feed everything to the model, turns one pasted book into sixty model calls. What we shipped instead is stride sampling. Long content is cut into up to eight windows spaced evenly from the first page to the last; each window is distilled on its own; the results merge, summaries joined, themes ranked by how many windows they recur in, people deduplicated. Head, middle, and tail all get read. The cost is capped no matter how large the document. A theme that only lives in the final third of something long finally exists in the system.
The silent embedding truncation became a loud, named constant, and since long items now embed their merged artifact instead of raw text, the vector finally carries whole-document meaning too. The eight affected documents were re-distilled that afternoon; the indexer's freshness tracking picked them up like any other edit.
About that feeling
Here's the part that isn't really about truncation. The founder hedged, "could be my assumption." It wasn't. People who genuinely inhabit a knowledge system develop accurate instincts about whether it knows things, well before they can prove it, the way you can tell a friend is off across a room before you could say what's wrong. The instinct arrives with no error attached, which is what makes it so easy to wave away, and what makes waving it away the characteristic failure of building this kind of product.
He was right, and it took one query to show it. And honestly, the thing I felt when the 1.7% came back wasn't panic. It was relief. Better to learn your system has been reading the first page of the book than to keep trusting a summary of a book nobody finished.