An AI privacy bug: sealing a board did not seal its cards

Margin has a Vault. It's the region of your Mind that even our own AI can't read, and we were proud of it.

What the Mind holds about a sealed item is encrypted at rest under a key derived from your passphrase; the search vectors stay usable while the words stay sealed; and every AI surface in the product, recall and suggestions and the graph, is gated so that locked means locked. (The note or card itself is stored like the rest of your content. The Vault is a wall against the assistant rather than end-to-end encryption, and the docs say so.) We wrote adversarial tests that try to sneak vault rows past every query. The reason the Vault exists at all is in an earlier chapter.

Then, dogfooding a workspace with real and test data mixed together, I saw something in the review queue, the surface where Margin proposes connections between your items. There were suggestions about content we had sealed. Titles, in plain sight, with the vault locked.

A perfect wall and a door frame#

Here is the shape of the leak: every gate we'd built worked exactly as designed. The sealed items themselves were gated out of every query. What we'd missed was containment.

In Margin you can seal a whole board. A board is a container. The board is one row in the database, but every card on it is its own row, and our vault membership was per-item. Sealing the board sealed the board. The cards on it were, individually, just cards. The background indexer looked at each one, found no vault marker, and did its job faithfully: embedded them, summarized them, pulled out the people they mentioned, proposed connections between them.

I counted the sealed cards the indexer had quietly woven into the open product.

One hundred and ten.

One hundred and ten cards of sealed content, threaded through the app by a system in which every individual component was working correctly. Security people have a name for this class of bug: the policy was right and the closure of the policy was wrong. We had sealed the node and left the subtree open.

What we did, in order#

1. Purge first, philosophize later. Before touching code we cleared every trace from the live system. The plaintext embeddings deleted. The suggestions and mentions flagged back into the vault, where they reappear when you unlock, hidden rather than destroyed. The handful of entities that existed only because of sealed content deleted outright, graph edges and all. We verified down to zero with counting queries, not vibes.
2. Make containment transitive. Sealing a board now seals its cards. The indexer computes the vault set through the containment edge, and the seal operation walks the board's cards itself, encrypting each existing embedding in place so the vector stays searchable while the words go dark, and stamping the vault flag onto everything derived from them. Unsealing walks it back. The board is the only container in Margin whose children are woven independently, which is what let the fix be precise instead of sprawling.
3. Write the test that would have caught it. Our vault tests all checked that a sealed item stays sealed. Not one of them checked that a sealed container's contents stay sealed. That test exists now.

Sensitivity has layers#

The incident made us write down something we'd been treating as obvious: privacy comes in more than one kind.

Three layers of sensitivity

The Vault

content the assistant should not be able to read at all until you unlock it.

Secrets

the passwords and keys and tokens people paste into notes. These shouldn't be woven at all, ever. A credential inside an AI memory system is a liability with no matching upside, so those now get scrubbed out of the pipeline before any model or index ever sees them.

Quiet content

things you're happy to have woven but wouldn't want leading the queue. A diary entry that's fine to remember and wrong to volunteer.

When I first wrote this, the third layer was a design on the roadmap. It has since shipped as weave privately: the Mind still learns from the item, but its suggestions go into a collapsed group, recall never leads with it, and it stays out of exports. I still don't have a clean answer for where the line falls between "remember quietly" and "never surface," because that line runs differently for every person, and probably differently for the same person on two different days.