Permissions on chunks, and keeping them in sync

01Why permissions do not belong in the vector payload

The obvious design is to store the list of allowed users on each chunk, right next to the vector, and filter on it at search time. It works on day one. It stops working the first time a document with four hundred chunks is shared with a new team, because now four hundred records need their metadata rewritten, and in many vector databases a metadata change means re-upserting the record. Multiply by every share, unshare, team change and offboarding across the source systems, and the index is being rewritten all day to track something that has nothing to do with the embeddings.

Permissions change far more often than content does, and on a different schedule. That is the signal that they belong in a different store.

02Step 1: static references on every chunk

At ingestion, attach only metadata that will not change for the lifetime of the chunk: the document id it came from and the tenant id. Nothing about users or groups. The vector record is now immutable with respect to permissions, so the only reason to touch it again is a change to the document's content.

chunk     { id, vector, text, doc_id, tenant_id, section_id }
not here  allowed_users, allowed_groups, owner, sharing state

Verification. Fetch a sample chunk through the vector store's API and confirm the payload has a document id and tenant id and no user or group arrays. If a user id can be found in any chunk, the design has already been bypassed somewhere in the ingestion code.

03Step 2: a document ACL store

Keep the current document-level permissions in a low-latency store that is good at small updates: PostgreSQL, DynamoDB or Redis all work. One row per document.

doc_acl   doc_id → { tenant_id, allowed_users[], allowed_groups[], updated_at }

This is the only place permissions live. It is written by the sync layer in step 4 and read by the retrieval layer in step 3. It should be small enough to sit in memory for a tenant, and it should carry a timestamp so you can tell how stale it is.

Verification. Query the store for a known document id and confirm it returns exactly the users and groups the source system shows for that document, no more and no fewer.

04Step 3: pre-filter at query time

When a user searches:

1. Take the user id and group memberships from the validated authentication token. Not from a request parameter, and not from a cache that could be stale by a session.
2. Resolve the set of document ids that user may read, by looking up the ACL store for their user id and groups within their tenant.
3. Run the vector search with a mandatory pre-filter: document id in the allowed set, and tenant id equal to the caller's tenant. The filter is a parameter of the search call, enforced by the database during traversal.

Verification. Search as an unprivileged test account for content that lives only in restricted documents. Nothing from those documents should appear at any retrieval depth, and the account's own result count should not change depending on whether the query targets restricted content. The canary and starvation probe from the previous note is the test for this.

05Step 4: webhooks and change data capture

The ACL store is only as good as its freshness. The source systems already know when a permission changes, and most of them will tell you: Google Drive push notifications, SharePoint and OneDrive delta queries, Salesforce change data capture, or a database's own change stream for an in-house system.

1. Subscribe to permission-change events from each source.
2. On an event, publish a message to a sync worker with the document id and the source.
3. The worker re-reads the document's current permissions from the source and overwrites the row in the ACL store. Re-reading rather than applying the delta makes the worker idempotent and safe to replay.

Back the event stream with a periodic full reconciliation, daily or hourly depending on the stakes, because webhooks get dropped and subscriptions expire. The reconciliation walks every document and compares, and the gap it finds is a measurement of how much to trust the event path.

Verification. Change a document's sharing in the source system and watch the ACL store row. It should update within seconds. Then revoke, and confirm the next search as the revoked user no longer returns that document.

06Filter on groups, not users

The allowed set in step 3 can be large. A user in a big organisation may be able to read tens of thousands of documents, and a filter clause with tens of thousands of ids is slow or unsupported in some vector databases. The fix is to resolve permissions at the group level wherever the source system supports it.

Most permission changes are not "share document X with user Y". They are "add user Y to team Z", which grants access to everything the team can see. If the ACL store maps documents to groups and the filter is on group membership, a team change is one row in the identity store and touches neither the ACL store nor the index. Only direct per-user shares need a per-user entry, and those are usually few.

When even the group set is large, flip the filter: store the allowed group ids on the chunk as metadata and filter on the caller's groups, which is a short list. That is the alternative design below, applied to groups, and it combines well with the external store for the per-user cases.

07The alternative: metadata patching in place

If running a separate ACL store is not practical, several vector databases, Qdrant, Pinecone and Milvus among them, support fast metadata-only updates. Then:

• Store the allowed groups, or users, directly in the chunk payload.
• On a permission change, issue one batch metadata update targeting every chunk with that document id.
• The update rewrites the payload in place. No re-embedding, and in these databases no re-indexing of the vector itself.

This keeps the whole thing inside one system, which is simpler to operate. The costs are that every permission change still fans out to every chunk of the document, that the index becomes the system of record for permissions, and that not every database does this cheaply. It is the right choice for small corpora and the wrong one for a large, frequently reshared one.

08Never post-filter

Whichever design you pick, the filter must run inside the vector engine during the search, not on the result list afterwards. Filtering after similarity scoring means the top-k is filled by chunks the caller may not read, and then emptied. The caller sees fewer or zero results, their recall depends on other people's documents, and the suppression itself tells them that restricted content matching their query exists. That failure and the test for it are the subject of the cross-tenant leakage note.

09Takeaways

1. Permissions change on a different schedule from content. Store them in a different place.
2. The chunk carries a document id and a tenant id, nothing else. Vectors stay immutable with respect to access.
3. Resolve the allowed set at query time and pre-filter on it. Inside the search, from the auth token, every request.
4. Let the source system drive the sync. Webhooks or change data capture into a worker that re-reads and overwrites, plus a periodic reconciliation.
5. Filter on groups where you can. Team changes become one row instead of a fan-out.
6. If permissions must live in metadata, patch in place. Batch update by document id, never re-embed.