Cross-tenant leakage in vector search, and how to test for it
A multi-tenant RAG system can have an access-control filter on every query and still leak between tenants. The filter is usually real. The problem is where it runs. Here are the four places it goes wrong, what each leak actually exposes, and a canary test that finds all of them.
"We filter by tenant" is not a security property until you know whether the filter runs inside the index search, after it, or at the prompt. Only the first is isolation. The other two are a cleanup step that leaks through its side effects.
Contents
01The threat model
Two tenants, A and B, share one vector collection. Each chunk carries a tenant id in its metadata. Every query is supposed to return only the caller's own chunks. Tenant B is the attacker: an ordinary authenticated user who can issue any query they like, choose the retrieval depth if the API exposes it, and read whatever comes back, including scores, latency and headers.
The leaks below range from the serious, foreign text landing in the model's context, to the subtle, learning that a foreign document exists and roughly what it is about. The subtle ones matter because they are repeatable, and a repeatable one-bit oracle becomes an extraction tool with enough queries.
02Leak 1: post-filtering after top-k
The commonest mistake. The vector database searches the whole collection for the k nearest neighbours by similarity alone, say k of 10, and the application applies the tenant filter afterwards to that list of ten.
If tenant A's chunks are semantically closer to the query than tenant B's, A's chunks fill the ten slots. The filter then removes them, and B gets zero or very few results, even though B has perfectly good documents a little further down the ranking.
The direct effect is a correctness bug: B's recall depends on A's data. The security effect is an oracle. B can issue a query and observe whether their own results are suppressed. Suppression means something in another tenant's data is closer to this query than my own documents. By varying the query, B maps out what topics, names and phrasings exist in A's data, without ever seeing a byte of it. Force the top-k to be exhausted by foreign chunks and the signal is binary and clean.
Not A's text. The existence, topic and vocabulary of A's documents, one bit per query, via the size of B's own result set. That is enough to confirm a project name, a client, or a figure, by asking about it and watching your own results vanish.
03Leak 2: filtered graph traversal and late filters
Most vector databases use a graph index such as HNSW. Search starts at entry nodes and hops to ever-closer neighbours. When pre-filtering is switched on and the filter hides most nodes, the visible part of the graph can fragment into disconnected islands. The traversal gets stuck in a local cluster and returns poor results for the filtered tenant, which is why databases offer dedicated filtered-search modes and why a restrictive filter can hurt recall in ways that look random.
Two consequences matter for isolation. First, teams that hit this degradation sometimes "fix" it by moving the filter out of the index and applying it later, which recreates leak 1. Second, and worse, some pipelines apply the tenant check at the prompt layer or in the model's instructions rather than in the retrieval path. By then the foreign chunks are already in the context window. A model told to ignore text it should not have seen is not an access control; it is a request.
Whether a filtered traversal itself leaks through timing is more theoretical than practical in most deployments, but it is one more reason the clean answer is a hard boundary rather than a clever filter.
04Leak 3: a shared semantic cache
Semantic caches at the gateway store query-response pairs, or query embeddings and the retrieved context, so that a repeated or near-identical question is answered without running the pipeline again. If the cache key is the query, or the query embedding, and not the query plus the tenant id plus the caller's permissions, then the first user to ask a question warms the cache for everyone.
Tenant A asks about their confidential budget and gets an answer built from their private chunk. Tenant B asks the same question and gets A's answer back from the cache, fast, with no retrieval and no filter in the path. This is the one leak on this list that hands over the actual content, and it is the easiest to introduce, because caches get added late by someone optimising latency rather than security.
05Leak 4: exposed distances and scores
If the search API returns raw similarity scores or distances in the response metadata, an attacker can probe the embedding space. Send a family of queries that vary one detail and watch how the scores of your own results shift. Combined with leak 1, where foreign chunks displace yours, the scores tell you how close the foreign cluster is and in which direction, which turns the one-bit oracle into a gradient. Scores are useful for debugging and for rerankers. They should not reach an end user in a multi-tenant system.
06How to test: canaries and starvation probes
The test is black-box against the retrieval API, plus a look at headers and traces where you have them. It takes two accounts and an afternoon.
Step 1: plant a canary
In tenant A, index one chunk containing a unique secret with distinctive vocabulary, something like a line stating the internal budget for a made-up project with an unusual name and a specific figure. In tenant B, the attacker account, index several generic documents on the same general topic, budgets and projects, that overlap semantically with the canary but do not contain the secret name or figure.
Step 2: the top-k starvation probe
Authenticate as B. Ask the question the canary answers, using A's project name. Set the retrieval depth low, k of 3, then raise it.
as B "What is the budget for Project Zephyr?" k=3 → results: 0 as B "What is the budget for Project Zephyr?" k=10 → results: 0 as B "What is the budget for Project Zephyr?" k=50 → results: 4 (B's own generic docs) as B "What is the budget for Project Nothing?" k=3 → results: 3 (B's own generic docs)
- Safe. The system pre-filters inside the index. B gets their own best-matching generic documents at k of 3, or an empty set if nothing is relevant, and the result count does not depend on whether the query names A's project.
- Vulnerable. B gets zero results at k of 3 and results only appear when k is raised far enough for B's own documents to survive. A's canary and its neighbours occupied the top slots during scoring and were dropped afterwards. The result count is an oracle for A's content.
Repeat with queries that name the project but not the figure, and queries that use the figure but not the project, to confirm the signal tracks A's vocabulary and not just general topic density.
Step 3: the cache probe
As A, ask the canary question and confirm the secret comes back. Immediately, as B, send the exact same query string. Then send a paraphrase. Inspect the response body, the latency, and any cache headers or trace spans.
- If B gets A's answer or A's context, the cache is unscoped. Stop and fix this before anything else.
- If B gets a suspiciously fast response with a cache-hit flag but a correct empty or own-tenant answer, check what was cached: a shared retrieval result that is filtered on read is still a shared retrieval result.
Step 4: the score probe
Look at the raw API response as B for scores, distances or vector ids. If they are present, run the starvation probe again and record scores across a family of queries that vary the project name. If the scores of B's own results move in a way that correlates with how close the query is to A's canary, the scores are leaking the shape of A's data.
Step 5: trace the filter
With access to the code or traces, find where the tenant filter runs. It should be a parameter of the vector search call itself, enforced by the database. If it is a list comprehension after the search returns, or a line in the system prompt, it is leak 1 or leak 2 regardless of what the probes showed on this data set.
07Fixes, by root cause
| Symptom | Root cause | Fix |
|---|---|---|
| Top-k starvation | Filtering after the vector search | Native pre-filtering in the vector database, as a parameter of the search call. |
| Recall collapses under a restrictive filter | Graph fragmentation during filtered traversal | Use the database's filtered-search mode, or give each tenant its own namespace or index so no filter is needed. |
| Cache bleed | Cache key lacks tenant and permissions | Bind the tenant id and a hash of the caller's ACL into every cache key. Cache nothing above the retrieval layer that was built from private chunks. |
| Filter at the prompt layer | Access control treated as an instruction | Move the check into the retrieval path. The model must never see what the caller cannot. |
| Scores in responses | Debug metadata exposed to users | Strip scores, distances and internal ids before the response leaves the service. |
| Everything in one collection | No row-level isolation | Logical multi-tenancy with enforced tenant-scoped rules, or physical multi-tenancy with separate namespaces per tenant for anything sensitive. |
The decision between logical and physical isolation is the usual one. Separate namespaces per tenant are the simplest thing to reason about and remove the filter entirely, at the cost of many small indexes. A single collection with enforced pre-filtering scales better and is fine when the database genuinely enforces the filter inside the search. What is never fine is a single collection where the application promises to filter later.
08Takeaways
- Ask where the filter runs. Inside the index is isolation. After the search or in the prompt is a leak.
- Suppression is a signal. Post-filtering leaks the existence and vocabulary of foreign data through your own result count.
- A shared semantic cache leaks content, not just metadata. Tenant id and permissions belong in the cache key.
- Do not return scores to users. They turn a one-bit oracle into a gradient.
- Plant a canary and run the starvation probe. Two accounts, one secret chunk, vary k, watch the count.
- Prefer separate namespaces for anything sensitive. A filter you do not need cannot be applied in the wrong place.
Draws on write-ups by Rewanth Tammana and the OSAI AI red-teaming lesson on cross-tenant leakage in multi-tenant RAG.