Blog

Modelling Your Data: Collections, Multitenancy and Named Vectors

Written by

One collection or many, how to isolate tenants without a collection each, and what named vectors, multivectors and sparse vectors are for. The design decisions you make once and live with.

Part 7 left a question open: if per-tenant filters sit exactly in the selectivity range that fragments the graph, should each tenant get its own collection instead?

That’s one of several modelling decisions that are cheap now and expensive later. This part works through them: how many collections, how to isolate tenants, how to store more than one vector per thing, and when a single point should carry several representations.

Qdrant 1.19.1 throughout, with the API shapes read from a live server.

Try this first

You’re building search for a product with 5,000 customers. Each has between 20 and 2,000,000 documents, and no customer may ever see another’s.

Sketch the collection layout. Then answer one question about it: what happens when customer number 5,001 signs up at 3am?

That question is the whole section.

Multitenancy: three designs

One collection per tenant

The obvious isolation. No filter needed, because the collection is the boundary.

It works well for a handful of large tenants and becomes an operational problem at scale. Every collection carries its own segments, its own index structures and its own memory overhead, so thousands of collections means thousands of sets of fixed costs — and most of your tenants are small, so that overhead dominates their actual data.

It also makes cross-tenant operations awkward, and makes provisioning a runtime concern: customer 5,001 signing up at 3am requires creating a collection before their first write.

Use it when: you have few tenants, they’re large, or you have a hard requirement for physical separation.

One collection, tenant in the payload

Everything in one collection, with tenant as an indexed payload field and every query filtered by it.

client.create_payload_index(
    "docs", field_name="tenant", field_schema=models.PayloadSchemaType.KEYWORD,
)

hits = client.query_points(
    "docs", query=vector, limit=10,
    query_filter=models.Filter(must=[
        models.FieldCondition(key="tenant", match=models.MatchValue(value="acme")),
    ]),
).points

One set of fixed costs, no provisioning step, and a new tenant is just a new payload value.

The problem is exactly Part 7’s: a per-tenant filter is selective, and selective filters fragment the graph.

One collection, tenant in the payload, with payload_m

The same as above plus the fix, and this is the normal answer:

client.update_collection(
    "docs",
    hnsw_config=models.HnswConfigDiff(m=16, payload_m=16),
)

payload_m builds extra graph links between points that share a payload value, so each tenant’s points form a connected subgraph by construction. Per-tenant search then traverses a graph that was built for that question.

Combined with full_scan_threshold, the two mechanisms cover the whole range of tenant sizes: tiny tenants fall below the threshold and get an exact scan, large tenants get a navigable per-tenant subgraph. That is why the payload approach scales to thousands of tenants where collection-per-tenant does not.

Four tenants, three layouts acme globex initech umbrella

Schematic — no timings, because this part runs no lab. What it compares is structure: how many indexes exist and where the tenant boundary lives. Note that in the payload layouts the boundary is enforced by your query carrying the filter, not by the database.

A warning about isolation. Payload-based multitenancy is a logical boundary enforced by your application remembering to add the filter. One missing filter is a cross-tenant data leak. Put the filter in one place — a wrapper function every query goes through — and never build filters ad hoc at call sites. If your compliance position requires that a bug cannot leak data across tenants, you need the collection boundary, and the operational cost is the price of that guarantee.

Named vectors: several representations, one point

A point can carry more than one vector, each with its own name, dimension and distance metric.

client.create_collection(
    "products",
    vectors_config={
        "text": models.VectorParams(size=384, distance=models.Distance.COSINE),
        "image": models.VectorParams(size=512, distance=models.Distance.COSINE),
    },
)

client.upsert("products", points=[
    models.PointStruct(id=1, vector={"text": text_vec, "image": image_vec},
                       payload={"sku": "A-1182"}),
])

hits = client.query_points("products", query=text_vec, using="text", limit=10).points

This is the right shape whenever one thing has several representations: a product with a description and a photo, a document with a title embedding and a body embedding, the same text under two different models during a migration.

The alternative — separate collections joined on an id in your application — means two round trips and keeping two collections consistent. Named vectors keep the point atomic: one upsert, one delete, one payload.

Each named vector is its own index. Memory is the sum, and so is build time. Adding a second 768-dimension vector to ten million points is another ~29 GiB of raw vectors before anything else, so this is a decision with an arithmetic consequence you can work out in advance (Part 5).

Sparse vectors: the other kind of search

A dense vector is a few hundred numbers, nearly all non-zero, produced by a neural model. A sparse vector is a much larger space — one dimension per vocabulary term — where almost every value is zero.

client.create_collection(
    "docs",
    vectors_config={"dense": models.VectorParams(size=384, distance=models.Distance.COSINE)},
    sparse_vectors_config={"lex": models.SparseVectorParams()},
)

client.upsert("docs", points=[
    models.PointStruct(
        id=1,
        vector={
            "dense": dense_vec,
            "lex": models.SparseVector(indices=[3, 17, 9042], values=[0.7, 0.4, 1.2]),
        },
    ),
])

You store only the non-zero entries: a list of indices and a list of values. That’s what indices and values are, and it’s why a sparse vector over a 30,000-term vocabulary costs about as much as a handful of numbers.

Sparse vectors are how you match exact terms — a part number, an error code, a surname — which is precisely what dense embeddings are bad at, as Part 2’s failed query showed. Storing both on the same point is the setup for Part 9.

Multivectors: many vectors under one name

Different again from named vectors. A multivector is a list of vectors under a single name, compared with a configurable aggregation.

The use case is late-interaction retrieval models, which represent a document as one vector per token rather than one vector per document. Matching then compares every query token against every document token and aggregates — much more precise than squeezing a document into a single point, and much more expensive.

The cost is the thing to notice: a document with 200 token vectors occupies 200 vectors’ worth of memory. Multivectors are a quality-for-resources trade at a scale most systems don’t need, and they’re worth knowing exist so you recognise the technique when you meet it.

Chunking: the decision that isn’t in the API

Nothing in Qdrant makes you choose a chunk size, and it affects your results more than most settings in this series.

A vector represents one piece of text. Too large and the embedding is an average of several topics, matching everything vaguely and nothing well. Too small and each chunk lacks the context to be interpretable, and your results are fragments.

Two things are worth knowing regardless of where you land:

One point per chunk, with the parent document id in the payload. That way you can retrieve chunks and group by document, or fetch the whole document for display, without duplicating text across points.

Overlap costs points. Overlapping chunks by a few sentences helps a query that straddles a boundary, and multiplies your point count by roughly the overlap factor — which is a memory and index-size decision, not just a quality one.

Where to land is measured, not reasoned about, and Part 14 sets up the harness for it.

Explain it like I’m ten

Imagine a school storing every pupil’s work.

A cupboard per class keeps everything tidy and separate. Fine with ten classes. With five thousand classes you need five thousand cupboards, most of them nearly empty, and every new class means buying furniture before anyone can hand anything in.

One big room with a name on every piece of work needs no furniture and handles a new class instantly. But now finding one pupil’s work means checking names — and if someone forgets to check, they hand back the wrong child’s homework. That’s the real risk: the separation is a rule people follow, not a wall.

Named vectors are filing the same piece of work under two systems at once — by subject and by date — so you can search either way without two copies.

Sparse vectors are the index at the back of a textbook. It doesn’t know what anything means; it knows exactly which page says “photosynthesis”. Useless for “explain plants to me”, perfect for finding one specific word.

Where the analogy breaks: cupboards in a real school genuinely stop someone opening the wrong one. In a single collection the “names” are checked by your code every time, so the separation is only as good as the code — which is why the choice between them is a security decision, not only a performance one.

The precise version

Collection-per-tenant gives T independent indexes. Fixed overhead — segments, index structures, metadata — is paid T times, so total cost is Θ(T·c + N) rather than Θ(c + N). With most tenants small, the T·c term dominates.

Payload multitenancy gives one index of N points; a tenant query is a filtered search of selectivity s = nt/N. From Part 7, the induced subgraph fragments as s·m approaches 1, which for small tenants in a large collection is exactly the regime. payload_m restores degree within the payload group independently of s, and full_scan_threshold bypasses the graph when nt is small — so the two mechanisms cover nt small and nt large respectively, leaving only the middle to measure.

Named vectors are v independent indexes over the same point set; storage is Σi N·di·b and each query names one. Multivectors store ntok vectors under one name with an aggregation over pairwise comparisons, so both storage and query cost scale with tokens per document rather than documents.

Trade-offs

Collection per tenant against payload multitenancy. Physical isolation and simple queries, against one set of fixed costs and instant provisioning. Few large tenants favours the first; many small ones strongly favours the second.

payload_m against memory. Extra links per payload group. Worth it for the field you filter on constantly; not for every field.

Named vectors against separate collections. Atomic points and one round trip, against independent lifecycle and the ability to scale each representation separately.

Sparse plus dense against dense alone. Better handling of exact terms and rare tokens, against a second index to build and maintain.

Chunk size. Larger chunks are fewer points, less memory and blurrier matches. Smaller chunks are sharper matches and more points, and a higher chance of returning something with no context.

Common mistakes

Collection per tenant at thousands of tenants. Fixed overhead multiplied by a large number, most of it for tenants with a few hundred points.

Payload multitenancy without payload_m. The default configuration for the exact filter selectivity that fragments the graph, and it degrades quietly.

Building the tenant filter at each call site. One forgotten filter is a cross-tenant leak. Wrap it once.

Treating payload multitenancy as a security boundary. It’s a logical boundary maintained by your code. If you need a guarantee, you need separate collections.

Two collections for two representations of one thing. Named vectors exist for this, and keep the point atomic.

Using dense vectors where a term match is wanted. Exact identifiers are what sparse vectors are for.

Chunking by a fixed character count without looking at the output. The cheapest quality improvement available to most systems is reading twenty of your own chunks.

Interview questions

1. Collection per tenant, or a tenant field?

Answer A tenant field with a payload index and `payload_m` set on it, for most systems. One set of fixed costs, instant provisioning, and the graph problem has a direct fix. Combined with `full_scan_threshold`, small and large tenants are both handled. Collection per tenant when tenants are few and large, or when you need physical isolation — because payload multitenancy is enforced by your application remembering the filter. **Follow-up:** what breaks collection-per-tenant at scale? Per-collection fixed overhead multiplied by tenant count, with most tenants too small to amortise it, plus provisioning becoming a runtime dependency.

2. Why does per-tenant filtering need payload_m?

Answer Because a tenant filter is selective, and HNSW links were built from vector proximity with no knowledge of tenants. The subgraph of one tenant’s points is likely to be fragmented, so a filtered walk gets trapped in one component and silently returns poor results. `payload_m` builds extra links between points sharing the payload value, making each tenant’s subgraph navigable by construction. **Follow-up:** what about a tenant with fifty points? It falls under `full_scan_threshold` and gets an exact scan, which is both faster and exact. The two mechanisms cover different tenant sizes.

3. What are named vectors for?

Answer Storing several representations of the same thing on one point — a product’s text and image embeddings, a document’s title and body, or the same content under two models during a migration. Each has its own dimension, metric and index, and a query names which to use. The alternative is separate collections joined by id in your application, which costs a round trip and a consistency problem. Named vectors keep one point, one upsert, one delete. **Follow-up:** what do they cost? Memory and build time are the sum of the indexes. Adding a 768-dimension vector to ten million points is another ~29 GiB of raw vectors.

4. What’s a sparse vector and when do you need one?

Answer A vector over a vocabulary-sized space where almost every entry is zero, stored as a list of indices and values. It encodes which terms appear and how important they are. You need one when exact terms matter — part numbers, error codes, names, rare jargon — which is exactly where dense embeddings are weakest, because they encode meaning rather than tokens. **Follow-up:** so do you replace dense with sparse? No, you keep both on the same point and combine the results, which is hybrid search.

5. How would you migrate to a new embedding model with no downtime?

Answer Two workable shapes. Add a second **named vector** for the new model, backfill it, verify quality against the old one, then switch queries to `using=”new”` and drop the old vector when you’re confident. Or build a whole new collection and swap an alias, which is Part 11. The named-vector route avoids a second copy of payloads and keeps points atomic; the alias route gives a cleaner rollback and lets you change dimension or metric, which cannot be altered in place. **Follow-up:** why can’t you just overwrite the vectors in place? You’d be serving a mix of two coordinate systems while the backfill runs, and scores between them are meaningless.

6. How do multivectors differ from named vectors?

Answer Named vectors are several *named* representations, one each, queried individually. Multivectors are a *list* of vectors under a single name, compared with an aggregation over pairwise comparisons. Multivectors exist for late-interaction models that represent a document as one vector per token. Much more precise matching, and the storage and query cost scale with tokens per document rather than with documents — which is a serious multiplier. **Follow-up:** when would you reach for them? When retrieval quality is the binding constraint and you can afford the resources, typically after simpler approaches like hybrid search and reranking.

7. Where should chunk boundaries go?

Answer Where the meaning is. Prefer natural boundaries — paragraphs, sections, headings — over a fixed character count, because a chunk that spans two topics embeds as the average of both and matches neither well. Keep one point per chunk with the parent document id in the payload so you can group results or fetch the whole document. Overlap helps queries that straddle a boundary and multiplies your point count. **Follow-up:** how do you choose the size? Measure it end to end on real queries. It affects results more than most index settings and cannot be reasoned to.

8. A tenant filter was missing from one code path. How bad is that, and how do you prevent it?

Answer It’s a cross-tenant data leak — one customer sees another’s documents. In payload multitenancy the boundary is your code, so a missing filter removes it entirely. Prevention is structural, not procedural: one wrapper that every query goes through and that takes the tenant as a required argument, so a query without one doesn’t compile or doesn’t run. Never construct filters ad hoc at call sites. **Follow-up:** and if that isn’t enough assurance? Separate collections, where the boundary is enforced by the database rather than by your code. That’s what the operational overhead buys.

Sources

What to remember

For multitenancy, the default answer is one collection with an indexed tenant field, payload_m set on it, and every query going through a single wrapper that adds the filter. Collection-per-tenant is for few large tenants or a hard isolation requirement, and its cost is fixed overhead multiplied by tenant count.

Named vectors let one point carry several representations. Sparse vectors handle the exact terms dense embeddings miss. Multivectors trade a lot of memory for late-interaction precision.

And the decision that isn’t in the API at all — where your chunks begin and end — will affect your results more than most of the ones that are.

Payload multitenancy is a boundary your code maintains. Decide whether that is the kind of boundary you need before you decide whether it is fast enough.

How useful was this post?

Click on a heart to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.