Blog

Storage, Memory and mmap: Where the RAM Actually Goes

Written by

Segments, the optimiser, the write-ahead log and what mmap really does. How a collection is laid out on disk, what has to be in RAM for search to be fast, and why your first query after a restart is the slow one.

Every part so far has treated a collection as a thing that holds vectors. This part opens the box: what is on disk, what is in memory, what moves between them and when.

It matters because the single biggest operational decision in a vector database is what you keep in RAM. Part 5 was about making the vectors smaller. This part is about deciding what has to be resident at all.

Qdrant 1.19.1, with the defaults read from a live collection.

Try this first

You restart a Qdrant node with ten million vectors. It comes up and reports ready.

The first query takes noticeably longer than the hundredth.

Write down why — and what you’d have to change to make the first query fast. There are two plausible answers and they lead to different configurations.

Segments: the unit of everything

A collection is not one big index. It is a set of segments, each an independent, largely self-contained store with its own vectors, its own payload index and its own HNSW graph.

A search runs against every segment and merges the results. A write goes into one segment.

That design explains a lot of Qdrant’s behaviour:

  • Why indexed_vectors_count disagrees with points_count. Indexing is per segment, and a segment below indexing_threshold — default 10,000 — has no HNSW graph at all. Its points are found by scanning. A collection is routinely a mix.
  • Why writes don’t block searches. New data goes into a small, growing segment while the large ones stay stable and readable.
  • Why performance changes without you doing anything. The optimiser is reorganising segments in the background.

The optimiser

Background work that keeps the segment layout sane. It merges small segments into larger ones, rebuilds indexes when a segment crosses a threshold, and reclaims space from deleted points.

The settings that matter, with their defaults on 1.19.1:

{
  "indexing_threshold": 10000,
  "memmap_threshold": null,
  "default_segment_number": 0,
  "flush_interval_sec": 5
}

indexing_threshold — a segment with fewer vectors than this gets no HNSW graph. Set it to 0 and indexing is disabled entirely, which is the standard trick for bulk loading: upload everything with no graph being built and rebuilt as segments grow, then raise the threshold and let the optimiser build once.

# Bulk load: no graph while the data goes in.
client.update_collection("docs",
    optimizers_config=models.OptimizersConfigDiff(indexing_threshold=0))
# ... upload everything ...
# Now build it, once.
client.update_collection("docs",
    optimizers_config=models.OptimizersConfigDiff(indexing_threshold=10000))

default_segment_number: 0 means Qdrant chooses based on available CPUs. More segments means more parallelism across cores and more per-segment overhead; fewer means bigger, more efficient indexes and less concurrency within one search.

Deletes are not immediate. A deleted point is flagged, not removed, and the space comes back when the optimiser rewrites that segment. A collection that has had heavy deletion keeps occupying the space until then.

The write-ahead log

Every write goes to a WAL before it is acknowledged. If the process dies, the WAL is replayed on startup and nothing acknowledged is lost.

flush_interval_sec — default 5 — controls how often in-memory state is flushed to permanent storage. It does not control durability of acknowledged writes; the WAL does that. What it affects is how much WAL has to be replayed after a crash, which is startup time rather than data loss.

This is the same design as a relational database’s write-ahead log, and for the same reason: appending sequentially is fast, and rewriting index structures on every write is not.

mmap: the important one

Here is the decision this part exists for.

Qdrant can either load vectors into its own memory or memory-map the files that hold them. With mmap, the file stays on disk and the operating system pages parts of it into RAM as they’re touched.

client.update_collection(
    "docs",
    vectors_config=models.VectorParamsDiff(on_disk=True),          # vectors via mmap
    hnsw_config=models.HnswConfigDiff(on_disk=True),               # the graph too
)

There is also optimizers_config.memmap_threshold — null by default — which makes a segment switch to mmap once it exceeds a given size, so small segments stay in memory and large ones don’t.

What mmap actually buys you: a collection larger than RAM. That is the whole point and it is a big point — it turns “we need a bigger machine” into “we need a faster disk”.

What it costs: a page fault whenever a query touches a page that isn’t resident. A graph traversal jumps around memory by design, so on a cold cache that’s many random reads. On NVMe that’s tolerable. On network storage it is not, and this is the configuration that produces “the database is fine and the p99 is terrible”.

The crucial subtlety: with mmap, your working set lives in the OS page cache, not in Qdrant’s memory. So “how much RAM does Qdrant use” becomes the wrong question — free will show memory as cache rather than as process memory, and a monitoring alert on process RSS will look healthy while the page cache is thrashing.

What to put where

The shape that works in production:

  • Quantized vectors in RAM (always_ram=True from Part 5).
  • Original vectors on disk via mmap, read only when rescoring.
  • The HNSW graph in RAM, because traversal is exactly the random-access pattern that punishes a cold page.
  • Payloads on disk, which is already the default (on_disk_payload: true), with payload indexes available for the fields you filter on.

That gives you a hot path that never touches disk and a rescoring path that does, which is the honest trade Part 5 described from the other side.

Where each structure lives RAM disk (mmap) HNSW graph random access, every query quantized vectors read on the fast pass original vectors read only when rescoring payloads read for returned results

Schematic — this part runs no lab, so nothing is timed. What it compares is which structures are resident and what each one’s access pattern is. The recommendation follows from the pattern: sequential reads tolerate mmap, and the random jumps of a graph traversal do not.

So why is the first query slow?

Back to the opening question. Two candidate answers, and they lead to different fixes.

The page cache is cold. With mmap, nothing is resident until it’s touched. Early queries fault pages in; later ones find them cached. The fix is to warm it — run representative queries after startup before taking traffic — or to not use mmap for the structures on the hot path.

The optimiser is still working. After a restart or a bulk load, segments may be unindexed or mid-merge. Searches against them scan. The fix is to wait for the collection to settle and check indexed_vectors_count before declaring readiness.

They look identical from the outside and the distinction is the first thing to establish. Check whether the collection is fully indexed; if it is, you’re looking at page faults.

A readiness check that actually means something is worth more than either fix:

info = client.get_collection("docs")
ready = (info.status == models.CollectionStatus.GREEN
         and info.indexed_vectors_count == info.points_count)

Explain it like I’m ten

Imagine a workshop with a small bench and a big storeroom.

Keeping things in memory is having every tool laid out on the bench. Instant to reach, and the bench only holds so much.

mmap is leaving the tools in the storeroom and fetching each one the first time you need it. You can have far more tools than the bench holds. The first time you reach for something there’s a walk to the storeroom; after that it’s on the bench.

The catch is what happens when the bench fills up. Something has to go back, and if your work keeps needing tools you just put away, you spend the whole day walking. That’s a system that technically works and is unusably slow, and it looks fine from outside because the bench is never empty.

Segments are the crates everything arrives in. You can work straight out of a crate, but it’s slow, so periodically you tip several crates into one properly organised drawer. That’s the optimiser, and it’s why the workshop gets faster a little while after a delivery.

Where the analogy breaks: you can see the bench filling up and decide what to put back. The operating system decides that for you, invisibly, and Qdrant isn’t told. That’s why the memory looks fine right up until the page cache stops holding your working set.

The precise version

A collection is a set of segments S1…Sn, each holding vector storage, payload storage, payload indexes and optionally an HNSW graph. Query cost is the sum over segments plus a merge; a segment with |Si| < indexing_threshold contributes an exact scan rather than a traversal.

on_disk selects the storage backend for a structure: resident allocation versus a memory mapping. With mmap, residency is the page cache’s decision, governed by access pattern and memory pressure, not by Qdrant.

The access patterns differ sharply and that is what drives the recommendation:

  • Vector storage during a scan is sequential — the kernel’s readahead works, and mmap costs little.
  • HNSW traversal is random over the whole graph by construction; readahead cannot help, and each miss is a fault. This is why the graph is the structure you least want paged out.
  • Rescoring (Part 5) reads original vectors for the candidate set: random, but bounded by limit × oversampling per query, so its cost is predictable.

Durability is the WAL’s: a write is acknowledged after it is in the log. flush_interval_sec governs checkpointing of in-memory state and therefore replay length at startup, not the durability of acknowledged writes.

Trade-offs

mmap against RAM. A collection bigger than memory, paid for in page faults on the hot path. Excellent on NVMe, poor on network storage.

Quantized in RAM, originals on disk. The standard production shape: the fast path stays resident, the exact path pays disk. Costs you I/O proportional to your oversampling factor.

indexing_threshold low against high. Low means more segments carry graphs — better search, more build work and more memory. 0 disables indexing, which is right during a bulk load and wrong afterwards.

Segment count. More segments parallelise across cores and add per-segment overhead; fewer give larger, more efficient indexes.

flush_interval_sec short against long. Shorter means less WAL to replay at startup and more frequent write amplification. It is a startup-time knob, not a durability one.

Common mistakes

Putting the HNSW graph on disk to save memory. Traversal is random access; this is the structure that suffers most. Move vectors before you move the graph.

Monitoring process RSS with mmap enabled. The working set is in the page cache, not process memory. RSS looks healthy while the machine pages heavily.

Running mmap on network storage. Random reads at network latency, per page fault, inside a graph traversal.

Benchmarking immediately after a restart or bulk load. Cold page cache plus an unfinished optimiser. Wait for indexed_vectors_count == points_count and warm the cache first.

Bulk loading with indexing on. The graph is built and rebuilt as segments grow and merge. Set indexing_threshold=0, load, then restore it.

Expecting deletes to free space immediately. They’re flagged; space returns when the optimiser rewrites the segment.

Treating flush_interval_sec as a durability setting. The WAL provides durability. This controls checkpointing and therefore restart time.

Interview questions

1. What is a segment, and why does Qdrant have several?

Answer An independent store with its own vectors, payload index and optionally an HNSW graph. A search runs against all of them and merges; a write goes into one. It gives you writes that don’t block reads, parallelism across cores, and background reorganisation. It’s also why `indexed_vectors_count` and `points_count` differ: indexing is per segment and a segment below `indexing_threshold` has no graph. **Follow-up:** what does the optimiser do about it? Merges small segments into larger ones, builds indexes as segments cross the threshold, and reclaims space from deleted points.

2. What does on_disk=True actually change?

Answer It memory-maps the structure instead of loading it into Qdrant’s own memory. The file stays on disk and the OS pages it in on access. That lets a collection exceed RAM, which is the point. The cost is a page fault on every access to a non-resident page, and the residency decision belongs to the kernel rather than to Qdrant. **Follow-up:** which structure would you never put on disk first? The HNSW graph — traversal is random access across the whole structure, so readahead can’t help and every miss is a fault.

3. Why is the first query after a restart slow?

Answer Two possibilities, and you have to tell them apart. Either the page cache is cold, so early queries are faulting data in; or the optimiser hasn’t finished, so some segments are unindexed and being scanned. Check `indexed_vectors_count == points_count` first — that distinguishes them in one call. If it’s fully indexed, you’re looking at page faults, and the fix is warming or not using mmap on the hot path. **Follow-up:** how would you warm it? Replay a sample of representative queries after startup, before the node takes traffic.

4. You’re loading 50 million vectors. How do you configure the load?

Answer Set `indexing_threshold=0` first, so no HNSW graph is built while data is arriving. Otherwise the optimiser builds and rebuilds graphs as segments grow and merge, and you pay for indexing many times over. Upload in batches, then restore `indexing_threshold` and let the optimiser build once. Wait for the collection to go green and for the indexed count to match before serving. **Follow-up:** what else would you check afterwards? Memory. 50 million vectors is where the quantization arithmetic from Part 5 stops being academic.

5. What’s the recommended storage layout for a large production collection?

Answer Quantized vectors pinned in RAM, original vectors on disk via mmap for rescoring, the HNSW graph in RAM, and payloads on disk — which is already the default. That keeps the hot path entirely resident and confines disk access to rescoring, whose cost is bounded by `limit × oversampling` per query and therefore predictable. **Follow-up:** what breaks it? Slow storage. The whole design assumes the rescoring reads are cheap, which means NVMe, not network storage.

6. Does flush_interval_sec affect durability?

Answer No. Durability comes from the write-ahead log: a write is acknowledged once it’s in the log, and the log is replayed after a crash. `flush_interval_sec` controls how often in-memory state is checkpointed, which determines how much WAL has to be replayed at startup. It’s a restart-time knob. **Follow-up:** so what would you tune it for? Startup time after a crash, traded against write amplification from more frequent flushing.

7. Memory usage looks fine but p99 latency is terrible. Where do you look?

Answer Page cache, if anything is memory-mapped. With mmap the working set lives in the OS cache rather than Qdrant’s process memory, so RSS looks healthy while the machine pages heavily. Look at page fault rates and disk read throughput, not process memory. Then check what’s on disk — if the HNSW graph is, that alone explains it. **Follow-up:** what’s the fix if the data genuinely doesn’t fit? Quantize so the hot path shrinks enough to be resident, and accept disk only on the rescoring path.

8. You deleted 30% of a collection and nothing was freed. Why?

Answer Deletes are flags, not removals. The points stop appearing in results immediately and the space comes back when the optimiser rewrites the affected segments. Until then you’re carrying the storage and, for unrewritten segments, some of the search cost. On a collection with heavy churn this is worth watching, because the gap between logical and physical size can get large. **Follow-up:** can you force it? You can adjust optimiser settings to trigger the work sooner, and the underlying constraint is that reclaiming space means rewriting segments, which costs I/O.

Sources

  • Qdrant documentation, https://qdrant.tech/documentation/concepts/storage/ — segments, the optimiser, memory mapping and the write-ahead log.
  • Defaults read from a fresh collection on Qdrant 1.19.1: indexing_threshold: 10000, memmap_threshold: null, default_segment_number: 0, flush_interval_sec: 5, and on_disk_payload: true.
  • The access-pattern argument for keeping the HNSW graph resident follows from the traversal described in Part 3 and the algorithm in Malkov & Yashunin (arXiv:1603.09320).

What to remember

A collection is segments, each with its own storage and possibly its own graph, reorganised continuously by the optimiser. That’s why indexed and total counts differ, why performance shifts on its own, and why bulk loading wants indexing switched off.

mmap is how a collection outgrows RAM, and it moves your working set into the OS page cache where your process metrics can’t see it. Put the graph and the quantized vectors in memory, the originals and payloads on disk, and make sure the disk is fast.

And when the first query after a restart is slow, find out whether you’re waiting on the page cache or on the optimiser before you change anything.

With mmap, “how much memory is Qdrant using” stops being the question. The question is whether your working set is in the page cache, and nothing in your process metrics will tell you.

How useful was this post?

Click on a heart to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.