How to Clean Up a Qdrant Collection

Every crawl, retried job, and embedding pipeline change writes points into a vector collection. The stored data keeps moving even when the query code never changes, and the top results move with it.

At first, little looks wrong. Search returns results, and latency stays normal. In our baseline run, the context-relevance score sat at 0.92 out of 1.00 while four answers in ten came back wrong, because duplicate chunks and one outdated record were filling the five results the agent could read.

We reproduced this pattern in a Qdrant and Future AGI webinar using a controlled Pokédex collection. The example was small enough to inspect by hand, but the same failure modes appear in product catalogs, support knowledge bases, recommendations, and agent memory.

Growth Exposes Flaws That Shipped With the First Ingest

A larger collection creates more competition for each result slot. Weak identity rules, stale records, and poor retrieval choices are usually there from the first ingest. Growth only raises how often a query meets them.

We ran 33 of our 37 test questions against the collection at three sizes, from 1,314 points up to 22,946, and changed nothing else.

The smallest collection is the one to look at: with 1,314 points, repeats already held 44% of the five slots.

When quality falls after an ingest, inspect the collection before changing prompts or agent logic. Compare the point count with the source count, sample the top-k results your application receives, and check whether the same content or an older version of it appears more than once.

Deduplication Works by Freeing Result Slots

Repeated ingestion had turned 8,416 distinct points into 22,946 total points. Removing the 14,530 unintended copies dropped queries with a duplicate in the top five from 36 of 37 to 2 of 37, and answer correctness, scored against known-good answers, rose from 0.57 to 0.76.

The agent reads the first five results of each search and nothing below them, so those five slots are all the evidence one search can offer. With the copies gone, they held five different chunks, and answer quality followed. The point count fell as a side effect.

Stable point IDs prevent most duplication at the source. Qdrant point loading is idempotent, so a retry under the same ID updates the point already there instead of adding another one. Duplicates accumulate when each ingest assigns fresh IDs to content the collection already holds.

An exact copy is easy to find: hash the text of each point and compare the hashes. Deleting one takes more care, because the same text can legitimately sit in two places, once per tenant or once per language. Records that are merely similar have no automatic rule, because whether they count as the same thing depends on what your application does with them.

Better Retrieval Can Still Produce a Worse Answer

Retrieval quality and answer quality move independently, so we grouped the 37 test questions to exercise one failure at a time. A stronger embedding model took the 14 questions built on near-identical entries from 0.64 to 1.00 on Recall@5, and answer correctness across the full set reached 0.92.

Hybrid search adds two steps to the query path, fusion and reranking. On the 18 ranking questions, fusing sparse and dense results scored 0.72, below the 0.78 plain dense search already reached, and the ColBERT reranking step is what carried the group to 0.89. Fusion on its own would have been a regression.

Then answer correctness fell, 0.92 to 0.86, on the change that improved every ranking metric.
The agent had been rewording failed queries and searching again, so the answer column never registered the ranking problem and had nothing to gain from the fix. Searches per question dropped from 2.0 at baseline to 1.14, which is where the improvement showed up instead.

Two checks disagreed about those same answers. Groundedness passed them, because the claims did come from the retrieved text. The hallucination check posted its worst reading of the run, because the answers also carried detail the sources never mentioned.

An agent that retries covers for bad retrieval, which is why the answer column missed both the problem and the fix.

Freshness Belongs in the Data Model

Similarity can’t decide which of two conflicting records is current. An older policy, price, or product state may be a close semantic match and still be wrong for the request.

Our collection contained one outdated type-chart record that was correct for its historical version. Every retrieval and grounding check passed the answer built on it, because the answer reflected that record accurately. Only the correctness judge stayed red, and it stayed red through all four retrieval upgrades.

An is_current payload filter removed the stale record from current queries, which recovered answer correctness to 0.92.

Old records can stay in the collection, as long as something marks which of them is current. status, version, is_current, and updated_at are the usual fields, and they let each query state what it should retrieve. A payload index on every one of them keeps lifecycle rules part of retrieval, and it is what lets filtering run at all on a cluster that rejects unindexed fields.

Pick Metrics That Can See the Failure

Chunk utilization, which measures how much of the retrieved context the generator uses, read 0.85 before deduplication and 0.85 after, while answer correctness rose 0.19 over the same change. Five copies of one chunk score the same as five distinct chunks, so the metric never had a way to register duplication.

One score rarely locates the failing layer, and two read together usually do. Low context relevance with low chunk utilization points at retrieval. High relevance with a failing correctness or groundedness score points at the generator or at the data behind it, and that is the reading that would have pointed at the outdated type chart four stages earlier.

One failure stays invisible to every check in this post: a record that never got ingested. Retrieval metrics score what came back, and answer metrics score what the agent said. Comparing the collection against the source list is the check that catches it.

Watch the Full Walkthrough

The recording follows these problems through a working RAG system. Dylan Couzon changes the retrieval path in Qdrant, while Rishav Hada traces and evaluates the agent in Future AGI.

Watch the recording on YouTube, or see Future AGI’s partner recap for more on its evaluation workflow.