Qdrant and Minima Deliver 2.92x More Agentic RAG Tasks per GPU-Hour

Reducing Retrieval and Calls

When a retrieval-augmented generation (RAG) agent runs, it often has to plan a search, check the evidence it gets back, and try again when that evidence falls short. Those inefficiencies compound. Every extra retrieval and every extra model call adds latency, context, and inference cost.

To attack that cost, Minima built a bounded retrieval agent. It planned the query, searched Qdrant, decided whether the evidence was sufficient, and rephrased the query when it was not. It then generated a cited answer with Qwen3.6-27B. Qdrant handled hybrid search, applied payload filters, and ran late-interaction reranking. Minima served each request on a single 96 GB NVIDIA RTX PRO 6000 Blackwell GPU.

Across 1,800 evaluated tasks and 10,000 full agent episodes, the joint stack reached 3,750 tasks per GPU-hour, compared to 1,350 for dense retrieval with BF16 inference. Median task latency fell from 21.3 seconds to 7.7 seconds, grounded task success rose from 80.1% to 84.2%, and successful throughput climbed from 1,081 to 3,158 tasks per GPU-hour.

What We Tested

Minima ran 1,800 multi-step tasks across three public benchmarks, SciFact, FiQA, and HotpotQA, plus a fourth set that Minima built to test payload filtering. In that fourth set, every chunk carries a tenant ID and a document version in its Qdrant payload, and every query has exactly one correct tenant-and-version slice. Any result returned from outside that slice counts as a violation, so the set scores pass or fail with no grader judgment involved. This is the set behind the 50,000-query tenant-policy test reported later in this post. Every run used the same agent prompt, tool schema, stopping rule, and a maximum of two Qdrant calls per episode. Answers were capped at 256 output tokens. Minima then replayed 10,000 complete episodes against one million 400-token chunks and ran a separate 50,000-query adversarial filtering test for each retrieval condition.

All three configurations used the same hardware, Qwen3.6-27B checkpoint, sampling settings, agent prompt template, concurrency, and endpoint. Qdrant ran on its own host. Query encoding ran on a separate CPU-only FastEmbed service, on ONNX Runtime, using the same models in all three configurations: sentence-transformers/all-MiniLM-L6-v2 for 384-dimensional dense vectors with cosine distance, qdrant/bm25 for sparse vectors, and answerdotai/answerai-colbert-small-v1 for 96-dimensional late-interaction multivectors. Qdrant stored and searched the sparse vectors and applied its IDF modifier. No GPU did any encoding work, so the RTX PRO 6000 Blackwell was dedicated to the Minima Qwen3.6-27B endpoint. Episode latency and task throughput were measured end to end, including encoding and retrieval, while the GPU-hour and GPU-cost figures count only that dedicated inference GPU. Retrieval strategy and Minima compression were the only variables that changed between runs. Minima accepted a configuration only if grounded task quality stayed within 1 percentage point of BF16 with retrieval fixed, citation quality held, and at least 99.5% of episodes completed. Query vectors were precomputed only for the isolated Qdrant latency measurement. The end-to-end agent run included planning, embedding, retrieval, evidence checking, and generation.

Why the First Qdrant Call Was Usually Enough

Dense retrieval handled semantic similarity well. Minima added BM25 to recover the names, IDs, and version strings that embeddings can miss. Qdrant fused the two result sets with reciprocal rank fusion (RRF), then reranked the shortlist with token-level late interaction. Payload filters enforced tenant, language, document type, and version at query time.

The numbers below cover retrieval and the agent loop. Recall@10 and nDCG@10 use the same top-10 ranked evaluation list. The agent prompt was then truncated to the stated dense top 16 or hybrid top 8 context budget. “First-pass evidence sufficient” means the agent did not invoke its optional second search. Per-call latency is warmed Qdrant query time with query vectors precomputed.

Qdrant’s p95 query time was 43.2 milliseconds, less than 0.3% of the 20.8-second p95 BF16 agent episode. The first search was sufficient in 87% of tasks, and mean context fell from 5.2K to 2.3K tokens. Even before Minima was enabled, median task latency dropped from 21.3 to 14.6 seconds and successful throughput rose from 1,081 to 1,669 tasks per GPU-hour, a 54% gain.

“Inside an agent loop, a weak first retrieval costs more than one extra search. It triggers another round of planning, retrieval and inference, so getting sufficient evidence on the first pass is one of the simplest ways to make the whole system faster and more efficient.” — Sergii Kozyrev, Co-founder and CEO, Minima AI

How Minima Accelerated Every Model Call

Minima stored Qwen3.6-27B weights in NVFP4 W4A4 and ran them with native Blackwell kernels. Recent and anchor KV stayed in FP8. Stale pages moved to the 3-bit TQ3 tier. Minima disabled Qdrant vector quantization so the retrieval and LLM inference effects stayed separate.

Compression held task quality. With Qdrant retrieval fixed, grounded task success was 84.3% for BF16 and 84.2% for Minima, citation F1 was 90.8% and 90.7%, and valid tool calls were 99.8% for both. The paired task-quality delta was -0.1 percentage point (95% CI [-0.7, +0.5]), which cleared the pre-registered non-inferiority gate.

The Joint Result: 2.92x More Successful Tasks per GPU

With BF16 unchanged, Qdrant raised raw capacity from 1,350 to 1,980 tasks per GPU-hour. Holding Qdrant fixed, Minima raised it to 3,750. Applying the grounded task success rate gives 3,158 successful tasks per GPU-hour, 2.92x the baseline.

The table below reports agent results at concurrency 8, with final answers capped at 256 tokens. Task rates are wall-clock completions per GPU-hour.

At the rate of USD 1.50 per GPU-hour that Minima used for test accounting, GPU cost per 1,000 successful agent tasks fell from USD 1.39 to USD 0.48, a 65% reduction. This GPU-only comparison excludes the Qdrant host and embedding services. The same provisioned services stayed online across all three conditions, though the hybrid pipeline put more work on them.

Minima did not multiply the 3.2x weight compression, 3.5x KV compression, and smaller retrieval context into a single system claim. They affect different bottlenecks. The measured end-to-end results were 2.78x more raw task capacity and 2.92x more successful tasks per GPU-hour.

Why This Matters for Agentic RAG

Qdrant and Minima address different costs inside the loop. Qdrant made the first search sufficient more often and reduced the evidence passed to the model on each attempt. Minima reduced the memory and compute cost of planning, checking, and answering.

A production agent is limited by the whole run, not by vector search or model throughput in isolation. In this test, Qdrant improved the evidence passed to the model and Minima increased the amount of inference one GPU could serve. Together they delivered 2.92x more successful tasks without reducing tool-call validity, grounded quality, or citation quality.

Reproduce This on Your Corpus

If you run agentic RAG on Qdrant, Qdrant and Minima would like to reproduce this benchmark on your corpus and agent loop. Contact Qdrant or contact Minima to get started.

Technical References

The Qdrant guide to agentic vector search explains why retrieval latency, memory, filtering, and reranking matter inside multi-step agent workflows.

The agentic RAG with LangGraph and Qdrant tutorial covers tool selection, repeated retrieval, and stateful agent control flow.

The Qdrant hybrid and multi-stage queries documentation describes dense and sparse prefetch, RRF and DBSF fusion, and multi-stage ranking.

The Qdrant hybrid search with reranking tutorial walks through the dense, sparse, and ColBERT-style late-interaction workflow.

The Qdrant multivectors and late interaction tutorial covers native multivector representations and MaxSim scoring.

The Qdrant filtering documentation describes payload and point-ID conditions for application-defined constraints.

The NVIDIA RTX PRO 6000 Blackwell product page lists the 96 GB GDDR7 memory and Blackwell FP4 support.

The Minima site covers model-weight, KV-cache, and serving optimization.