Measuring Retrieval Relevance

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Measuring Retrieval Relevance

Time: 40 min	Level: Intermediate

This tutorial focuses on retrieval relevance: how well retrieved results match real user intent. To measure retrieval relevance, you need a labeled dataset of queries paired with their expected relevant documents (commonly called a golden query set or ground truth). This tutorial covers both building that dataset and running it through Qdrant to compute relevance metrics.

Two related tutorials cover the other retrieval-evaluation concerns: Measuring ANN Recall (does the approximate index match exact kNN?) and Evaluating Pipeline Output Quality (does the end-to-end pipeline produce the right output?).

Evaluating Pipeline Output Quality

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Evaluating Pipeline Output Quality

Time: 45 min	Level: Intermediate

This tutorial focuses on pipeline output quality: whether the full retrieval pipeline produces the right output once retrieved results reach a consumer, most often an LLM generator in a RAG system. To measure pipeline output quality, you run your golden set through the full pipeline, capture each (question, retrieved_context, answer) triple, and score the triples against judgment metrics like faithfulness, answer relevancy, and context precision.

Improve Search on Qdrant - Vector Search Engine

Measuring Retrieval Relevance

Measuring Retrieval Relevance

Evaluating Pipeline Output Quality

Evaluating Pipeline Output Quality