Module 5: Multimodal Supplier Risk Intelligence on Qdrant - Vector Search Engine

Module Overview

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

Multimodal Supplier Risk Intelligence

Apply every concept from Modules 1 through 4 in a single end-to-end system: ingest daily news, transcripts, and satellite imagery about your suppliers, cluster them into risk themes, and query all of it from one collection.

Follow-along code: Module 5 notebook

Overview

Module 4 turned the building blocks into a design. In this module, you’ll build that design into a working system. You’ll explore named vectors that hold text and image evidence on a single point, then see how clustering groups those signals into the events they describe. You’ll also learn how to search images with text and extend the system across languages. By the end, you’ll have ingested, clustered, and queried multimodal signals from one collection.

Project Overview

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

Project Overview

A factory fire at a supplier’s plant reaches you four ways. A local news report, a satellite image, an earnings call where an executive gets asked about it, and the supplier’s own filing weeks later. You are building the system an analyst uses to see all four, and none of them arrives labeled as an incident.

This is the news search system you designed in Module 4, extended in three ways:

System Architecture

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

System Architecture

The system has four stages. Each maps to Qdrant primitives you already know.

Ingest: collect the day’s signals from news APIs, image feeds, and transcript files. Chunk anything longer than a paragraph.
Embed: hand each part of a signal to the model for its modality, producing named vectors: text_dense, text_sparse, image.
Store: upsert each signal as one PointStruct carrying every vector it has, plus a payload: supplier, source type, country, publication date, risk score.
Cluster and Query: a daily batch tags signals with a cluster_id; on demand, analysts run hybrid and image queries against the same collection.

Signal Sources and Embedding Models

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

Signal Sources and Embedding Models

Two models cover every signal here, and both run through FastEmbed exactly as in Modules 3 and 4: name the model, pass the content, and the client embeds it locally before upload.

Signal source	Modality	Embedding model	Vectors it produces
News articles	text	all-MiniLM-L6-v2 and Qdrant/bm25	`text_dense`, `text_sparse`
Financial filings	text	all-MiniLM-L6-v2 and Qdrant/bm25	`text_dense`, `text_sparse`
Earnings-call transcripts	text, transcribed	all-MiniLM-L6-v2 and Qdrant/bm25	`text_dense`, `text_sparse`
Satellite imagery	image and caption	clip-ViT-B-32-vision, and the two text models on the caption	`image`, `text_dense`, `text_sparse`

Satellite captures are the row worth reading twice. The caption is what gives an image its text vectors, and Clustering Risk Signals depends on those: an uncaptioned image can never join a text cluster.

Ingestion Pipeline

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

Ingestion Pipeline

The daily job collects signals, embeds each modality, and upserts them. A risk scoring step assigns an initial risk_score, which analysts later filter on.

Risk scoring is a keyword baseline, deliberately simple, and the first thing to replace once you have labeled signals of your own:

import re

# Highest-weighted term wins, so one mention of "fire" outranks three of "delay".
RISK_TERMS = {
 "fire": 0.9, "explosion": 0.9, "halted": 0.8, "shutdown": 0.8,
 "recall": 0.7, "strike": 0.7, "flood": 0.7,
 "investigation": 0.5, "shortage": 0.5, "delay": 0.5,
 "backlog": 0.4, "inspection": 0.4,
}

def score_risk(text: str) -> float:
 """A baseline to beat, not a model. Returns 0.0 when nothing matches."""
 lowered = text.lower()
 return max(
 (weight for term, weight in RISK_TERMS.items()
 # \b stops "fire" matching "firearm" and "strike" matching "striking"
 if re.search(rf"\b{term}\b", lowered)),
 default=0.0,
 )

Collection Setup

import uuid
from qdrant_client import QdrantClient, models

client = QdrantClient(
 url="https://YOUR-CLUSTER.cloud.qdrant.io",
 api_key="YOUR_API_KEY",
)

client.create_collection(
 collection_name="supplier_signals",
 vectors_config={
 "text_dense": models.VectorParams(size=384, distance=models.Distance.COSINE),
 "image": models.VectorParams(size=512, distance=models.Distance.COSINE),
 },
 sparse_vectors_config={
 "text_sparse": models.SparseVectorParams(
 modifier=models.Modifier.IDF # required for BM25 scoring, as in Module 4
 ),
 },
)

# Every field an analyst filters on gets an index, and all of them are created
# before ingestion so the HNSW graph picks up its filter-aware edges (Module 4).
for field in ["supplier_id", "source_type", "language", "country", "facility_id"]:
 client.create_payload_index(
 collection_name="supplier_signals",
 field_name=field,
 field_schema=models.PayloadSchemaType.KEYWORD,
 )

client.create_payload_index(
 collection_name="supplier_signals",
 field_name="published_at",
 field_schema=models.PayloadSchemaType.DATETIME,
)
client.create_payload_index(
 collection_name="supplier_signals",
 field_name="risk_score",
 field_schema=models.PayloadSchemaType.FLOAT,
)
client.create_payload_index(
 collection_name="supplier_signals",
 field_name="cluster_id", # values arrive after clustering, index now anyway
 field_schema=models.PayloadSchemaType.INTEGER,
)

risk_score and cluster_id are easy to forget here, because nothing filters on them until Clustering Risk Signals and Analyst Queries. Skip them and the analyst query doesn’t just run slowly: Qdrant Cloud enables strict mode by default, so a query filtering an unindexed field is rejected outright.

Clustering Risk Signals

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

Clustering Risk Signals

Clustering groups signals that describe the same underlying event, even when they arrive from different sources. A factory fire appears in a local news article, a captioned satellite image, and an earnings call answer. Because Ingestion Pipeline put a text_dense vector on all three, clustering can surface them as one event.

Qdrant uses the word for one other thing: a Qdrant Cloud cluster is the deployment that holds your collection. The clustering in this section runs in your own code, over the vectors already stored there.

Analyst Queries

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

Analyst Queries

One collection, three named vectors. Two ways to ask here, plus the centroid query from Clustering Risk Signals.

Searching Images With Text

The satellite signals are searchable by what they show, with no caption needed at query time. The query text goes through CLIP’s text encoder so it lands in the image vector space:

def search_facility_images(query_text: str, supplier_id: str, limit: int = 10):
 return client.query_points(
 collection_name="supplier_signals",
 query=models.Document(text=query_text, model=IMAGE_TEXT_MODEL),
 using="image",
 query_filter=models.Filter(
 must=[models.FieldCondition(
 key="supplier_id", match=models.MatchValue(value=supplier_id),
 )]
 ),
 limit=limit,
 with_payload=True,
 )

smoke = search_facility_images("smoke above factory roof", supplier_id="SUP-7291")

Each named vector is its own space, so the query has to be embedded by the model that produced the vectors it is searching. Swap using="text_dense" and DENSE_MODEL and the same call searches article text instead.

Knowledge Check

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

Knowledge Check

Work through these before you call the capstone done.

Why does the collection use named vectors instead of one collection per modality?

One signal, one point. A single event can carry text and image evidence at the same time, and named vectors keep all of it on that one point, queryable separately, sharing a single payload for filtering. Splitting by modality would scatter one event across collections, duplicate the filtering logic, and leave you joining results in application code.

Course Summary

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

Course Summary

This module completes the Qdrant Beginners course. Here’s what was covered:

Module	Theme	Key concepts covered
Module 1	Let’s Understand Search	Why keyword search fails; how embeddings and semantic search work; the shift from words to meaning.
Module 2	First Principles of Vector Search	Collections, points, vectors, payloads, HNSW, chunking strategies, and the full ingestion pipeline.
Module 3	Sparse vs Dense vs Hybrid Search	BM25 against embeddings; when each fails; hybrid search with rank fusion.
Module 4	Designing a Vector Search System	The five layers of the stack; what to decide before ingesting; what changes as the collection grows; when to add machines; where generation fits; where to run Qdrant.
Module 5	Multimodal Supplier Risk Intelligence	End-to-end capstone: ingest news, transcripts, and images on shared points; cluster risk signals; query every modality.
Module 6	Beyond Similarity (Bonus)	Optional further reading: score boosting, MMR diversity, two-stage reranking, grouping, relevance feedback, and discovery.

Next, get #QdrantCertified with the official Beginners exam, which covers Modules 1 through 5.

References and Further Reading

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Module 5

References and Further Reading

Named Vectors: declaring more than one vector per point and querying a named one with using.
Hybrid Queries: prefetch semantics, Reciprocal Rank Fusion with weights, Distribution-Based Score Fusion, and formula queries.
Indexing and Filterable HNSW: payload index types, why indexes come before ingestion, and the IDF modifier that BM25 scoring needs.
Filtering: full filter syntax used throughout the capstone, including MatchAny and datetime ranges.
Bulk Upload: batch sizes and index ordering for the daily ingestion job.
FastEmbed: the local embedding path behind models.Document and models.Image, and every model name it accepts.
Multimodal and Multilingual Search: a Cohere Embed 4.0 tutorial building retrieval over images and text in a shared embedding space.
multilingual-e5-large: the multilingual swap from Analyst Queries, with its 100 languages, 1024 dimensions, and required query and passage prefixes.
CLIP ViT-B/32: model card for the image model behind Qdrant/clip-ViT-B-32-vision and its text counterpart.