Edge API on Qdrant - Vector Search Engine

Shard Lifecycle

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Shard Lifecycle

An Edge Shard is backed by a directory on local disk. The lifecycle of a shard is:

Create a new shard with EdgeShard.create (Python) or EdgeShard::new (Rust), or load an existing one with load.
Use the shard to update and query data.
Flush pending changes to disk, and close the shard to release its resources.

Because the shard owns the files in its directory, only one EdgeShard may be open on a given directory at a time.

Configuration

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Configuration

EdgeConfig describes the vectors an Edge Shard stores and the parameters that govern how it indexes, stores, and searches them. Pass it to create/new when starting a new shard, and optionally to load when reopening one.

Every parameter except vectors and sparse_vectors is optional. A parameter left unset is considered as not specified rather than set to the default: when loading an existing shard, each unspecified parameter resolves through provided - persisted in edge_config.json - derived from the existing segments - default, so it keeps whatever the shard already has. This is why a configuration that sets only wal_options leaves the rest of the shard’s configuration untouched.

Updating Data

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Updating Data

Every write to an Edge Shard goes through a single method, update, which takes one UpdateOperation describing what to change. The operation is written to the write-ahead log before it is applied to storage, so an update that has returned survives a crash.

update

def update(self, operation: UpdateOperation) -> None

pub fn update(&self, operation: UpdateOperation) -> OperationResult<()>

Parameter	Description
`operation`	The operation to apply.

Returns nothing in Python. In Rust, returns Ok(()) on success.

Reading Data

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Reading Data

An Edge Shard offers several ways to read data. query is the similarity search entry point, supporting prefetches, fusion, and reranking. scroll pages through points without scoring them, and retrieve, count, and facet read points and payload statistics directly.

In Rust, query_groups and search_matrix require the EdgeShardRead trait in scope: use qdrant_edge::EdgeShardRead;

query

Runs a query, optionally combining the results of nested prefetch queries.

def query(self, query: QueryRequest) -> List[ScoredPoint]

pub fn query(&self, request: QueryRequest) -> OperationResult<Vec<ScoredPoint>>

Returns a list of ScoredPoint, ordered by score.

Snapshots

info@qdrant.tech (Andrey Vasnetsov) — Mon, 01 Jan 0001 00:00:00 +0000

Snapshots

Snapshots move data between an Edge Shard and a Qdrant server collection. This section covers the methods; for how to combine them, refer to Data Synchronization Patterns.

unpack_snapshot

Unpacks a snapshot archive on disk so it can be loaded as a shard. A static method in Python and an associated function in Rust, so it needs no shard instance.

@staticmethod
def unpack_snapshot(snapshot_path: str, target_path: str) -> None

pub fn unpack_snapshot(snapshot_path: &Path, target_path: &Path) -> OperationResult<()>

Parameter	Description
`snapshot_path`	Path to the downloaded snapshot file.
`target_path`	Directory to unpack into.

After unpacking, open the directory with load. The resulting shard keeps the configuration and file layout of the collection the snapshot came from, including its vector and payload indexes.