Ollama

Ollama downloads open embedding models and serves them from your own machine through a local HTTP API on port 11434. Your application sends text to Ollama’s /api/embed endpoint and stores the returned vectors in Qdrant, so the text never leaves your infrastructure.

Models

Ollama lists its embedding models at ollama.com/search?c=embedding. These are good starting points:

Model	Dimensions	Max Tokens	Download Size
`qwen3-embedding:0.6b`	1024 (reducible to 32)	32,768	639 MB
`qwen3-embedding:4b`	2560 (reducible to 32)	32,768	2.5 GB
`qwen3-embedding:8b` (the default `qwen3-embedding` tag)	4096 (reducible to 32)	32,768	4.7 GB
`embeddinggemma-2:270m`	768 (or 512, 256, 128)	8,192	304 MB
`nomic-embed-text-v2-moe`	768 (reducible to 256)	512	958 MB
`bge-m3`	1024	8,192	1.2 GB

nomic-embed-text, mxbai-embed-large, all-minilm, and embeddinggemma are earlier models. For new projects, use a model from the Models table.

/api/embed returns vectors normalized to length 1, so the Cosine and Dot distance metrics rank results identically. Use Cosine.

Setup

Install Ollama and pull the model:

ollama pull qwen3-embedding:0.6b

Install the Ollama SDK and the Qdrant client:

Python: pip install ollama qdrant-client
TypeScript: npm install ollama @qdrant/js-client-rest

Create both clients and a collection sized for the model. The Python client connects to the address in the OLLAMA_HOST environment variable, or to http://localhost:11434 if it’s not set. The JavaScript client connects to http://localhost:11434.

import ollama
from qdrant_client import QdrantClient, models

ollama_client = ollama.Client()
client = QdrantClient(url="http://localhost:6333")

MODEL = "qwen3-embedding:0.6b"

client.create_collection(
    collection_name="{collection_name}",
    vectors_config=models.VectorParams(size=1024, distance=models.Distance.COSINE),
)

import ollama from "ollama";
import { QdrantClient } from "@qdrant/js-client-rest";

const client = new QdrantClient({ url: "http://localhost:6333" });

const MODEL = "qwen3-embedding:0.6b";

await client.createCollection("{collection_name}", {
  vectors: { size: 1024, distance: "Cosine" },
});

Embed Documents

embed takes a list of inputs and returns one vector per input, in the same order. This example uses the position of each input as the point ID:

documents = [
    "Qdrant is a vector search engine.",
    "Ollama runs open models on your own hardware.",
]

response = ollama_client.embed(model=MODEL, input=documents)

client.upsert(
    collection_name="{collection_name}",
    points=[
        models.PointStruct(id=idx, vector=list(vector), payload={"text": documents[idx]})
        for idx, vector in enumerate(response.embeddings)
    ],
)

const documents = [
  "Qdrant is a vector search engine.",
  "Ollama runs open models on your own hardware.",
];

const response = await ollama.embed({ model: MODEL, input: documents });

await client.upsert("{collection_name}", {
  points: response.embeddings.map((vector, idx) => ({
    id: idx,
    vector,
    payload: { text: documents[idx] },
  })),
});

Ollama truncates inputs that are longer than the model’s context window. Pass truncate=False to get an error instead. Older code calls embeddings and the /api/embeddings endpoint, which take a single prompt. That method is deprecated in favor of embed.

Embed Queries

Ollama sends text to the model as is. Qwen3 Embedding expects an instruction in front of each query, and none in front of documents:

TASK = "Given a web search query, retrieve relevant passages that answer the query"

query_vector = ollama_client.embed(
    model=MODEL,
    input=f"Instruct: {TASK}\nQuery:What is Qdrant?",
).embeddings[0]

hits = client.query_points(
    collection_name="{collection_name}",
    query=list(query_vector),
    limit=3,
).points

const TASK =
  "Given a web search query, retrieve relevant passages that answer the query";

const queryResponse = await ollama.embed({
  model: MODEL,
  input: `Instruct: ${TASK}\nQuery:What is Qdrant?`,
});

const { points: hits } = await client.query("{collection_name}", {
  query: queryResponse.embeddings[0],
  limit: 3,
});

Other models use different prompts. Add them in your code, because Ollama does not:

Model	Query Prompt	Document Prompt
`qwen3-embedding`	`Instruct: {task}\nQuery:{query}`	None
`embeddinggemma-2`	`task: search result \| query: {query}`	`title: none \| text: {document}`
`nomic-embed-text-v2-moe`	`search_query: {query}`	`search_document: {document}`
`bge-m3`	None	None

Reduce Vector Size

embed accepts a dimensions parameter. Ollama keeps the leading values of each vector and normalizes the result again. Use it only with models trained for shortened vectors, such as qwen3-embedding, embeddinggemma-2, and nomic-embed-text-v2-moe. Set the collection’s size to the same value, and pass the same dimensions when you embed queries:

response = ollama_client.embed(model=MODEL, input=documents, dimensions=256)

const reduced = await ollama.embed({
  model: MODEL,
  input: documents,
  dimensions: 256,
});

Next Steps

Read the Ollama embeddings guide and the /api/embed reference.
Filter search results by payload fields.
Combine Ollama vectors with sparse vectors for hybrid search.
Reduce memory use with quantization.

Ollama

Models

Setup

Embed Documents

Embed Queries

Reduce Vector Size

Store and Search the Vectors

Create a Collection

Upload Vectors

Search

Next Steps

Was this page useful?