An embedding model turns text into a vector so that similar meanings land close together. Contextator runs one locally on your CPU by default — no API key, no account, nothing leaving the machine — and can use OpenAI instead if you prefer.
The default
| Model | Xenova/multilingual-e5-small |
| Dimensions | 384 |
| Languages | 100, including Turkish. Ask in the language the answer is written in: matching across languages is a measured limit of the model, not a switch |
| Download | ~470 MB once (fp32), cached in a Docker volume |
| Runs on | CPU. No GPU is used or needed |
Smaller and faster options
EMBEDDING_DTYPE=q8 # same model, ~120 MB, quantized
EMBEDDING_MODEL=Xenova/all-MiniLM-L6-v2 # English-only, ~90 MB, also 384 dimensions
EMBEDDING_DIMENSIONS=384
Any transformers.js feature-extraction model works. EMBEDDING_DIMENSIONS must match what the model
produces — if it does not, the server tells you the correct number.
Using OpenAI instead
EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=sk-…
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
EMBEDDING_DIMENSIONS=1536
RESET_VECTORS=1 # only for the first start after the change — see below
Better retrieval on long chunks and a much larger input window, at the cost of an API call per chunk indexed and per query asked. Your documentation text is sent to OpenAI; if that is not acceptable, stay local, or use a server of your own, below.
Using a server of your own
EMBEDDING_PROVIDER=openai speaks to any server that offers an OpenAI-compatible /embeddings
endpoint. Set EMBEDDING_BASE_URL to it, with the path, and name the model that server serves in
OPENAI_EMBEDDING_MODEL. OPENAI_API_KEY is optional with a base URL; when set, it is sent as a bearer
token. EMBEDDING_DIMENSIONS must match the model’s vector size.
# Ollama
EMBEDDING_PROVIDER=openai
EMBEDDING_BASE_URL=http://localhost:11434/v1
OPENAI_EMBEDDING_MODEL=nomic-embed-text
EMBEDDING_DIMENSIONS=768
# LM Studio: its local server
EMBEDDING_BASE_URL=http://localhost:1234/v1
# Text Embeddings Inference, started with -p 8080:80
EMBEDDING_BASE_URL=http://localhost:8080/v1
# vLLM: the host and port you started `vllm serve` on
EMBEDDING_BASE_URL=http://gpu-box:<port>/v1
EMBEDDING_REQUEST_DIMENSIONS=always # only for a model that shortens its vectors on request
Set the model name and EMBEDDING_DIMENSIONS to the model you loaded in every case; the lines above show
only what differs. RESET_VECTORS=1 is needed only when the database already records a vector dimension and the new one differs from it, as moving a 384-dimension install to a 768-dimension model does; a first install needs none. Set it for one start only — see
Changing the model. EMBEDDING_REQUEST_DIMENSIONS is auto by default and sends dimensions only for
text-embedding-3-*; use always for a model that accepts it and never for a server that rejects it.
Counting tokens with the model’s own tokenizer. Chunks are sized by an estimate (characters ÷ 4) unless
EMBEDDING_TOKENIZER names a Hugging Face model whose tokenizer.json matches the served model. Only the tokenizer is
downloaded, into MODEL_CACHE_DIR, and startup stops if it cannot be loaded. With it, an endpoint cuts the same chunks
the local provider cuts from the same model. It is part of the model id, so setting or changing it re-indexes every
project once.
# multilingual-e5-small behind TEI or vLLM
EMBEDDING_PROVIDER=openai
EMBEDDING_BASE_URL=http://localhost:8080/v1 # TEI; vLLM: http://gpu-box:<port>/v1
OPENAI_EMBEDDING_MODEL=intfloat/multilingual-e5-small # the real name, so the e5 prefixes apply
EMBEDDING_DIMENSIONS=384
EMBEDDING_TOKENIZER=Xenova/multilingual-e5-small
The model’s name decides the prefixes. The query: and passage: prefixes of the multilingual-e5-* family
are chosen from the name in OPENAI_EMBEDDING_MODEL, so the model’s real name has to be there. If your server serves the model under another name (a local alias, a path), the name matches no family: the model runs without prefixes and recall quietly drops. The server says so once at startup, at info level, unless you have set either prefix variable. Either put the model’s real name in OPENAI_EMBEDDING_MODEL, or keep your own name and set EMBEDDING_QUERY_PREFIX="query: " and EMBEDDING_PASSAGE_PREFIX="passage: " yourself; either change re-indexes every project once.
Inside the Contextator container, localhost is the container, not your machine: use the address the
container can reach the server on. The endpoint’s host and port become part of the model id every project
is stamped with, so pointing at another server re-indexes every project — see
Changing the model. Text sent to such a server stays wherever that server runs.
Changing the model
Same dimensions (e.g. between the 384-dimension local models)
- Change
EMBEDDING_MODEL. - Restart.
- Re-index each project.
Contextator records which model indexed each project. When the server’s model differs, the next run is
automatically a full re-index, and search_docs refuses to search that project until then — with a
message saying exactly that, rather than returning confidently wrong results.
Different dimensions (e.g. moving to OpenAI’s 1536)
The vector column’s width is fixed when the database is created, so this is a one-time destructive step:
- Set the provider, key and
EMBEDDING_DIMENSIONS. - Start once with
RESET_VECTORS=1. Every chunk and document is dropped and the column is re-typed. - Remove
RESET_VECTORS(or set it back to0). - Re-index every project.
Without the flag, the server refuses to start and prints this instruction:
The database was created with EMBEDDING_DIMENSIONS=384 but the current config says 1536.
Either set EMBEDDING_DIMENSIONS=384, or start once with RESET_VECTORS=1
(drops every indexed chunk; all projects must be re-indexed), or wipe the pgdata volume.
How search actually works
Search is hybrid: meaning and exact wording are searched at once and fused by rank.
- Your question is embedded with the same model that indexed the documents.
- PostgreSQL runs two rankings scoped to that project — nearest chunks by cosine similarity over an
HNSW index, and a
tsvectorkeyword match over a GIN index — and combines them with reciprocal rank fusion, so an environment variable finds its page as readily as a question does. - The top matches are returned with their file path, heading breadcrumb and score.
Scores are cosine similarities: higher is better, and with the default model they sit high and close
together. On the shipped configuration the mean score is 0.881, and a wrong hit sits only about 0.02
below a correct one. A score is therefore not a truth signal, and nothing in the product
reads one to decide ranking or correctness; only the off-domain floor below looks at it.
The relevance floor, SEARCH_SCORE_FLOOR=0.82 by default, was measured against something else
entirely: questions with nothing to do with your documentation. On the committed set of twelve
such questions it refuses ten, and it refuses none of the sixty-four golden questions. That is the
whole of what the floor claims: it catches most off-domain questions and costs an answerable one
nothing. It does not separate the two classes — the highest off-domain top hit (0.839) sits above
the lowest top hit of a real question (0.833), so no threshold does. Nor does it catch a question
shaped like your product whose answer is simply not written down — those score exactly where real
questions score. Below the floor
search_docs answers no good match rather than handing over a hit an agent would cite.
Scores are comparable within one project and one model, and meaningless across different models — change
EMBEDDING_MODEL and the floor has to be re-measured with npm run eval.
SEARCH_SCORE_FLOOR (default 0.82) is the server’s floor. A project can set its own from the query
panel, or turn the floor off for itself, when its corpus scores differently — prose-heavy corpora usually
want a lower one. The panel shows what a floor would have done to the searches already logged before it
is applied, and the server’s SEARCH_SCORE_FLOOR=0 still turns every project’s floor off. See
Configuration.
Ranks are fused, not scores. Cosine similarity and PostgreSQL’s ts_rank are not comparable
quantities, so the fusion combines the rank each chunk held on each side — Σ 1/(60 + rank) over
whichever lists it appeared on (reciprocal rank fusion) — rather than a weighted sum that would have to
be re-learned every time the embedding model changed. The consequence: a result’s score no longer
explains its position — one scoring 0.86 can sit above one scoring 0.88. GET /api/projects/:id/search
and the dashboard show the rank each result held on each side as D3 L1; a result with an L and no D
is an identifier the vector search alone could not see. On the golden set, rank fusion took the thirty
identifier-shaped questions from recall@5 76.7 % to 83.3 % and did not move the thirty-four
natural-language questions at all; recall@1 fell from 78.1 % to 75.0 %, the trade fusion makes — a
chunk found by one half alone cannot outrank one found respectably by both.
The default model is asymmetric: it was trained with query: in front of a search query and
passage: in front of an indexed passage, so the server adds them automatically — nothing you index or
search ever mentions them, and a model without prefixes encodes a query and a passage identically at no
cost. On the retrieval corpus the prefixes were worth one question at rank 1 and nothing at recall@5;
EMBEDDING_QUERY_PREFIX=none with EMBEDDING_PASSAGE_PREFIX=none turns them off and restores the
previous model id exactly.
The keyword half indexes each source in PostgreSQL’s simple configuration by default — words as
written, no stemming — which is the right choice for reference material: HALYARD_DISPATCH_TIMEOUT and
HLY-4015 survive into the index as the strings they are. A source can instead name its language (the
Language field on the source form), which switches it to a stemmed configuration — a Turkish question
asking about anahtarı then finds a page that says anahtarın, two strings simple treats as unrelated.
The two live in one project without interfering: each source is searched in its own configuration and a
single query reaches all of them, each contributing its own ranked list to the fusion. Changing a
source’s language re-indexes it.
Cross-lingual search is a known limit, and it is not being fixed
Write the question in the same language as the documentation that should answer it. Turkish pages
surface for phrasing shaped like Turkish. English pages surface for phrasing shaped like English.
Crossing that boundary mostly returns nothing, and this is a property of the embedding model rather than
a setting to turn on. Every MCP client is a language model, and list_topics names a source’s language
while instructions directs the calling agent to write its search_docs query to match — a direction to
the caller, not a change to search itself.
The measurement is in eval/BASELINE.md
and it is not close: thirty questions were written from the corpus, each about a page in the language the
question was not phrased in — fifteen per direction. Four are answered in the top five. Twenty-seven of
the thirty instead return a page in the phrasing’s own language at rank 1: the model is not failing to
understand what was asked, it is ranking the language of the phrasing above the answer to it. Both
directions fail equally.
Identifiers are the exception. HALYARD_DISPATCH_TIMEOUT, HLY-4015, X-Halyard-Signature are the
same string in both languages, so the keyword half of search finds them regardless. Of the nine
cross-lingual questions in the set that name an identifier, four are answered; of the twenty-one phrased
as ordinary questions, none is.
What to do instead of waiting for a fix. Keep each language in its own source and let an agent scope
its search with source, path_prefix or version — two sources that each answer well beat one
collection that answers either language badly. Both known remedies have already been measured: hybrid
search (above) recovered cross-lingual identifier retrieval and no natural-language question; a
multilingual cross-encoder rerank behind SEARCH_RERANK took cross-lingual recall@5 from 13.3 % to
33.3 % — still below the 42.9 % the embedding model before this one managed — while costing thirteen
ordinary questions their rank-1 answer and a search’s latency going from 12 ms to 1.2 s. It is off by
default and should stay off. A second, translation-trained encoder as an additional index would actually
fix this — roughly four times the download, a second vector column, a full re-index on every
installation — and that price is not worth paying for a documentation server. If cross-language search
is essential to your corpus, that is a reason to choose a different tool rather than a reason to wait for
this one.
Air-gapped installations
The only outbound request a default installation makes is the one-time model download. To remove even that:
- Pre-populate the models volume (
/app/.cache/models) with the model files — for example by running once on a connected host and copying the volume. Do it with the sameEMBEDDING_MODELandEMBEDDING_DTYPEthe air-gapped host will use. - Set
EMBEDDING_OFFLINE=1.
The server will then never attempt a download, and will fail loudly if the model is missing.
The reranker
The optional reranker (SEARCH_RERANK=on, off by default) is a second model, SEARCH_RERANK_MODEL
(Xenova/bge-reranker-base by default), loaded into the same cache directory and governed by the same
EMBEDDING_OFFLINE. With the reranker on and the flag set, its files must already be in the volume too:
warm the cache on the connected host with SEARCH_RERANK=on and the same SEARCH_RERANK_MODEL and
SEARCH_RERANK_DTYPE before you copy it.
Moving the image and the models
On a connected host, pull the image and write it to a file, carry the file across, and load it:
docker pull contextator/contextator:<tag>
docker save --output contextator.tar contextator/contextator:<tag>
# on the air-gapped host
docker load --input contextator.tar
Use the tag you run, not latest, so the two hosts agree on the version. The models volume is not part of
the image: copy it separately (the default volume is contextator-models), or mount a directory with
CONTEXTATOR_MODELS_PATH.
What still leaves the host
| Source type or feature | Outbound request |
|---|---|
Embedding, default local model |
None once the cache is filled; otherwise the one-time model download from the Hugging Face Hub |
Tokenizer named by EMBEDDING_TOKENIZER (with EMBEDDING_PROVIDER=openai) |
Downloaded from the Hugging Face Hub unless the cache already holds it; subject to EMBEDDING_OFFLINE, and startup stops if it cannot be found |
| Reranker | None once the cache is filled; otherwise its one-time model download from the Hugging Face Hub |
Embedding with EMBEDDING_PROVIDER=openai |
Document text and queries, to EMBEDDING_BASE_URL (OpenAI unless you changed it) |
| Local directory, upload | None |
| Git repository | Your git host, over HTTP(S), on each sync and on a webhook-triggered one |
| Notion | Notion’s API, on each sync |
| Confluence | Your Confluence instance, on each sync |
| Documentation site | The site you pointed it at, plus its robots.txt |
Remote sources are fetched from this server, not from the browser that set them up, so an air-gapped host that needs one of them needs a route to that host and no other.
Performance notes
- Embedding is CPU-bound and is the slow part of indexing; that is why one project is indexed at a time.
EMBEDDING_BATCH_SIZE(default 16) controls how many chunks are embedded per call.- A query embeds exactly one short text, so search latency is dominated by the database, not the model.
- The model is loaded in the background at startup — the dashboard works while it loads, but indexing and search wait for it.