Contextator
ENTR

Operating

Embedding Models

The default local model, switching to OpenAI, changing models safely, how search actually scores results, and running fully air-gapped.

Updated:

An embedding model turns text into a vector so that similar meanings land close together. Contextator runs one locally on your CPU by default — no API key, no account, nothing leaving the machine — and can use OpenAI instead if you prefer.

The default

Model Xenova/multilingual-e5-small
Dimensions 384
Languages 100, including Turkish. Ask in the language the answer is written in: matching across languages is a measured limit of the model, not a switch
Download ~470 MB once (fp32), cached in a Docker volume
Runs on CPU. No GPU is used or needed

Smaller and faster options

EMBEDDING_DTYPE=q8                       # same model, ~120 MB, quantized
EMBEDDING_MODEL=Xenova/all-MiniLM-L6-v2  # English-only, ~90 MB, also 384 dimensions
EMBEDDING_DIMENSIONS=384

Any transformers.js feature-extraction model works. EMBEDDING_DIMENSIONS must match what the model produces — if it does not, the server tells you the correct number.

Using OpenAI instead

EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=sk-…
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
EMBEDDING_DIMENSIONS=1536
RESET_VECTORS=1        # only for the first start after the change — see below

Better retrieval on long chunks and a much larger input window, at the cost of an API call per chunk indexed and per query asked. Your documentation text is sent to OpenAI; if that is not acceptable, stay local, or use a server of your own, below.

Using a server of your own

EMBEDDING_PROVIDER=openai speaks to any server that offers an OpenAI-compatible /embeddings endpoint. Set EMBEDDING_BASE_URL to it, with the path, and name the model that server serves in OPENAI_EMBEDDING_MODEL. OPENAI_API_KEY is optional with a base URL; when set, it is sent as a bearer token. EMBEDDING_DIMENSIONS must match the model’s vector size.

# Ollama
EMBEDDING_PROVIDER=openai
EMBEDDING_BASE_URL=http://localhost:11434/v1
OPENAI_EMBEDDING_MODEL=nomic-embed-text
EMBEDDING_DIMENSIONS=768

# LM Studio: its local server
EMBEDDING_BASE_URL=http://localhost:1234/v1

# Text Embeddings Inference, started with -p 8080:80
EMBEDDING_BASE_URL=http://localhost:8080/v1

# vLLM: the host and port you started `vllm serve` on
EMBEDDING_BASE_URL=http://gpu-box:<port>/v1
EMBEDDING_REQUEST_DIMENSIONS=always   # only for a model that shortens its vectors on request

Set the model name and EMBEDDING_DIMENSIONS to the model you loaded in every case; the lines above show only what differs. RESET_VECTORS=1 is needed only when the database already records a vector dimension and the new one differs from it, as moving a 384-dimension install to a 768-dimension model does; a first install needs none. Set it for one start only — see Changing the model. EMBEDDING_REQUEST_DIMENSIONS is auto by default and sends dimensions only for text-embedding-3-*; use always for a model that accepts it and never for a server that rejects it.

Counting tokens with the model’s own tokenizer. Chunks are sized by an estimate (characters ÷ 4) unless EMBEDDING_TOKENIZER names a Hugging Face model whose tokenizer.json matches the served model. Only the tokenizer is downloaded, into MODEL_CACHE_DIR, and startup stops if it cannot be loaded. With it, an endpoint cuts the same chunks the local provider cuts from the same model. It is part of the model id, so setting or changing it re-indexes every project once.

# multilingual-e5-small behind TEI or vLLM
EMBEDDING_PROVIDER=openai
EMBEDDING_BASE_URL=http://localhost:8080/v1        # TEI; vLLM: http://gpu-box:<port>/v1
OPENAI_EMBEDDING_MODEL=intfloat/multilingual-e5-small   # the real name, so the e5 prefixes apply
EMBEDDING_DIMENSIONS=384
EMBEDDING_TOKENIZER=Xenova/multilingual-e5-small

The model’s name decides the prefixes. The query: and passage: prefixes of the multilingual-e5-* family are chosen from the name in OPENAI_EMBEDDING_MODEL, so the model’s real name has to be there. If your server serves the model under another name (a local alias, a path), the name matches no family: the model runs without prefixes and recall quietly drops. The server says so once at startup, at info level, unless you have set either prefix variable. Either put the model’s real name in OPENAI_EMBEDDING_MODEL, or keep your own name and set EMBEDDING_QUERY_PREFIX="query: " and EMBEDDING_PASSAGE_PREFIX="passage: " yourself; either change re-indexes every project once.

Inside the Contextator container, localhost is the container, not your machine: use the address the container can reach the server on. The endpoint’s host and port become part of the model id every project is stamped with, so pointing at another server re-indexes every project — see Changing the model. Text sent to such a server stays wherever that server runs.


Changing the model

Same dimensions (e.g. between the 384-dimension local models)

  1. Change EMBEDDING_MODEL.
  2. Restart.
  3. Re-index each project.

Contextator records which model indexed each project. When the server’s model differs, the next run is automatically a full re-index, and search_docs refuses to search that project until then — with a message saying exactly that, rather than returning confidently wrong results.

Different dimensions (e.g. moving to OpenAI’s 1536)

The vector column’s width is fixed when the database is created, so this is a one-time destructive step:

  1. Set the provider, key and EMBEDDING_DIMENSIONS.
  2. Start once with RESET_VECTORS=1. Every chunk and document is dropped and the column is re-typed.
  3. Remove RESET_VECTORS (or set it back to 0).
  4. Re-index every project.

Without the flag, the server refuses to start and prints this instruction:

The database was created with EMBEDDING_DIMENSIONS=384 but the current config says 1536.
Either set EMBEDDING_DIMENSIONS=384, or start once with RESET_VECTORS=1
(drops every indexed chunk; all projects must be re-indexed), or wipe the pgdata volume.

How search actually works

Search is hybrid: meaning and exact wording are searched at once and fused by rank.

  1. Your question is embedded with the same model that indexed the documents.
  2. PostgreSQL runs two rankings scoped to that project — nearest chunks by cosine similarity over an HNSW index, and a tsvector keyword match over a GIN index — and combines them with reciprocal rank fusion, so an environment variable finds its page as readily as a question does.
  3. The top matches are returned with their file path, heading breadcrumb and score.

Scores are cosine similarities: higher is better, and with the default model they sit high and close together. On the shipped configuration the mean score is 0.881, and a wrong hit sits only about 0.02 below a correct one. A score is therefore not a truth signal, and nothing in the product reads one to decide ranking or correctness; only the off-domain floor below looks at it.

The relevance floor, SEARCH_SCORE_FLOOR=0.82 by default, was measured against something else entirely: questions with nothing to do with your documentation. On the committed set of twelve such questions it refuses ten, and it refuses none of the sixty-four golden questions. That is the whole of what the floor claims: it catches most off-domain questions and costs an answerable one nothing. It does not separate the two classes — the highest off-domain top hit (0.839) sits above the lowest top hit of a real question (0.833), so no threshold does. Nor does it catch a question shaped like your product whose answer is simply not written down — those score exactly where real questions score. Below the floor search_docs answers no good match rather than handing over a hit an agent would cite.

Scores are comparable within one project and one model, and meaningless across different models — change EMBEDDING_MODEL and the floor has to be re-measured with npm run eval.

SEARCH_SCORE_FLOOR (default 0.82) is the server’s floor. A project can set its own from the query panel, or turn the floor off for itself, when its corpus scores differently — prose-heavy corpora usually want a lower one. The panel shows what a floor would have done to the searches already logged before it is applied, and the server’s SEARCH_SCORE_FLOOR=0 still turns every project’s floor off. See Configuration.

Ranks are fused, not scores. Cosine similarity and PostgreSQL’s ts_rank are not comparable quantities, so the fusion combines the rank each chunk held on each side — Σ 1/(60 + rank) over whichever lists it appeared on (reciprocal rank fusion) — rather than a weighted sum that would have to be re-learned every time the embedding model changed. The consequence: a result’s score no longer explains its position — one scoring 0.86 can sit above one scoring 0.88. GET /api/projects/:id/search and the dashboard show the rank each result held on each side as D3 L1; a result with an L and no D is an identifier the vector search alone could not see. On the golden set, rank fusion took the thirty identifier-shaped questions from recall@5 76.7 % to 83.3 % and did not move the thirty-four natural-language questions at all; recall@1 fell from 78.1 % to 75.0 %, the trade fusion makes — a chunk found by one half alone cannot outrank one found respectably by both.

The default model is asymmetric: it was trained with query: in front of a search query and passage: in front of an indexed passage, so the server adds them automatically — nothing you index or search ever mentions them, and a model without prefixes encodes a query and a passage identically at no cost. On the retrieval corpus the prefixes were worth one question at rank 1 and nothing at recall@5; EMBEDDING_QUERY_PREFIX=none with EMBEDDING_PASSAGE_PREFIX=none turns them off and restores the previous model id exactly.

The keyword half indexes each source in PostgreSQL’s simple configuration by default — words as written, no stemming — which is the right choice for reference material: HALYARD_DISPATCH_TIMEOUT and HLY-4015 survive into the index as the strings they are. A source can instead name its language (the Language field on the source form), which switches it to a stemmed configuration — a Turkish question asking about anahtarı then finds a page that says anahtarın, two strings simple treats as unrelated. The two live in one project without interfering: each source is searched in its own configuration and a single query reaches all of them, each contributing its own ranked list to the fusion. Changing a source’s language re-indexes it.

Cross-lingual search is a known limit, and it is not being fixed

Write the question in the same language as the documentation that should answer it. Turkish pages surface for phrasing shaped like Turkish. English pages surface for phrasing shaped like English. Crossing that boundary mostly returns nothing, and this is a property of the embedding model rather than a setting to turn on. Every MCP client is a language model, and list_topics names a source’s language while instructions directs the calling agent to write its search_docs query to match — a direction to the caller, not a change to search itself.

The measurement is in eval/BASELINE.md and it is not close: thirty questions were written from the corpus, each about a page in the language the question was not phrased in — fifteen per direction. Four are answered in the top five. Twenty-seven of the thirty instead return a page in the phrasing’s own language at rank 1: the model is not failing to understand what was asked, it is ranking the language of the phrasing above the answer to it. Both directions fail equally.

Identifiers are the exception. HALYARD_DISPATCH_TIMEOUT, HLY-4015, X-Halyard-Signature are the same string in both languages, so the keyword half of search finds them regardless. Of the nine cross-lingual questions in the set that name an identifier, four are answered; of the twenty-one phrased as ordinary questions, none is.

What to do instead of waiting for a fix. Keep each language in its own source and let an agent scope its search with source, path_prefix or version — two sources that each answer well beat one collection that answers either language badly. Both known remedies have already been measured: hybrid search (above) recovered cross-lingual identifier retrieval and no natural-language question; a multilingual cross-encoder rerank behind SEARCH_RERANK took cross-lingual recall@5 from 13.3 % to 33.3 % — still below the 42.9 % the embedding model before this one managed — while costing thirteen ordinary questions their rank-1 answer and a search’s latency going from 12 ms to 1.2 s. It is off by default and should stay off. A second, translation-trained encoder as an additional index would actually fix this — roughly four times the download, a second vector column, a full re-index on every installation — and that price is not worth paying for a documentation server. If cross-language search is essential to your corpus, that is a reason to choose a different tool rather than a reason to wait for this one.

Air-gapped installations

The only outbound request a default installation makes is the one-time model download. To remove even that:

  1. Pre-populate the models volume (/app/.cache/models) with the model files — for example by running once on a connected host and copying the volume. Do it with the same EMBEDDING_MODEL and EMBEDDING_DTYPE the air-gapped host will use.
  2. Set EMBEDDING_OFFLINE=1.

The server will then never attempt a download, and will fail loudly if the model is missing.

The reranker

The optional reranker (SEARCH_RERANK=on, off by default) is a second model, SEARCH_RERANK_MODEL (Xenova/bge-reranker-base by default), loaded into the same cache directory and governed by the same EMBEDDING_OFFLINE. With the reranker on and the flag set, its files must already be in the volume too: warm the cache on the connected host with SEARCH_RERANK=on and the same SEARCH_RERANK_MODEL and SEARCH_RERANK_DTYPE before you copy it.

Moving the image and the models

On a connected host, pull the image and write it to a file, carry the file across, and load it:

docker pull contextator/contextator:<tag>
docker save --output contextator.tar contextator/contextator:<tag>
# on the air-gapped host
docker load --input contextator.tar

Use the tag you run, not latest, so the two hosts agree on the version. The models volume is not part of the image: copy it separately (the default volume is contextator-models), or mount a directory with CONTEXTATOR_MODELS_PATH.

What still leaves the host

Source type or feature Outbound request
Embedding, default local model None once the cache is filled; otherwise the one-time model download from the Hugging Face Hub
Tokenizer named by EMBEDDING_TOKENIZER (with EMBEDDING_PROVIDER=openai) Downloaded from the Hugging Face Hub unless the cache already holds it; subject to EMBEDDING_OFFLINE, and startup stops if it cannot be found
Reranker None once the cache is filled; otherwise its one-time model download from the Hugging Face Hub
Embedding with EMBEDDING_PROVIDER=openai Document text and queries, to EMBEDDING_BASE_URL (OpenAI unless you changed it)
Local directory, upload None
Git repository Your git host, over HTTP(S), on each sync and on a webhook-triggered one
Notion Notion’s API, on each sync
Confluence Your Confluence instance, on each sync
Documentation site The site you pointed it at, plus its robots.txt

Remote sources are fetched from this server, not from the browser that set them up, so an air-gapped host that needs one of them needs a route to that host and no other.

Performance notes

  • Embedding is CPU-bound and is the slow part of indexing; that is why one project is indexed at a time.
  • EMBEDDING_BATCH_SIZE (default 16) controls how many chunks are embedded per call.
  • A query embeds exactly one short text, so search latency is dominated by the database, not the model.
  • The model is loaded in the background at startup — the dashboard works while it loads, but indexing and search wait for it.

Arrow keys to move, Enter to open.