Models and providers
RememberStack calls language models to read your documents and an
embedding model to make them searchable. A self-hosted deployment sends
every one of those calls through OpenRouter with
the key in REMEMBERSTACK_OPENROUTER_API_KEY, and pays for them on that
account. This page lists each place a model is used (a seat), what it
does, and the settings that shape the calls.
Retrieval itself calls no language model. A search embeds your query text once; the operations, graph and SQL queries then read the database.
Model seats
Each seat is one environment variable holding an OpenRouter model id. The
defaults are the values compose.yaml passes when .env leaves them
unset.
| Seat | Variable | Default | What it does |
|---|---|---|---|
| Structure fallback | REMEMBERSTACK_STRUCTURER_MODEL | openai/gpt-5.6-luna | Proposes section anchors when a document's own headings do not give a usable outline. A document whose headings pass the structure checks never reaches this seat. |
| Skeleton check | REMEMBERSTACK_SKELETON_CHECK_MODEL | z-ai/glm-4.7-flash | Checks a proposed section outline, independently of the model that proposed it. |
| Section role | REMEMBERSTACK_ROLE_MODEL | z-ai/glm-4.7-flash | Classifies what each section is for, from its title. |
| Section summary | REMEMBERSTACK_SUMMARY_MODEL | z-ai/glm-4.7-flash | Writes the short summary of each section. |
| Claim extraction | REMEMBERSTACK_E2_EXTRACT_MODEL | openai/gpt-5.6-luna | Selects the sentences that state something and turns them into claims. Used by both the extract_claims and ground_claims workers. |
| Relation normalisation | REMEMBERSTACK_E3_NORMALIZE_MODEL | openai/gpt-5.6-luna | Turns claims into relations between entities. |
| Entity resolution | REMEMBERSTACK_OBS_SMALL_MODEL | openai/gpt-5.6-luna | Decides between candidate entities when the deterministic name matching cannot. |
| Fact adjudication | REMEMBERSTACK_FACT_MODEL | openai/gpt-5.6-luna | Decides whether a new statement confirms, contradicts or replaces a fact the memory holds. |
| Embeddings | REMEMBERSTACK_P1_EMBEDDING_MODEL | qwen/qwen3-embedding-8b | Every vector: chunks, claims, facts, entity profiles, and the query text of each search. |
GET /deployment reports the models the running deployment is bound to
under model_bindings, so you can check what is serving:
curl http://localhost:8000/deploymentFact adjudication
Fact adjudication decides what the memory holds true, so it has two extra settings:
| Variable | Default | Meaning |
|---|---|---|
REMEMBERSTACK_FACT_ADJUDICATION_ENGINE | prompt | prompt asks REMEMBERSTACK_FACT_MODEL through OpenRouter. jev asks a TypeSafe AI System One model instead. |
REMEMBERSTACK_FACT_CONFIDENCE_FLOOR | 0.75 | Below this confidence the adjudicator does not replace or contradict anything: the new statement and the existing fact are both kept. |
The jev engine needs its own key and settings:
| Variable | Default |
|---|---|
REMEMBERSTACK_TYPESAFE_API_KEY | none (required with jev; the fact workers refuse to start without it) |
REMEMBERSTACK_TYPESAFE_MODEL | jev-latest |
REMEMBERSTACK_TYPESAFE_BASE_URL | https://api.typesafe.ai/v1 |
REMEMBERSTACK_TYPESAFE_TIMEOUT_S | 30.0 |
REMEMBERSTACK_FACT_FALLBACK_TO_PROMPT | false. When true, a failed TypeSafe call falls back to the prompt engine instead of failing the work item. |
Change a model
Put the new model id in .env and apply it:
# .env
REMEMBERSTACK_E2_EXTRACT_MODEL=openai/gpt-5.6-soldocker compose up -dUse exact model ids. A rotating router id such as openrouter/free makes
results impossible to reproduce and to attribute. A chat seat needs a model
that supports structured (JSON schema) output, because every seat asks for
a typed answer and rejects one that does not match.
A new model applies to work done from then on. Documents already processed keep what the old model produced; RememberStack does not re-run them on its own.
Changing the embedding model
Every vector in RememberStack has 1,536 dimensions. The column types,
the search channels and every request are fixed at that size: the engine
asks the provider for 1,536 dimensions and rejects a response of any other
length. An embedding model can replace qwen/qwen3-embedding-8b only if it
can return 1,536-dimension vectors on request.
Each stored vector is stamped with the model that produced it, and search compares a query only with vectors of the same model. Nothing re-embeds stored chunks, claims and facts, so the embedding model is fixed once anything has been embedded:
- Before the first document is embedded, you can change
REMEMBERSTACK_P1_EMBEDDING_MODELfreely. - After that,
setuprefuses to run with a different model. It exits with an error that names the configured model and the stored one, and the API and workers do not start. Set the variable back to the stored model.
To change it before anything is embedded, stop the workers first
(docker compose stop), then change the variable and run
docker compose up -d. A worker still running on the old model could
otherwise embed with it after the switch; the next setup would then
refuse the new model.
To use a different embedding model, start a new deployment and send the documents again.
Optional OpenAI Decisions backup for Jev
With REMEMBERSTACK_FACT_ADJUDICATION_ENGINE=jev, Jev stays primary. Set
REMEMBERSTACK_TYPESAFE_BACKUP=openai_decisions and
REMEMBERSTACK_OPENAI_DECISIONS_API_KEY to permit one native Decisions request
after exhausted connection-establishment, rate-limit (429), or overload (529)
failures. The default is none. The backup uses gpt-6-luna on OpenAI's native
API; OpenRouter credit does not pay for it.
Outside Compose, OPENAI_API_KEY is also accepted. Compose forwards only the
prefixed credential, so set that variable for container deployments.
Native response handling has only been validated with offline mock transports.
Run a small live API check with your native credential before enabling the backup.
Valid low confidence, invalid credentials, malformed answers, ambiguous read timeouts, generic gateway errors and missing usage do not trigger the backup. The adapter supports choice, yes/no and score questions and retains the full option set. Decisions cannot silently invent an option or replace an uncertain answer with one from another provider. Evaluate both providers on your own labeled cases before sharing a confidence threshold between them.
Usage records identify openai-decisions/<served-model>. Costs include ordinary
input, cache reads, 30-minute cache writes, long-context rates and regional
multipliers; output tokens are free on this endpoint. The initial verified
global short-context rates are $0.10/$0.01/$0.125 per million tokens for ordinary,
cached and cache-write input, respectively. Price settings are configurable;
keep them current with OpenAI's pricing.
The current LoCoMo profile explicitly disables this backup.
OpenRouter routing and limits
Entity resolution gives the model a closed list of existing candidate IDs and an explicit new-entity outcome. Both the provider schema and local validation reject IDs outside that list. Choosing a new entity uses a null ID; the engine creates the ID. Invalid matches never implicitly create entities.
These settings apply to every call through OpenRouter.
| Variable | Default | Meaning |
|---|---|---|
REMEMBERSTACK_OPENROUTER_CHAT_PROVIDER_ORDER | unset | Comma-separated provider slugs to try first for chat calls. Other providers remain a fallback unless the same set is also supplied as the hard allowlist. |
REMEMBERSTACK_OPENROUTER_CHAT_PROVIDER_ONLY | unset | Comma-separated provider slugs that are the only ones allowed. Alone it disables fallback. Combine with an order naming exactly the same unique slugs to allow fallback within this set. |
REMEMBERSTACK_OPENROUTER_EMBEDDING_PROVIDER_ORDER | unset | Provider slugs to try first for embeddings, with fallback, for example nebius,deepinfra,siliconflow. |
REMEMBERSTACK_OPENROUTER_EMBEDDING_PROVIDER | unset | One provider slug that embeddings must use, with no fallback. The order wins if both are set. |
REMEMBERSTACK_OPENROUTER_ZDR | false | Restricts chat calls to zero-data-retention endpoints. |
REMEMBERSTACK_OPENROUTER_REASONING_EFFORT | unset (model default) | One of none, minimal, low, medium, high, xhigh, max, applied to every chat call. |
REMEMBERSTACK_OPENROUTER_REASONING_EFFORT_MAP | unset | A JSON object of model id to effort. An entry wins over the global value for that model. |
REMEMBERSTACK_OPENROUTER_MAX_COMPLETION_TOKENS | 32000 | Output allowance per chat call, reasoning included. |
REMEMBERSTACK_OPENROUTER_TIMEOUT_S | 120 | Per-request timeout in seconds. |
REMEMBERSTACK_OPENROUTER_CHAT_THROTTLE_RETRIES | 3 | How many times a chat call that gets HTTP 429 is retried, moving to the next listed provider. These retries do not count against the work item's attempts. |
REMEMBERSTACK_OPENROUTER_CHAT_UPSTREAM_OVERLOAD_MAX_RETRY_AFTER_S | 30 | The longest single wait before retrying an overloaded provider. |
REMEMBERSTACK_OPENROUTER_BASE_URL | https://openrouter.ai/api/v1 | The OpenRouter API address. |
When you set a provider list or ZDR, each chat call also tells OpenRouter
not to route to providers that collect data (data_collection: deny). With
none of them set, OpenRouter applies your account's own routing settings.
An explicit Flex capacity rejection can arrive inside HTTP 200 and include
an ID even though processing did not start. When that error carries neither
choices nor usage, the adapter applies its bounded capacity retries while
keeping your provider restriction. To prefer Flex and permit normal OpenAI
processing when Flex is unavailable, set both chat provider variables to
openai/flex,openai. This keeps the requested model and reasoning effort;
standard processing can cost more, and no other provider is admitted. A
Flex-only allowlist continues to forbid standard-price fallback.
The same bounded retry path handles a transient error whose
error.metadata.error_type is provider_unavailable, even when OpenRouter has
allocated an ID, provided neither choices nor usage is present. This follows
OpenRouter's failed-attempt billing policy.
Generic ID-bearing errors and any response with choices or usage still require
accounting recovery; this rule does not assume missing usage means zero cost.
If a completed response has unusable token or cost accounting, the adapter looks up that existing generation's metadata. If recovery fails, the work item stops with an accounting error containing the generation ID and lookup status. Use those identifiers to reconcile the request with the provider before retrying paid work. The adapter does not invent a zero cost or automatically generate the same potentially paid response again.
A reply can arrive with a success status but a body that is not a JSON
object: empty, only whitespace, an HTML page, a cut-off object, or a JSON
value such as null or []. The call then fails with an ordinary OpenRouter
provider error, on chat and embedding calls alike. The message names the
endpoint, the status, the body size and the kind of problem, for example:
OpenRouter /chat/completions returned 200 with an undecodable body (body=whitespace_only, body_bytes=26, decode_error='Expecting value', decode_pos=26)The message never contains the body text, headers or your API key. A success status does not prove the request was free, so usage stays unknown rather than zero. The adapter does not treat this as a capacity rejection, so it does not retry, rotate to another provider, or start a new retry budget. Callers handle it as an ordinary provider error, using their existing retry or fallback policy: some stages retry the work item, while optional steps, such as section summaries, continue without the model's answer. Valid replies and the capacity retries above work as before.
Lowering reasoning effort makes extraction faster and cheaper; the comment
in .env.example notes that none can reduce adjudication quality and
that some models reject it. A map lets you turn it off for the small models
only:
REMEMBERSTACK_OPENROUTER_REASONING_EFFORT_MAP={"z-ai/glm-4.7-flash":"none","openai/gpt-5.6-luna":"high"}For debugging, REMEMBERSTACK_OPENROUTER_INVALID_COMPLETION_CAPTURE_DIR
(an absolute path, for example
/var/lib/rememberstack/openrouter-invalid-completions) keeps every model
answer that failed schema validation and each response whose accounting could
not be recovered, one file each, readable only by the engine user. Accounting
captures keep the original provider response for reconciliation and do not
authorize replay. These files can repeat text from your documents; treat the
directory as customer data and leave the setting off otherwise.
Cost
Every document costs model calls: section work during structuring, claim extraction for every chunk, relation normalisation, entity resolution and fact adjudication for what it says, and embeddings for its chunks, claims and facts. Each search costs one query embedding. The size of the bill depends on your documents and the models you choose; RememberStack has published no per-document cost figures.
Every billed call is written to the cost ledger with its model, tokens, cost in US dollars and the pipeline stage that made it. Read it with the cost export, and cap it per stage with spend budgets. The provider account's own limit is the final monetary boundary.