Scaling

RememberStack publishes no throughput figures: how many documents per hour, or queries per second, a machine handles depends mostly on your model provider and corpus. What exists are explicit bounds on every process, so a busy deployment slows down or refuses work. The one budget you keep yourself is PostgreSQL's connection limit, once you add replicas. This page lists those bounds and how to move them.

The model provider is usually the first limit you meet. Most pipeline stages spend their time waiting on OpenRouter calls, and the provider's own rate limits apply to your account.

Workers

Each pipeline stage runs in its own container. A worker process claims one item of work at a time from PostgreSQL, runs it, and claims the next. It wakes on a notification when new work is committed and also polls in case a notification was missed.

Variable	Default	Meaning
`REMEMBERSTACK_SELFHOST_WORKER_RATE_PER_S`	`20`	Claims per second one worker process may make.
`REMEMBERSTACK_SELFHOST_WORKER_BURST`	`20`	How many claims it may make at once before the rate applies.
`REMEMBERSTACK_SELFHOST_WORKER_FALLBACK_POLL_S`	`5`	Seconds between polls when no notification arrives.
`REMEMBERSTACK_SELFHOST_WORKER_SESSION_S`	`3600`	Length of one worker session in seconds; the worker then starts the next one.

Set them in .env. Lowering the rate is the way to stay under a provider's rate limit; the rate applies per process, so it multiplies with replicas.

More replicas of a stage

Items are claimed with SELECT … FOR UPDATE SKIP LOCKED, so several processes can serve the same stage without taking the same item. Add replicas of the stage that is behind:

docker compose up -d --scale worker-extract-claims=3 --scale worker-normalize-relations=2

To find the stage that is behind, look at the pending counts per stage in remember ops inspect (Operating the pipeline). The claim-heavy stages (extract_claims, ground_claims, normalize_relations, adjudicate_observations) make the most model calls per document.

Every replica holds its own database connections; see Database connections before adding many.

Database connections

Each engine process has its own connection pools. Compose starts PostgreSQL with max_connections set from REMEMBERSTACK_POSTGRES_MAX_CONNECTIONS (default 300); the engine does not size itself against that limit. The ceilings per process are:

Process	Ceiling	Made of
`api`	23	General pool 15 (5 kept open, 10 more under load), retrieval pool 4, graph pool 4
Each `worker-…` replica	16	General pool 15, plus 1 connection that listens for new work

The general pool's size is fixed. The retrieval and graph pools follow REMEMBERSTACK_SELFHOST_RETRIEVAL_POOL_SIZE and REMEMBERSTACK_SELFHOST_GRAPH_POOL_SIZE (see Retrieval and graph queries).

Pools open connections only on demand, so a running stack holds fewer than the sum of its ceilings. The default stack's ceilings add up to 215 (23 + 12 × 16), within the default limit of 300. When you add replicas or API processes, keep the sum of their ceilings under the limit, and when PostgreSQL refuses connections (too many clients already in the logs), raise it in .env and restart PostgreSQL:

echo 'REMEMBERSTACK_POSTGRES_MAX_CONNECTIONS=600' >> .env
docker compose up -d postgres

Every PostgreSQL connection takes memory on the database host, so raise the limit only as far as that machine allows.

Embedding batches

Variable	Default	Range
`REMEMBERSTACK_E1_EMBED_BATCH_SIZE`	`64`	1–512 chunks per embedding request
`REMEMBERSTACK_P1_EMBED_BATCH_SIZE`	`64`	1–1,024 claims or facts per embedding request

Retrieval and graph queries

The API process keeps two dedicated connection pools, apart from its main one, so a burst of searches or graph traversals cannot take every connection.

Variable	Default	Range	Meaning
`REMEMBERSTACK_SELFHOST_RETRIEVAL_POOL_SIZE`	`4`	1–32	Connections for searches and fact reads
`REMEMBERSTACK_SELFHOST_RETRIEVAL_MAX_CONCURRENCY`	`4`	1–32, at most the pool size	Retrievals running at once
`REMEMBERSTACK_SELFHOST_RETRIEVAL_POOL_TIMEOUT_S`	`1`	up to 30	Seconds a retrieval waits for a connection before failing
`REMEMBERSTACK_SELFHOST_GRAPH_POOL_SIZE`	`4`	1–32	Connections for graph neighbourhoods and paths
`REMEMBERSTACK_SELFHOST_GRAPH_MAX_CONCURRENCY`	`2`	1–32, at most the pool size	Graph queries running at once
`REMEMBERSTACK_SELFHOST_GRAPH_POOL_TIMEOUT_S`	`1`	up to 30	Seconds a graph query waits for a slot before failing
`REMEMBERSTACK_SELFHOST_GRAPH_WORK_MEM_KIB`	`16384`	64–65,536	PostgreSQL `work_mem` for each graph query

compose.yaml passes all seven. A concurrency larger than its pool stops the API at start.

When the graph limit is reached, graph routes answer 503 with live graph is busy. When a retrieval waits longer than its timeout, the request fails; that failure is not yet mapped to a specific status code.

Raise the pool, the concurrency and work_mem together, and only as far as PostgreSQL's memory allows: each running graph query may use up to GRAPH_WORK_MEM_KIB for each sort or hash step it performs.

API admission limits

Optional, and off by default: the API does not limit the request rate or the number of requests running at once until you set one of these in .env. Each set limit counts every request (except GET /healthz) per credential or for the whole deployment; over it the API answers 429 with Retry-After.

Variable	Default	Meaning
`REMEMBERSTACK_SELFHOST_API_ADMISSION_KEY_PER_MINUTE`	unset (no limit)	Requests per minute per signed credential, in bursts of up to a quarter of it
`REMEMBERSTACK_SELFHOST_API_ADMISSION_KEY_IN_FLIGHT`	unset (no limit)	Requests running at once per signed credential
`REMEMBERSTACK_SELFHOST_API_ADMISSION_DEPLOYMENT_PER_MINUTE`	unset (no limit)	Requests per minute for the deployment, in bursts of up to a quarter of it
`REMEMBERSTACK_SELFHOST_API_ADMISSION_DEPLOYMENT_IN_FLIGHT`	unset (no limit)	Requests running at once for the deployment

0 also means no limit. Set them when a deployment is shared by callers you do not control and one of them could crowd out the rest. The shared secret, and every caller when authentication is off, meets the deployment limits only. With limits set, a script that ingests many files in parallel should keep its concurrency under the in-flight limit and wait Retry-After on a 429; the remember client does not retry by itself. The counters live in the API process, so running more API processes multiplies the effective limits. See Admission limits.

SQL queries

SQL queries run in a sandbox over the query space (memory_v1, the prepared read-only views and functions); every statement is validated against it before it runs. The sandbox limits are fixed in code for a self-hosted deployment:

Limit	Default	Maximum a caller can request
Statement time	5 s	15 s
Rows returned	200	1,000
Bytes returned	1 MiB	8 MiB
`work_mem`	16 MiB	—
Temporary files	64 MiB	—
Statements at once, per caller	2	—
Statements at once, per deployment	8	—
Statement seconds per minute, per caller	30	—
Statement seconds per minute, per deployment	120	—

The code also defines a larger analytical tier (60 s statements, 10,000 rows). A self-hosted deployment does not enable it.

Ingest size

REMEMBERSTACK_SELFHOST_INGEST_BODY_MAX_BYTES caps the size of one upload to POST /ingest. Unset, the default, means the engine imposes no cap. Converters have their own ceilings (50,000,000 bytes for Mistral OCR, 10,000,000 for images).