Scaling
RememberStack publishes no throughput figures: how many documents per hour, or queries per second, a machine handles depends mostly on your model provider and corpus. What exists are explicit bounds on every process, so a busy deployment slows down or refuses work. The one budget you keep yourself is PostgreSQL's connection limit, once you raise concurrency or add containers. This page lists those bounds and how to move them.
The model provider is usually the first limit you meet. Most pipeline stages spend their time waiting on OpenRouter calls, and the provider's own rate limits apply to your account.
Workers
The pipeline runs in two worker containers. worker-convert converts
uploaded files one at a time, so its memory stays bounded by the largest
file you accept. worker-llm runs every other stage with up to 20 items in
flight at once, because those stages spend most of their time waiting on
model calls. Both claim work from PostgreSQL, wake on a notification when
new work is committed, and also poll in case a notification was missed. The
llm worker takes items further along the pipeline before new ones, and
live work before backfill work.
| Variable | Default | Meaning |
|---|---|---|
REMEMBERSTACK_SELFHOST_WORKER_RATE_PER_S | 20 | Claims per second one worker process may make. |
REMEMBERSTACK_SELFHOST_WORKER_BURST | 20 | How many claims it may make at once before the rate applies. |
REMEMBERSTACK_SELFHOST_WORKER_FALLBACK_POLL_S | 5 | Seconds between polls when no notification arrives. |
REMEMBERSTACK_SELFHOST_WORKER_SESSION_S | 3600 | Length of one worker session in seconds; the worker then starts the next one. |
Set them in .env. Lowering the rate is the way to stay under a provider's rate limit; the
rate applies per process, so it multiplies with containers.
More items in flight
Raise a worker's concurrency in .env and recreate it:
| Variable | Default | Meaning |
|---|---|---|
REMEMBERSTACK_SELFHOST_LLM_CONCURRENCY | 20 | Items worker-llm runs at once. |
REMEMBERSTACK_SELFHOST_CONVERT_CONCURRENCY | 1 | Files worker-convert converts at once. Each conversion can take as much memory as the file needs. |
echo 'REMEMBERSTACK_SELFHOST_LLM_CONCURRENCY=40' >> .env
docker compose up -d worker-llmItems are claimed with SELECT … FOR UPDATE SKIP LOCKED, so you can also
run more than one container of a worker without two taking the same item:
docker compose up -d --scale worker-llm=2To see which stage is behind, look at the pending counts per stage in
remember ops inspect (Operating the pipeline).
The claim-heavy stages (extract_claims, ground_claims,
normalize_relations, adjudicate_observations) make the most model calls
per document. A higher concurrency also means more calls at once to your
model provider, whose rate limits still apply.
Every container holds its own database connections, and more items in flight need more of them; see Database connections.
Database connections
Each engine process has its own connection pools. Compose starts
PostgreSQL with max_connections set from
REMEMBERSTACK_POSTGRES_MAX_CONNECTIONS (default 300); the engine does not
size itself against that limit. The ceilings per process are:
| Process | Ceiling | Made of |
|---|---|---|
api | 23 | General pool 15 (5 kept open, 10 more under load), retrieval pool 4, graph pool 4 |
| Each worker container | 2 × concurrency + 3 | General pool of concurrency + 2 kept open and as many more under load (one task can briefly need two connections), plus 1 connection that listens for new work. worker-llm at 20: 43; worker-convert at 1: 5 |
The API's general pool size is fixed. The retrieval and graph pools follow
REMEMBERSTACK_SELFHOST_RETRIEVAL_POOL_SIZE and
REMEMBERSTACK_SELFHOST_GRAPH_POOL_SIZE (see
Retrieval and graph queries).
Pools open connections only on demand, so a running stack holds fewer than
the sum of its ceilings. The default stack's ceilings add up to 71 (23 +
43 + 5), within the default limit of 300. When you raise concurrency or add
containers or API
processes, keep the sum of their ceilings under the limit, and when
PostgreSQL refuses connections (too many clients already in the logs),
raise it in .env and restart PostgreSQL:
echo 'REMEMBERSTACK_POSTGRES_MAX_CONNECTIONS=600' >> .env
docker compose up -d postgresEvery PostgreSQL connection takes memory on the database host, so raise the limit only as far as that machine allows.
Embedding batches
| Variable | Default | Range |
|---|---|---|
REMEMBERSTACK_E1_EMBED_BATCH_SIZE | 64 | 1–512 chunks per embedding request |
REMEMBERSTACK_P1_EMBED_BATCH_SIZE | 64 | 1–1,024 claims or facts per embedding request |
Retrieval and graph queries
The API process keeps two dedicated connection pools, apart from its main one, so a burst of searches or graph traversals cannot take every connection.
| Variable | Default | Range | Meaning |
|---|---|---|---|
REMEMBERSTACK_SELFHOST_RETRIEVAL_POOL_SIZE | 4 | 1–32 | Connections for searches and fact reads |
REMEMBERSTACK_SELFHOST_RETRIEVAL_MAX_CONCURRENCY | 4 | 1–32, at most the pool size | Retrievals running at once |
REMEMBERSTACK_SELFHOST_RETRIEVAL_POOL_TIMEOUT_S | 1 | up to 30 | Seconds a retrieval waits for a connection before failing |
REMEMBERSTACK_SELFHOST_GRAPH_POOL_SIZE | 4 | 1–32 | Connections for graph neighbourhoods and paths |
REMEMBERSTACK_SELFHOST_GRAPH_MAX_CONCURRENCY | 2 | 1–32, at most the pool size | Graph queries running at once |
REMEMBERSTACK_SELFHOST_GRAPH_POOL_TIMEOUT_S | 1 | up to 30 | Seconds a graph query waits for a slot before failing |
REMEMBERSTACK_SELFHOST_GRAPH_WORK_MEM_KIB | 16384 | 64–65,536 | PostgreSQL work_mem for each graph query |
compose.yaml passes all seven. A concurrency larger than its pool stops
the API at start.
When the graph limit is reached, graph routes answer 503 with
live graph is busy. When a retrieval waits longer than its timeout, the
request fails; that failure is not yet mapped to a specific status code.
Raise the pool, the concurrency and work_mem together, and only as far as
PostgreSQL's memory allows: each running graph query may use up to
GRAPH_WORK_MEM_KIB for each sort or hash step it performs.
API admission limits
Optional, and off by default: the API does not limit the request rate or the
number of requests running at once until you set one of these in .env.
Each set limit counts every request (except GET /healthz) per credential or
for the whole deployment; over it the API answers 429 with Retry-After.
| Variable | Default | Meaning |
|---|---|---|
REMEMBERSTACK_SELFHOST_API_ADMISSION_KEY_PER_MINUTE | unset (no limit) | Requests per minute per signed credential, in bursts of up to a quarter of it |
REMEMBERSTACK_SELFHOST_API_ADMISSION_KEY_IN_FLIGHT | unset (no limit) | Requests running at once per signed credential |
REMEMBERSTACK_SELFHOST_API_ADMISSION_DEPLOYMENT_PER_MINUTE | unset (no limit) | Requests per minute for the deployment, in bursts of up to a quarter of it |
REMEMBERSTACK_SELFHOST_API_ADMISSION_DEPLOYMENT_IN_FLIGHT | unset (no limit) | Requests running at once for the deployment |
0 also means no limit. Set them when a deployment is shared by callers you
do not control and one of them could crowd out the rest. The shared secret,
and every caller when authentication is off, meets the deployment limits
only. With limits set, a script that ingests many files in parallel should
keep its concurrency under the in-flight limit and wait Retry-After on a
429; the remember client does not retry by itself. The counters live in
the API process, so running more API processes multiplies the effective
limits. See
Admission limits.
SQL queries
SQL queries run in a sandbox over the query space
(memory_v1, the prepared read-only views and functions); every statement
is validated against it before it runs. The sandbox limits are fixed in
code for a self-hosted deployment:
| Limit | Default | Maximum a caller can request |
|---|---|---|
| Statement time | 5 s | 15 s |
| Rows returned | 200 | 1,000 |
| Bytes returned | 1 MiB | 8 MiB |
work_mem | 16 MiB | — |
| Temporary files | 64 MiB | — |
| Statements at once, per caller | 2 | — |
| Statements at once, per deployment | 8 | — |
| Statement seconds per minute, per caller | 30 | — |
| Statement seconds per minute, per deployment | 120 | — |
The code also defines a larger analytical tier (60 s statements, 10,000 rows). A self-hosted deployment does not enable it.
Ingest size
REMEMBERSTACK_SELFHOST_INGEST_BODY_MAX_BYTES caps the size of one upload
to POST /ingest. Unset, the default, means the engine imposes no cap.
Converters have their own ceilings (50,000,000 bytes for Mistral OCR,
10,000,000 for images).