Upgrades and migrations
A self-hosted deployment is yours to upgrade and to back up. This page covers how releases are published, what happens to the database on an upgrade, the order to do it in, and how to take a backup you can restore.
Releases and image tags
Each release has a Git tag (v0.17.0) and an image with the same version
number and no v: ghcr.io/writeitai/remember-stack:0.17.0. There is no
latest tag. The release on GitHub attaches the compose.yaml and
.env.example for that version (the second downloads as
default.env.example) and openapi.json.
compose.yaml names the image tag, so the version you run is the version
of the compose.yaml you start. The client package on PyPI, remember,
is released from the same tag with the same version number.
What setup does on an upgrade
The setup service runs before the API and workers every time you run
docker compose up. It:
- applies every database migration the new release brings
(
alembic upgrade head); - checks that the deployment id, slug and name still match the database;
- installs any assured operations and example saved queries the release adds;
- checks that the embedding model still matches the stored vectors, then re-publishes the search channels, rebuilding entity profile vectors when needed.
The API and the workers start only after setup exits successfully. If a
migration fails, setup exits with an error, nothing else starts, and
docker compose logs setup shows why.
Each migration revision commits separately; online index phases can also commit inside a revision. If a later revision fails, completed revisions stay applied. Migrations only go forward. Some refuse to convert data in a way that would lose it, and there is no supported downgrade. To go back to an earlier release, restore a backup taken before the upgrade.
Upgrading does not reprocess documents already in the memory. They keep what the earlier release extracted from them; new documents are processed by the new code.
Some releases cannot convert existing data
When a release changes what stored data means in a way that cannot be
converted, its migrations stop instead of guessing, and setup exits
with an error that ends in "recreate the deployment and ingest its
sources again". Start from an empty deployment and send your source
documents again. Keep your source files for this reason. See
setup refuses an existing database.
Upgrade step by step
-
Read the release notes on GitHub for the version you are moving to.
-
Back up, as described below.
-
Get the new files:
git fetch --tags git checkout v<new-version> diff .env .env.exampleAdd any new required variable to your
.env. Keep your deployment id, slug and name as they are. -
Stop the old processes, so no old worker runs against a database that is being migrated:
docker compose down --remove-orphansdownremoves the containers and keeps the volumes, and with them the memory.--remove-orphansalso stops containers of services the newcompose.yamlno longer has, such as the per-stage workers of releases before the two worker containers. -
Get the new images:
docker compose pull api postgresAll app services share the image
pull apifetches. Each release also publishes its own PostgreSQL image, sopostgresis pulled too. -
Start:
docker compose up -d --wait docker compose logs setup curl http://localhost:8000/deploymentbuild_revisionin/deploymentnames the source commit of the image now serving.
Several deployments on one PostgreSQL server
The Compose file runs one deployment with its own PostgreSQL. If you run several deployments against one PostgreSQL server instead, each keeps everything in its own database. Upgrade them one database at a time, or several at once: the migrations of one database never change another's, and wait for another only while the shared role below is first created.
Run setup once per deployment, each with that deployment's own settings:
its REMEMBERSTACK_DATABASE_URL, deployment id, slug, name, buckets and
keys. For example, with one env file per deployment and the release's
engine image (replace VERSION):
pids=()
for env_file in deployments/*.env; do
docker run --rm --env-file "$env_file" \
ghcr.io/writeitai/remember-stack:VERSION setup &
pids+=("$!")
done
failed=0
for pid in "${pids[@]}"; do
wait "$pid" || failed=1
done
exit "$failed"Start each deployment's API and workers only after its own setup has
succeeded. RememberStack does not coordinate the loop for you: if one run
fails, rerun it for that database. Running a release's migrations again on a
database that already has them changes nothing.
Three things are shared across the server, so keep them in mind:
- Database names are at most 43 characters of
a–z,0–9and_, not starting with a digit. Each database gets its own roles,rememberstack_query_<database>andrememberstack_graph_<database>. PostgreSQL lowercases those role names and cuts them at 63 bytes, so a longer or mixed-case database name could share a role with another database. Migrations refuse such a name. - One role is shared by every database:
rememberstack_view_owner, which owns the public query views. The first migration on a server creates it. If several first migrations run at once, the later ones wait for the first one to finish and then continue. - Partition maintenance runs per database. pg_partman's background worker
maintains only the databases listed, comma-separated, in the
pg_partman_bgw.dbnameserver setting, so add each deployment's database there. Or leave the background worker out ofshared_preload_librariesand runCALL run_maintenance_proc();in each database on a schedule (hourly is ample for monthly partitions). RememberStack does not depend on the background worker itself.
PostgreSQL server versions
PostgreSQL 18 is the reference server. Compose builds the pinned official 18.x base with the project's extensions. PostgreSQL 19 was adopted for SQL/PGQ property graphs, but upstream removed that feature and the engine uses ordinary joins, so a prerelease server is no longer needed.
This is a major-version change. A PostgreSQL 19 data directory cannot be
opened by PostgreSQL 18, and pg_upgrade cannot downgrade it. With no user
databases, the supported development path is fresh PostgreSQL 18 storage,
setup, and replaying the source documents. To remove only the database
volume, stop the stack, identify its postgres-data volume with
docker volume ls, and remove that volume before starting with --build.
Keep object-store and other volumes when their contents must survive.
If you must preserve data, qualify a logical export/import or source replay in a separate operator migration plan before replacing the server. This release provides no automatic PG19 conversion or dual-version runtime. Release notes record subsequent PostgreSQL 18 minor image changes.
The deployment id is permanent
REMEMBERSTACK_SELFHOST_DEPLOYMENT_ID names the deployment: its trust
domain. Every token is bound to it, and every stored row belongs to it.
The install step generates it once, and the first setup records it.
A database holds exactly one deployment. If the id in .env differs from
the recorded one, setup refuses to start and names the recorded id, so
the API and workers do not start either. Restore the recorded value. The
same check covers the slug, the name, the default language and the bucket
names: setup names the ones that changed. Change none of them after the
first start.
To start over under a new id, remove the volumes
(docker compose down --volumes) and send your documents again.
Back up
Backups are your responsibility. RememberStack does not schedule them, and the project does not test a restore procedure. What follows is grounded in how the Compose deployment stores its data.
A deployment's state lives in four Docker volumes:
| Volume | Must be backed up |
|---|---|
rememberstack_postgres-data | Yes: the database |
rememberstack_object-store-data | Yes: original files and derived artifacts |
rememberstack_forget-manifests | Yes, separately durable: never roll portable intent back with an older data backup |
rememberstack_app-state | Optional: working directories and debug captures |
Corpus folders in the object store can be reconciled; everything else in the first three volumes cannot. The database and the object store refer to each other, so back them up at the same moment.
A cold backup of the volumes
The simplest consistent backup stops the stack and copies the volumes:
docker compose stop
mkdir -p backup
for volume in postgres-data object-store-data forget-manifests app-state; do
docker run --rm \
-v "rememberstack_${volume}:/source:ro" \
-v "$PWD/backup:/backup" \
alpine tar -czf "/backup/${volume}.tar.gz" -C /source .
done
docker compose startKeep the backup/ directory and your .env together: a restore needs the
same deployment id and the same database and object-store credentials.
For a same-machine restore, retain the current forget-manifests volume.
Restore only the ordinary data volumes from the older backup, with the same
.env. On another machine, first provision the manifest volume from your latest
separately durable intent store; the copy captured with an older data backup can
be missing later forget requests.
docker compose down
# Keep rememberstack_forget-manifests and its latest accepted intent.
docker volume rm rememberstack_postgres-data rememberstack_object-store-data rememberstack_app-state
docker compose create
for volume in postgres-data object-store-data app-state; do
docker run --rm \
-v "rememberstack_${volume}:/target" \
-v "$PWD/backup:/backup:ro" \
alpine tar -xzf "/backup/${volume}.tar.gz" -C /target
done
docker compose up -d --wait
docker compose exec -T api remember ops graph-catalog ensureThe last command checks that the graph views and functions in the restored database match this release (Operating the pipeline).
A logical backup
pg_dump backs up the database while it runs:
docker compose exec -T postgres \
pg_dump -U rememberstack -d rememberstack --format=custom > rememberstack.dump
docker compose exec -T postgres \
pg_dumpall -U rememberstack --roles-only > roles.sqlUse your own user and database names from .env. The second file matters:
the migrations create a separate, restricted PostgreSQL role for SQL
queries, and roles are not part of pg_dump. Copy the object-store buckets
(remember-raw, remember-artifacts, remember-corpusfs) at the same
time with an S3 tool such as rclone sync. Restoring a logical backup means
creating the roles, restoring the dump into an empty database with
pg_restore, copying the buckets back, and running setup; the project
has not tested this path.
Hard-forget manifests
The forget-manifests volume holds portable, content-free intent for permanently
removed documents. Keep it independently durable and carry all accepted intent
into a restore. An older database or object bucket cannot undo that intent.
Before any API or controlled mount reopens, startup rematerializes every manifest,
scrubs restored authority, scans the corpus for the document's deterministic
suffix, erases its copies/uploads and retained versions, and regenerates shared
indexes. It repeats this even when PostgreSQL still says complete, covering
bucket-only, database-only and combined restores. A failure leaves admission
closed for retry. Other documents' originals and media remain in place.
The PostgreSQL + real MinIO canary exercises these independent data restores; transporting production backups, backup expiry and provisioning remain operator responsibilities. Stop and remount operator-managed FUSE sessions. A composing host must supply cache acknowledgements for sessions it controls, and K purge and rebuild capabilities when K history exists.