Filesystem views
The remember-corpusfs bucket is a live filesystem for readers. Each completed
source has one folder containing its original, exact converted Markdown, active
PageIndex, available source map and reader media. Publication runs automatically
through the ordinary llm worker class. It makes no model calls.
A newer stored or parked upload leaves the previous completed reading in place. A source with no completed reading has no corpus folder; its original remains available through the version-addressed API.
Mount the corpus
Use an S3 filesystem tool with read-only credentials and options to mount the configured corpus bucket. Bound its stat, listing and data caches. The bucket root is the filesystem root; separate raw and artifact mounts are unnecessary. RememberStack does not provision a FUSE mount or enforce local permissions. Provider access logging must include reads of corpus originals.
Layout
<corpus root>/
├── llms.txt
├── _index.md
├── pdf/
│ ├── _index.md
│ └── finance/annual-reports/
│ ├── _index.md
│ └── annual-report--9bf3e4c0c2d4861075a6ef82/
│ ├── _index.md
│ ├── document.json
│ ├── original.pdf
│ ├── document.md
│ ├── pageindex.json
│ ├── source_map.json
│ └── media/
└── …The fixed types are email, chat, image, audio, video, pdf,
spreadsheet, presentation, document, data and other. A document can
have an optional topic under its type. Large member tables use navigation pages
under a reserved ~/ subtree; pagination never moves document folders.
Start at llms.txt, follow type/topic indexes, then open a folder's _index.md.
document.json records the document, displayed version, representation,
inventory and deterministic publication identity. Pipeline manifests, extraction
state and historical readings are excluded. Plane K is a separate repository.
Resolve a folder through MCP search_documents with doc_ids, the SDK's
search_documents(filters=DocumentSearchFilters(doc_ids=(doc_id,))), or
POST /documents/search with filters.doc_ids. p3_path is relative to the
corpus root. It is non-null only when the selected version is served and its
nullable publication marker matches its current export inputs. Raw-only,
unpublished, updating, failed and historical results carry null.
served_version describes authority currency independently of publication.
Title, topic or type changes can move the folder. Re-resolve after a move;
only the suffix is stable within its path. Keep the doc_id; its suffix
is the first 24 hex characters of SHA-256 over the UUID's 16 bytes. For offline
lookup, substitute that suffix in:
find <root> -type d -name '*--<suffix>'Check doc_id and the displayed version in document.json before treating the
folder as a particular edition. The search can cost a full tree listing.
A matching asset directory beneath another document folder is not a separate
document: verify the first document-folder component and its document.json.
A suffix match can locate the served folder even when a temporal query selects
another edition or no edition is in force; it does not change that selection.
There is no public source_open operation for whole historical files yet.
The corpus is a sources-layer projection: switching claims or facts off does not change its layout. K remains a standalone repository and may be checked out separately.
Publication and repair
Each object is replaced completely, but the folder and tree are not atomic.
During an update, complete old and new files can coexist while the folder index
says updating; a move can briefly expose two folders. Check the ready folder's
publication identity before and after a read when consistency matters.
require_p3=True in readiness checks each
requested served version's current publication marker. It reports
corpus_publication_incomplete for missing or mismatching markers and
p3_version_not_served for a historical edition. An unrelated publication cannot
satisfy this check. version is the export digest for one ready requested version and null for
multiple versions. built_at and published_at are null.
Digest-verifying repair runs daily through the existing delayed-work ledger,
leases, retries and delivery. Configure
REMEMBERSTACK_SELFHOST_CORPUS_VERIFY_INTERVAL_S to a positive number of seconds;
its default is 86400. Setup seeds the first task and a successful verification
seeds its successor. A dead-lettered task remains visible for operator replay.
To verify and reconcile immediately:
docker compose --profile operations run --rm projectionsThis runs project --plane p3, repairs the same live tree and prints the number
of served documents reconciled. Matching digests avoid recopying originals and
media. Ordinary publication touches only the changed document and its navigation
ancestors.
Deletion, forget and restore
Normal document deletion removes that document's recorded corpus folders and repairs navigation before reporting success. A store failure reports failure and leaves durable retry work.
Before coordinating hard-forget on operator-managed processes, stop ordinary writers and wait for their outstanding storage calls. An expired work lease is not proof that an issued provider request has stopped.
Hard-forget closes admission, drains work, erases every matching suffix folder,
unfinished upload and retained object version, then regenerates navigation from
scrubbed database authority. It preserves other document files in place. When a
provider does not implement multipart-upload listing, purge records
provider_managed coverage and logs a warning; unfinished-part cleanup follows
that provider's policy. Published-object absence is still verified.
Before restoring traffic, the startup gate replays every portable manifest, including ones previously marked complete. Keep that intent outside the ordinary restore set. A replay failure keeps admission closed. Engine-controlled sessions must stop serving and acknowledge cache erasure; operator-managed FUSE mounts must be stopped and remounted for forget. See hard-forget recovery.