RememberStackremember.dev/docs

Documents, versions and sources

An agent's memory is only as trustworthy as its knowledge of where each piece of knowledge came from. If the spec for the billing migration changes three times, you need to know which text said what, and when. If the same meeting notes are sent twice, you do not want them counted twice. If a file is renamed, it is still the same file.

RememberStack handles this at the very first step, before any model reads your text. Every file you send becomes a version of a document, and every document is tied to the place it came from, its source.

Document and version

A document (doc_id) is one logical file over its whole life: the billing migration spec, one meeting transcript, one chat thread. It has a stable identity that does not change when the content changes.

A version (version_id) is one snapshot of that document's bytes, as you sent it at one moment. Versions are append-only: a new upload with different bytes adds a new version and never edits an old one. Each version records:

  • the SHA-256 hash of its bytes (content_hash),
  • the time the source says it was last modified (source_modified_at),
  • an optional revision marker from the source system (source_version_ref),
  • an optional permanent name you give it (version_key, see Editions in force),
  • who ingested it (ingested_by, see below).

Everything RememberStack later derives from the file (its converted text, sections, chunks, claims) hangs off one specific version. That is what lets an answer point back to the exact text that produced it, even after the document has moved on.

The bytes themselves are stored once per content hash, as a content object. If two documents contain identical bytes, or a document is changed and later changed back, the bytes are not stored or converted a second time.

Source: source_kind and source_ref

A source tells RememberStack which real-world thing a document stands for. You give it as a pair:

  • source_kind: the class of source, a short name you choose, for example notes, drive, slack, agent.
  • source_ref: the stable identifier of this item inside that class, for example a file path, a Drive file ID, or a thread ID.

The pair is the document's identity. Send the same source_kind and source_ref again with different bytes and you get a new version of the same document. Send a new source_ref and you get a new document.

The two values are always given together. A source_ref alone is ambiguous (42 could be a ticket, a message or a file), so the engine refuses one without the other: the HTTP API returns 422 and the Python client raises ValueError with source_kind and source_ref must be supplied together.

from datetime import datetime, UTC
 
import remember
 
with remember.Client() as memory:
    version = memory.ingest(
        "specs/billing-migration.md",
        source_kind="notes",
        source_ref="specs/billing-migration.md",
        source_modified_at=datetime(2026, 3, 4, 15, 0, tzinfo=UTC),
    )
    print(version.doc_id, version.version_id, version.created)

Note

Choose a source_ref that stays the same for the life of the item. A path is fine if files do not move; a Drive file ID or a database key is better if they do. Renaming is metadata; a new source_ref is a new document.

Ingesting without a source

You can leave out both values. RememberStack then treats the upload as a one-shot file: source_kind becomes upload and source_ref becomes the content hash, so the document's identity is its bytes. Sending the same bytes again is a no-op; sending edited bytes creates an unrelated document.

Use this only for files that will never change. For anything an agent will write to again (notes, a running log, a spec), give a source so that later writes become versions of one document. Without a source you also cannot set source_modified_at, source_version_ref, living mode, a version_key or an effective period; the engine rejects them with source timestamps, revisions, living mode, version keys and effective periods require source_kind/source_ref.

source_modified_at becomes the said-on time

source_modified_at is when the source says the content was written or last changed: the timestamp of the meeting, the send time of a message, the last-modified time of a file. It must be timezone-aware UTC.

RememberStack copies it onto every claim extracted from that version as asserted_at, the time the source said it. This is one of the three clocks described in Time. It also anchors relative dates: when a transcript from 2026-03-04 says "last Friday", the extractor resolves that against the document's date.

If you leave it out, asserted_at is empty and relative dates can be resolved only against a date written inside the document itself.

source_modified_at is fixed once the version exists. Re-sending identical bytes with a different timestamp does not change it, because the claims were already extracted against the original value.

Unchanged passages keep their date

A new version usually has a new source_modified_at, but most of its passages are the same words as before. Each passage (chunk) therefore records its own text origin time: when its words were first written in this document, as far as the engine can tell. A passage of a new version that is unchanged, with the same neighbouring text, from a passage of an earlier version keeps that earlier passage's date, as long as the date is not later than the new version's own. Such a passage is not read again: its claims and embeddings are reused, and they keep the asserted_at they were stamped with. A changed passage, or one whose neighbours changed, takes the new version's date and is read afresh.

The rule only ever looks back to earlier dates. If you ingest the 2026 edition of a document first and the identical 2024 edition later, the 2024 passages take 2024 and are read afresh; nothing already stamped is re-dated. Passages of deleted versions are never drawn on, and the date is recorded once and never changes. chunks_live.text_origin_at in the query space shows it. Passages written by an earlier release have no recorded origin, so the first new version of each document after an upgrade is read in full once.

Re-ingesting the same bytes: created=false

Every ingest returns an IngestedVersion:

FieldMeaning
deployment_idThe deployment that stored it.
doc_idThe document (stable across versions).
version_idThe version this call produced or matched.
content_hashSHA-256 of the bytes.
createdtrue if this call created a new version; false if the bytes match the document's latest version.
mimeThe MIME type conversion uses for these bytes.
titleThe document's title, set by its first ingest.
versioning_modesnapshot or living, set by its first ingest.
version_keyThe version's key, or null.

When the bytes match the document's latest version, and you send no version_key the document has not used yet, nothing new is created and no processing is scheduled. You get the existing version_id back with created=false. This makes ingest safe to retry and safe to run from a loop that re-sends everything: unchanged files cost nothing.

If you pass a new source_version_ref with unchanged bytes, the version's revision marker advances so a sync process does not fetch the same revision forever. Nothing else about the version changes.

Bytes that match an older version (the file was changed and then changed back) create a new version. The document moves forward; it never silently jumps back to an old snapshot.

Bytes sent again after the document was deleted also create a new version (created=true), and it is processed from the start.

Editions in force

Some documents come in editions that each apply for a period, such as a company's travel policy. Two things help with them.

A version key (version_key at ingest) is your own permanent name for one version, such as edition-2 or a publisher's edition id. It is unique within the document and never moves: a key the document has not used yet always creates a new version, even for bytes identical to the latest, and a key that already names a version cannot be reused for another. It is how a pinned reference names exactly one edition. It is separate from source_version_ref, which moves when a sync sees a new revision.

An effective period (effective_from, optionally effective_until) says when a version's text is in force. Reads with a time scope then select the edition in force for that time rather than the newest one. Periods need a snapshot document. See Time and Read a document as of a date.

Section keys

A heading in Markdown can carry a stable key in a trailing attribute block, in the Pandoc and kramdown style:

## 4. Per-diem allowance {#per-diem}
### 5. Scope {#scope .note}

The #identifier becomes the section's section key and the whole {…} block is removed from the stored title; classes (.note) and name=value items are accepted and ignored. A key is 1 to 200 characters from A–Z a–z 0–9 _ . : / -, and a block may hold at most one. A block that breaks these rules is left in the title, and the heading gets no key. The block must end the heading line, on # headings and underlined headings alike.

Keys come only from the source text: RememberStack never invents one, and sections it had to find without headings have none. Within one version a key is unique; if two headings carry the same key, the first keeps it and the later one gets none.

Because the key stays the same when the text around it changes, it names "the same section" across versions. section_history lists what a key held in each version and whether it changed, and supplied references can point at a key in another document. Sections stored by an earlier release get their keys from their stored headings when setup runs after the upgrade, without reprocessing; until then they read as not_indexed.

Writing keys is a new way of reading headings, so the first new version of each existing document after the upgrade is read in full once.

Finding a document

Every version records the names it was stored under: its file name, the title the caller declared, and the source path. Sending the same bytes again under another name creates no version, but the new name is recorded next to the old ones. search_documents (POST /documents/search, MemoryClient.search_documents, remember documents search, or the search_documents MCP tool) finds documents by any of those names, even partial or misspelled, by their text, and by general metadata: format family, authors, recipients, dates, language and thread.

Each result is a document judged by one version, its current one by default, and it reports that version's status, so a file still being processed can already be found by name. Say John uploaded billing-migration-draft.md and later sent the same file as billing-migration.md: searching for either name finds the one document. See POST /documents/search.

Deleting a document

Deleting a document removes it from the memory: every version of it, in one call (DELETE /documents/{doc_id}, MemoryClient.delete_document, remember documents delete, or the delete_document MCP tool).

Say Dana uploaded a draft of the billing migration plan that says "the migration finishes in March", and John's status note says the same. Dana deletes the draft. From then on:

  • the draft is gone from GET /documents, search, facts, the graph and SQL queries;
  • its claims stop counting as evidence (their currency ends with reason version_deleted);
  • "the migration finishes in March" stays believed, because John's note still supports it; it now has one supporter instead of two;
  • a fact that only the draft supported is closed, and the closure is recorded as a retraction (retracted_source_removal), exactly as when a living document drops a passage.

Deleting is not erasing. The claims, the stored original and the record of what changed stay in the deployment as history, which is why a deletion can be audited. Erasing a document's bytes and every trace of it is a separate, heavier operator operation that is not offered through the API.

If you send the same file again later, it is added back as a new version and processed like a new document. The facts that were closed stay closed; its new claims support facts in the usual way.

A deletion is all or nothing. A document that was still being processed when you deleted it publishes no further claims, and anything it had already produced is never visible and is retired when its version reaches the reconcile stage. Details and errors: DELETE /documents/{doc_id}.

Document statuses

Each version carries a processing status for its early stages:

StatusMeaning
ingestingAccepted; bytes are being recorded.
convertingWaiting for or running conversion to text.
structuringConverted; sections are being detected.
readyConverted and structured.
failedConversion or structuring failed; the version's error says why.
deletedRemoved.

GET /documents lists documents newest first, reports the newest version's status, and adds a serving flag that is true when any version of the document has reached ready. A document whose newest upload failed can still be serving an older version. See Ingest, readiness, documents.

Who ingested a version

A version can record the actor that created it (ingested_by), as one of three kinds:

  • user: a person,
  • api_credential: a machine credential that a person created,
  • service: the deployment's own automation.

The three are never collapsed: activity by a token is not attributed to the person who minted it. The actor's identifier is opaque to the engine and treated as personal data that can be erased.

Attribution is sent in the X-Ingest-Principal-Kind and X-Ingest-Principal-Ref headers of POST /ingest. The engine honours them only when the deployment declares its network perimeter trusted and the caller holds full write authority; otherwise it ignores them without failing the upload. Attribution is set once, when a version is created. Re-sending the same bytes as someone else changes nothing.

The Python client does not send these headers, and no read route returns the recorded actor yet.

Where to go next