File formats and converters

Before RememberStack can read a file, the convert worker turns it into Markdown. Which converter handles which file is a table you configure: a map from MIME type to converter name. A stock deployment converts Markdown, plain text, HTML, Word (.docx), PowerPoint (.pptx) and Excel (.xlsx) with no API key. Everything else, including PDFs and images, is stored and waits until you add a route for it; the ingest response says so at once with "parked": "no_route".

The default table

With REMEMBERSTACK_SELFHOST_CONVERSION_ROUTES unset, the table is:

passthrough keeps the text as it is. It requires valid UTF-8; a file in another encoding fails conversion. markitdown runs in the worker with no network call (see markitdown).

PDFs and images are not in the default table: the converters that read them call a paid provider API with your own key. Add a route for them as shown below.

The MIME type is the one sent with the upload. The remember CLI, Python client and MCP ingest tool take it from the file extension, the same way on every Python installation for the formats a route can name (the table is in Ingest files), and fall back to application/octet-stream. Pass --mime (CLI) or mime= (Python) when the extension does not say what the file is. The match is exact: text/markdown and text/x-markdown are different keys.

Add routes

Set REMEMBERSTACK_SELFHOST_CONVERSION_ROUTES to a JSON object. It replaces the default table, so include the default routes again. This example adds PDFs and images:

# .env
REMEMBERSTACK_SELFHOST_CONVERSION_ROUTES={"text/markdown": "passthrough", "text/plain": "passthrough", "text/html": "markitdown", "application/vnd.openxmlformats-officedocument.wordprocessingml.document": "markitdown", "application/vnd.openxmlformats-officedocument.presentationml.presentation": "markitdown", "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet": "markitdown", "application/pdf": "mistral_ocr", "image/png": "image_ocr_description", "image/jpeg": "image_ocr_description"}

Then apply it with docker compose up -d. The API and the convert worker both read this variable. A route that names an unknown converter stops the convert worker at start with an error listing the four known names.

markitdown

markitdown converts in the worker process, with no network call and no cost. The image installs the markitdown library with the packages it needs for Word (.docx), PowerPoint (.pptx) and Excel (.xlsx); HTML needs none. It reads the text, headings, lists and tables of these files; it does not read text inside embedded images.

The image does not contain markitdown's PDF, audio or legacy Office (.doc, .ppt, .xls) packages, so a file of those types routed to markitdown fails conversion. Route PDFs to mistral_ocr instead.

mistral_ocr

mistral_ocr sends the whole file, base64-encoded, in one request to Mistral's /v1/ocr endpoint and turns the per-page result into Markdown. It keeps page structure, tables, headers and footers, embedded images and the provider's confidence scores as artifacts beside the Markdown. You bring the key; the call is billed to your Mistral account and recorded in the cost ledger.

Mistral rejecting a file (HTTP 400, 413 or 422) fails conversion at once. Other provider errors are retried.

image_ocr_description

image_ocr_description runs two calls on every PNG or JPEG: Mistral OCR reads the visible text, and a vision model on OpenRouter describes what the image shows. The Markdown has two sections, ## Visible text (OCR) and ## Visual description. Both calls must succeed. A finished call is saved, so a retry does not repeat it.

Set them in .env. Without the key, the convert worker refuses to start once a route names this converter.

Files no route accepts

An upload whose MIME type has no route is still accepted. The API stores the original bytes and creates the version, and its convert work is parked with the reason no_route. It uses no attempts and makes no model calls.

The ingest response says so immediately: its parked field is "no_route" (it is null when the version is not parked). remember ingest also prints a warning, and the MCP ingest tool tells the agent not to wait for readiness. Readiness shows the version's convert stage as pending.

After you add a route for that type and restart with docker compose up -d, release the parked work:

docker compose exec -T api \
  sh -c 'remember ops resume-no-route --deployment "$REMEMBERSTACK_SELFHOST_DEPLOYMENT_ID"'

The command prints {"released": [...]} with the processing ids it released. It releases only work whose stored MIME type the current table covers; the rest stays parked. See Operating the pipeline.

If the file was simply sent with the wrong type, send the same bytes again with a type that has a route (--mime text/markdown on the CLI, mime="text/markdown" in Python). The new type replaces the unrouted one and the parked conversion is released without an operator step.

Not supported yet

RememberStack does not ingest audio, video or web addresses yet. To add a web page, download it and send the HTML.