Live now · Get your API key

Agent memory

An experimental, opt-in memory system that recalls and captures context across turns, stored workspace-local in SQLite, off by default, inspectable and deletable.

Agent memory is Ambient Desktop's experimental system for keeping useful context across turns and sessions. It is workspace-local, off by default, and fully inspectable.

Ambient Desktop settings and workspace.
Each turn, Ambient recalls relevant prior context before the model answers, then captures the turn back into a workspace-local store.

Memory that lives outside the transcript#

Chat context is fragile: once a conversation is trimmed or a session ends, what the agent "knew" is gone. Agent memory keeps useful context in a durable store instead. Before each turn it recalls what is relevant; after each turn it captures what happened, so later work builds on earlier work without you re-explaining.

The engine is adapted from the open-source TencentDB-Agent-Memory project (TencentCloud), whose four-layer pipeline distills raw conversation into reusable facts, profiles, and task structure. Ambient Desktop wraps that engine behind its own host adapter, runs it workspace-local, and keeps it off by default while it is in development.

How a turn uses memory#

Memory hooks into the agent loop at two points: a read before the model runs, and a write after it finishes. The store itself stays on your machine.

Agent Memory Loop diagram. Each turn runs User message (the turn begins), Recall (search prior memory), Enriched prompt (context prepended), Assistant reply (the model responds), and Capture (record the turn). Recall reads from and Capture writes back to a workspace-local memory store: local SQLite plus optional vectors in .ambient, off by default, holding four layers L0 raw turns, L1 facts, L2 profiles, and L3 scenes.

Agent Memory Loop: each turn recalls relevant prior context before the model answers, then captures the turn back into a workspace-local store.

  1. Recall, before the model answers#

    When memory is enabled for a thread, Ambient Desktop searches the store for context relevant to your message and prepends a bounded set of matches to the prompt, capped in size so it informs the model without flooding it.

  2. Respond#

    The provider answers with that recalled context in view. Routing, approvals, and every other safeguard work exactly as they do without memory.

  3. Capture, after the turn#

    The completed turn is written back to the store as a raw record, and higher-level extraction distills it into structured memory for future recall.

The four layers#

The adapted pipeline organizes memory into four layers, from raw conversation up to task structure. Higher layers make recall sharper than a plain transcript search.

RAW → DISTILLEDL3Scenestask- and scene-level structure — sharper, situation-aware recallL2Profileslonger-lived conversation and preference profilesL1Factsstructured, reusable facts — the point, not the paragraphL0Raw turnsthe verbatim record of each exchange — the source of truth

A four-layer memory stack, distilled from the bottom up. L0, Raw turns: the verbatim record of each exchange, the source of truth. L1, Facts: structured, reusable facts, so the agent recalls the point rather than the paragraph. L2, Profiles: longer-lived conversation and preference profiles. L3, Scenes: task- and scene-level structure for sharper, situation-aware recall. An axis on the left runs from raw at the bottom to distilled at the top.

The adapted pipeline organizes memory into four layers, from the raw record at the base up to task-level structure: higher layers make recall sharper than a plain transcript search.
Layer What it holds
L0: raw turns The verbatim record of each captured exchange; the source of truth everything else is derived from
L1: facts Structured, reusable facts extracted from conversation, so the agent recalls the point rather than the paragraph
L2: profiles Longer-lived conversation and preference profiles that persist across many turns
L3: scenes Task- and scene-level structure that ties related work together for sharper, situation-aware recall

Turning it on, staying in control#

Agent memory is experimental and off by default. You opt in deliberately, and you can scope it down to a single thread.

  • Enable the system. Memory is gated by an experimental flag and a Settings switch, so it never starts capturing anything until you turn it on. Settings search finds it alongside the other provider, permission, and runtime configuration.
  • Per-thread control. Each thread carries its own memory toggle. Keep most chats memoryless and enable memory only where continuity helps.
  • Optional local embeddings. Semantic recall can use a local embedding model run through Ambient Desktop's managed llama.cpp runtime, so it stays local-first and no content is sent to the cloud to be remembered.
  • Inspect and delete. Built-in tools let you search, inspect, edit, and delete stored memories from chat. Memory you can see is memory you can correct.

Where your memories live#

Privacy is a design constraint here, not an afterthought. The store is workspace-local: a SQLite database (with optional local vectors) under the workspace's .ambient directory. It is not shared between workspaces and is not synced to a cloud service in this phase. Because you can list and delete entries, and because the whole system is opt-in, you stay in control of what is retained.

What ships today, and what is still planned#

The local pipeline is functional today, behind the experimental flag:

  • Recall before a turn and capture after it.
  • The L0–L3 layers and the extraction that populates them.
  • Optional local-embedding semantic search.
  • Short-term offload of large tool outputs.
  • Inspect, edit, and delete for stored memories.

Not here yet:

  • A cloud-backed vector store.
  • Memory shared across workspaces.
  • Graduation out of experimental status.

The building blocks work today; memory stays opt-in and behind the flag until the system has proven itself, which is why it is not on by default.

FAQ#

Is memory on by default? No. It is experimental and off by default, gated by a flag and a Settings switch, with a per-thread toggle on top. Nothing is captured until you opt in.

Does my conversation get sent somewhere to be remembered? No. The store is workspace-local SQLite under .ambient, and semantic recall can run on a local embedding model. There is no cloud memory backend in this phase.

Can I see and remove what it remembered? Yes. Built-in tools let you search, inspect, edit, and delete stored memories directly from chat, so you can audit and correct the store at any time.

How does this relate to durable goals? Durable goals persist the plan and evidence for one objective; agent memory persists reusable context across many turns and tasks. They complement each other.

Keep going#