gulupgulup/tiered-memory-agent

MemGPT-lite

An LLM agent that remembers you across sessions, built on the MemGPT paper's tiered-memory design.

[Live demo](https://huggingface.co/spaces/gulupgulup/tiered-memory-agent) · [Code](https://github.com/gulupgulup/tiered-memory-agent) · Paper: MemGPT (arXiv 2310.08560)

Demo

Four moments from a chat with the deployed agent. Tool calls happen server-side and don't appear in the chat bubbles, so each caption names what fired.

Panel 1 — Establishing facts

[image]

The user introduces themselves. Behind the scenes the model calls core_memory_append three times to write name, background, and current goal into the persistent <human> block, then replies with substantive advice grounded in what was just saved. Nothing about this conversation is in a prompt template — the writes are decisions the model made in response to the user's message.

Panel 2 — Destructive-edit gating with strict-undo

[image]

A correction arrives — the user clarifies they're actually targeting applied research scientist roles rather than engineering. The model attempts core_memory_replace, which is the one destructive tool in the suite, so the turn pauses on a confirmation card before touching stored state. (Screenshot pre-dates a fix for the missing tool-detail line — see `docs/demo_transcript.md`.)

[image]

We reject deliberately, to show what happens next. Under strict-undo semantics the model doesn't just skip the write — it also doesn't verbally comply with the rejected value on the current turn. The reply reverts to the original role and the <human> block stays intact. In normal use the user would approve; the rejection here is chosen because it's the interesting frame.

Panel 3 — Writing a long-form note

[image]

The user debriefs a phone screen. The model routes this to archival storage via archival_insert — long-form observations don't belong in the always-in-prompt core memory block, so they get embedded and indexed for later semantic retrieval instead. No confirmation card fires: archival writes are cheap and easy to search out later, so the system prompt's conservative save-heuristics are the only gate.

Panel 4 — Cross-session recall

[image]

After clicking New Session the FIFO event queue is empty — the model has no in-context memory of the prior conversation. When asked what the last interviewer covered, it calls archival_search, retrieves the note written in Panel 3, and answers with the retrieved specifics. This panel carries the memory-system-works argument by itself.

A full transcript with per-panel behind-the-scenes narration is in `docs/demo_transcript.md`.

What this is

MemGPT-lite is a minimal from-scratch implementation of the tiered-memory architecture described in the MemGPT paper (Packer et al., 2023). The paper's core idea is that a fixed-context LLM can be given the illusion of unbounded memory by borrowing virtual-memory paging from operating systems: the model gets a small set of tools that let it read from and write to external storage tiers, and the surrounding runtime decides what's in the prompt on any given turn.

This repo implements the memory-tier taxonomy, the tool-calling loop, and cross-session persistence. Six tools give the model surgical control over three storage tiers — always-in-prompt core memory for identity facts, keyword-searchable recall storage for past messages, and semantically-searchable archival storage for long-form notes. What this repo does not implement is the paper's queue-eviction and recursive-summarization behaviour (the "true" MemGPT paging loop), timed events, and document ingestion — those are scoped out for a future iteration. Honest scope disclosure lives in `docs/differences_from_memgpt.md`.

Architecture

flowchart LR
    User([User])
    LLM["LLM<br/>Gemini via Google ADK"]

    subgraph Memory["Memory tiers"]
        direction TB
        Core["Core<br/>&lt;persona&gt; + &lt;human&gt;<br/>always in prompt"]
        Recall[("Recall<br/>SQLite + FTS5<br/>keyword search")]
        Archival[("Archival<br/>SQLite + FAISS<br/>semantic search")]
    end

    User <-->|conversation| LLM

    Core -.->|injected every turn| LLM
    LLM -->|"core_memory_append<br/>core_memory_replace (HITL)"| Core
    LLM -.->|"mirror-write<br/>user turns"| Recall
    LLM -->|recall_search| Recall
    LLM -->|"archival_insert<br/>archival_search"| Archival

The six tools map one-to-one onto the memory tiers:

Core memory is the only tier that's always in the prompt — the <persona> and <human> blocks are injected into the system instruction on every turn via a before_model_callback. Recall storage mirrors every user message to a SQLite FTS5 index via an after_agent_callback; the model queries it with keywords when it needs to recover something from earlier in the conversation history. Archival storage is the model's own long-form notebook — it decides what's worth saving via archival_insert, and later retrieves notes by semantic similarity with archival_search using L2-normalized 384-dim vectors from sentence-transformers/all-MiniLM-L6-v2 indexed in FAISS.

The one destructive operation — core_memory_replace — pauses the turn on a confirmation card before touching stored state. Everything else runs without a gate; the conservative save-heuristics in the system prompt are the only thing standing between the model and a noisy archival table, which is a deliberate trade-off (discussed in the next section).

Design decisions

Three architectural choices shaped this project. Each was a deliberate trade-off rather than a default.

Google ADK over LangChain or a from-scratch framework. ADK provides the session lifecycle, event log, and function-calling loop out of the box, which meant the interesting engineering could concentrate on the memory layer rather than on infrastructure. The trade-off is coupling: session.events is the FIFO queue, session.state with the user: prefix backs core memory, and DatabaseSessionService handles cross-restart persistence — all conveniences that come with a specific framework's idioms and a specific vendor's evolving API. A from-scratch build would have taken twice as long and produced no more portfolio signal; LangChain's abstractions would have moved the coupling to a different framework with less alignment to the Gemini stack. ADK was the shortest path to a working three-tier memory system that a reviewer can actually reason about.

SQLite + FAISS over a hosted vector database. Recall storage lives in SQLite with FTS5 for keyword search; archival storage pairs SQLite for canonical text with a FAISS IndexIDMap(IndexFlatIP) for 384-dim cosine-similarity search. Nothing here talks to a network. The trade-off is scale: IndexFlatIP does exact brute-force search that stops being appropriate somewhere past 100k vectors, and FTS5's BM25 ranking is coarser than what a purpose-built vector DB would provide. At demo scale — hundreds of stored items, single-user conversations — both trade-offs are invisible. What's visible instead is that the entire memory system fits inside the repo, deploys to a free Hugging Face Space without a managed-service dependency, and can be inspected with sqlite3 on the command line. Legibility mattered more than headroom.

Sync tools with an async Runner. ADK's Runner is async, but five of the six tools (send_message, core_memory_append, core_memory_replace, archival_insert, archival_search) are defined as synchronous functions because their underlying work is blocking — SQLite writes, FAISS lookups, in-memory state edits. Only recall_search is async because it awaits ADK's search_memory. This mismatch is intentional: writing async def on tool bodies that do no awaiting is a Python smell that suggests concurrency where none exists. ADK dispatches sync and async tools transparently, so the shape follows the I/O honestly. The consequence for future work is that if any tool later gains network I/O (e.g. a hosted-embedding call), it flips to async and the others stay as they are.

Known limitations

Not implemented from the MemGPT paper. The queue-eviction and recursive-summarization loop that the paper describes as the "true" MemGPT paging mechanism is out of scope for this version — session.events grows unbounded and there is no summary at index 0. Timed events, request-heartbeat control flow, and document ingestion are also unimplemented. Full scope disclosure with paper-section-by-section mapping is in `docs/differences_from_memgpt.md`.

Archival dual-write is not crash-safe. The archival tier writes to SQLite first and then to the FAISS index. A process crash in the microsecond window between the two writes would leave the two stores inconsistent. At demo scale this window never fires; a production version would use SQLite WAL mode with FAISS treated as a rebuildable cache.

Recall storage covers user messages only. By design, the recall ingest filter accepts only events with role in ("user", "model") and non-empty content.parts[0].text. The model's user-facing replies are authored inside send_message's function-call arguments rather than on a text-typed event, so they don't match the filter and never enter recall. This is a deliberate v1 trade-off — widening the filter would let longer, vocabulary-varied agent replies dominate BM25 rankings over shorter user turns. The escape hatch is documented in `docs/differences_from_memgpt.md`.

Demo caps on the hosted Space

The public Hugging Face Space applies four hygiene caps to keep costs bounded against anonymous traffic. These are not correctness constraints — they exist because the Space is open to the internet without authentication, and they're the first thing to raise or remove when self-hosting.

To self-host without caps: fork the repo, edit the four values above (or raise MEMGPT_MAX_OUTPUT_TOKENS via env), and deploy. All four are single-line changes.

Quick start

Install

With `uv` (recommended):

git clone https://github.com/gulupgulup/tiered-memory-agent.git
cd tiered-memory-agent
uv sync

With `pip`:

git clone https://github.com/gulupgulup/tiered-memory-agent.git
cd tiered-memory-agent
pip install -e .

Configure

Copy the environment template and add your Gemini API key:

cp .env.example .env
# then edit .env and set GOOGLE_API_KEY

Get a key from Google AI Studio — the free tier is enough for demo use.

Run

uv run python -m app.gradio_app
# or: python -m app.gradio_app

The Gradio interface opens at http://localhost:7860. Introduce yourself in the first turn to seed core memory, then keep chatting — memory persists across restarts.

Configuration reference

Behavioural knobs live in config.yaml:

Runtime knobs live as environment variables:

Scope disclosure

This is a scoped implementation of the MemGPT paper, not a full reproduction. The tier taxonomy, tool loop, and cross-session persistence are all in; the queue-eviction and recursive-summarization mechanism the paper describes as "true" MemGPT paging is not. A section-by-section mapping of what's in, what's approximated, and what's out lives in `docs/differences_from_memgpt.md` — it's the honest read on what this repo does and doesn't do relative to the paper.

Future work

• —Queue eviction and recursive summarization. The paper's §2.2 mechanism for compacting the FIFO queue at a token threshold, replacing the evicted batch with an LLM-generated summary at index 0. The biggest single gap between this repo and the paper.
• —`request_heartbeat` control flow. Explicit tool-chaining signal from the paper, letting the model queue further tool calls without waiting for a user turn. This repo treats every tool call as an implicit chain until send_message fires.
• —Document ingestion mode. Bulk-load a corpus into archival storage at startup rather than requiring notes to arrive through conversation.
• —Multi-agent extension. Delegating memory-management operations to a second agent, closer to the Letta reference implementation's architecture.

References

• —MemGPT paper — Packer et al., 2023. arXiv 2310.08560.
• —Letta — the reference implementation of MemGPT (formerly the MemGPT project itself). github.com/letta-ai/letta.
• —Google Agent Development Kit — google.github.io/adk-docs.
• —`sentence-transformers` — sbert.net.
• —FAISS — faiss.ai.

License

MIT — see `LICENSE`.