SI Agent News
DevelopingClaudeOpenAIAgents

Agents now keep memories between sessions, and almost nobody checks them

A developer argues that agent memory tools can't say who wrote a fact, what it replaced or whether it was altered, and has built a tool to fix that.

27×score inflation the author found from mixed scoring in memory benchmarks

The short version

Why it matters

When an agent acts on remembered notes, a wrong or outdated note can quietly steer many later answers. Knowing who wrote a memory, and being able to see and edit it, is the difference between a helper you can supervise and one you can only hope is right.

The question nobody is asking

AI agents are starting to keep memory between sessions. Tools such as mem0, Zep, Letta and Cognee save what an agent has "learned" so it can use that knowledge next time. A post on dev.to argues that this raises a gap the industry mostly ignores.

The author names three questions these tools don't answer. Who wrote a given fact, meaning which agent in which session? What did that fact replace, for instance when the agent changed its mind? And could someone alter it silently, whether on purpose or by accident?

If an agent's decisions rest on remembered facts, the author says, these are governance questions. In the post's words, "governance without evidence is just hope."

How agent memory works

The post is an opinion piece, so it doesn't describe the inner workings of those products. What follows is a general picture, not a description of any one tool.

After a conversation, a system pulls out statements worth keeping, such as "this customer wants invoices in euros," and saves them in a database with a search index. In the next session, the relevant notes are fetched and placed into the prompt (the text the model reads) before it answers.

The model itself remembers nothing. It reads notes. That makes memory a kind of prompt text that you did not write, and which may have been created in a session you never saw.

What the author built

The author's answer is MemTether, a local-first memory hub, meaning it keeps data on your own machine. Its design follows the three questions.

The post also compares this with plain markdown files, which overwrite old notes, record no author, and let the last write win when two agents save at once. The author says you can export to markdown at any time but should treat the governed database as the source of truth.

How much to trust the claims

This is a builder describing their own project, so read it as a proposal. The author says all results come from their own test harness, with per-question records committed to the project's repository. That makes them checkable, but it isn't independent testing.

One detail stands out. The author says they documented a 27× score inflation caused by mixed denominators in memory benchmarks, which is why they report each metric under the exact scoring rule that produced it. In plain terms, comparing memory tools is easy to get wrong when different tests count different things.

The post also says the tool works with Claude Code, Codex, Cursor and 20 other agents through file-level pointers or MCP, a standard way for agents to connect to tools. It gives no adoption figures, and we have no outside review of it.

What this means in practice

You don't need MemTether to take the point. Memory is a place where errors persist. A wrong note the model misread can repeat every session, and a stale one can go on shaping answers after it stops being true. A private detail stored once can reappear where it shouldn't.

If you build agents, the post suggests a clear minimum. Record when each memory was written and from what source. Keep what a correction replaced. Let users see and edit what is stored.

If you only use agents, the equivalent is a habit: know where your tool keeps its memory, read it from time to time, and delete anything outdated or sensitive.

What to watch

Try it

Find where your AI tool stores its memory, read it, delete anything stale or sensitive, then ask the agent what it knows about you and compare.

Sources and links

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

Listen to the AI Wire headlines

Listen to the AI Wire

Listen to our daily AI shows every morning: on your commute, in the shower, wherever you are

Listen to the daily shows

First reported by dev.to · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

More AI news