SIAgentNews.com
← All news

LMLineage traces each sentence an AI agent writes back to its sources

A new preprint describes a tool that links every line an agent produces to the input text behind it, then uses that to trim stale material from the agent's memory.

· First reported by arxiv.org

1sper-turn attribution time on a single GPU

The short version

  • The system works from outside the model, so it needs no access to the model's internals and no extra runs of it.
  • It gives each output sentence a confidence-scored link to the input passages that support it.
  • Tests used 71 hand-labeled sentences plus 90 turns of multi-turn sessions across three models.
  • Attribution takes 1s per turn on one GPU and stayed steady up to 40 turns.
  • Drift detection caused 0% false compaction, but it misses cases when attribution is wrong.

Why it matters

Agents now read files, web pages and emails before acting, and it is hard to tell which of them shaped an answer. A tool that records this could make mistakes easier to investigate and agent memory easier to keep tidy. For now it is a research result, not a product you can use.

Tracing answers back to sources

As arxiv.org shows in a new preprint, a system called LMLineage tackles a plain question: when an AI agent writes something, which of the documents it was given did that sentence come from? The authors argue that explaining agents should go beyond studying a model's internal weights. It should also cover how the agent reasons over everything fed into it.

LMLineage links each output sentence to the input passages that back it up and attaches a confidence score to each link. It runs out-of-band, meaning beside the agent rather than inside it. It needs no view into the model's internals and no additional runs of the model.

Cleaning up memory that has gone stale

The authors use those links for a second job. Most agent tools wait until the context limit (the cap on how much text a model can hold at once) is reached, then summarize everything. LMLineage instead notices when documents used earlier in a session have stopped being cited, which the paper calls drift.

It then shrinks only those documents. The user's own instructions are kept word for word. The paper also describes a case study on tracing an incident and cleaning an agent's memory.

What the tests showed

The evaluation had two parts. One was 71 sentences labeled by hand, drawn from 10 Chinese web question-answering cases. The other was 90 turns of separately gathered multi-turn sessions using gpt-4o-mini, qwen3.8-27b and deepseek-v4-flash.

  • Matching small windows of text by embedding similarity (a way of comparing meaning, not exact words) fit the way language models paraphrase better than matching whole documents or exact words.
  • A greedy shortcut, which takes the best-matching window without exhaustive checking, beat a full line-by-line search on speed and on average accuracy.
  • Each turn took 1s on a single GPU and stayed steady as sessions grew to 40 turns.
  • Drift detection was cautious, with 0% false compaction, but its recall can only be as good as the attribution underneath it.

Limits and what comes next

The test sets are small, and the labeled cases are Chinese web question answering, so it is unclear how well the results carry over to other kinds of agent work. The authors also mention a companion patent application covering how the lineage is built, which may affect how freely others can use the method.

What to watch

  • Will the results hold on larger, more varied tests beyond Chinese web question answering?
  • Will the patent application limit who can build on this method?

Try it

If you run agents, log which documents or pages were in context for each answer. A simple record of this kind is a start toward the traceability the paper describes.

Sources and links

First reported by arxiv.org. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

More AI news