Why AI can't remember yesterday: lessons from a Pong game frozen mid-play

A frozen frame is not enough

The essay, written by Hugo of Robonaissance and translated by TechFlow, opens with the classic game Pong. Freeze it on any frame and you see two paddles, a ball and the score. You cannot say where the ball is heading, and a machine shown the same still image cannot either.

The piece separates two ideas. An observation is what the sensor catches in one instant. The state is the fuller picture the agent needs to choose its next move. In Pong the two come apart, because direction of travel is a fact about the last few moments, not about any single one.

Four frames as a stopgap

The essay then turns to DQN, the agent DeepMind researchers led by Volodymyr Mnih published in Nature in 2015. It learned dozens of Atari games from the screen alone. Its authors flagged the problem early: from one screen, many different situations look identical. They called this perceptual aliasing.

A ball flying toward your paddle and a ball flying away can produce the same picture. A system that only sees the picture must treat them as one case, and if they call for opposite moves, it will get one of them wrong.

The fix was simple. Instead of one screen, DQN receives a stack of recent frames, 4 by default, though 3 or 5 also worked. Motion becomes visible as a change in position. The paper did not call this memory; it was just a wider input.

The hard edge of the window

That wider input has a firm limit. Researchers Matthew Hausknecht and Stone pointed out in 2015 that a game requiring the player to remember events from more than four screens back is beyond DQN. Anything older never reaches the system at all.

The essay generalizes this. A system that acts over time must carry something from its past into the present, but never the whole past. It carries a summary, and what goes into it depends on what the system needs to predict. Whatever the summary leaves out cannot be used later.

What this means for chatbots

The essay's framing, as TechFlow presents it, extends this to large language models, which can hold a vast amount of human knowledge in a fixed form but cannot carry recent experience with them. In practice, do not assume an assistant knows last week's pricing or yesterday's meeting. Give it the current facts, and be wary of any product that claims it remembers you without saying what it stores.