The sleep framework: A new way for LLMs to retain knowledge in continual learnin

Google Research is experimenting with giving language models something resembling a sleep cycle: models operate normally during a “wake” phase, then periodically consolidate short-term information into longer-term parametric memory.¹

The work targets catastrophic forgetting, a longstanding continual-learning problem where acquiring new knowledge can degrade previously learned capabilities.² During the proposed sleep phase, the model expands selected memory capacity, distills useful information into it, and generates synthetic “dreams” for further self-improvement.³

Google reports results across factual learning, translation, reasoning, and extremely long-context tasks. The full research goes much deeper into how consolidation and dreaming work, and what would have to change for deployed LLMs to continually learn rather than remain largely frozen after training.