A visual research essay
LLMDream-BasedLearning
Can a conversation become
something a model learns?
and memory consolidation.Read the essay
Abstract
Language models do not normally update their parameters during a conversation. Context windows and retrieval can make earlier information available when answering, but they do not themselves train that information into the model.
This report explores a prototype of “Dream-Based Learning”: an offline cycle that generates possible future conversations from recent interactions, then trains on a mixture of those examples, recalled examples, and grounding data. In a small experiment over six simulated cycles, the model recalled selected user details after the context was cleared. The aim is to investigate continual learning through synthetic rehearsal.
The memory problem
Information in a context window is available for the current interaction. Once that context is removed, it is no longer part of the prompt. Storing or retrieving it elsewhere is different from changing the model’s parameters.
Training on new conversations can teach a model new information, but it can also cause overfitting or interfere with earlier capabilities.
Here, “sleep” is a name for an offline learning phase. The model processes recent experiences into synthetic examples of how those facts might be used later. We then fine-tune with a mixture of new examples, earlier examples, and general-knowledge data to explore whether new information can be integrated while limiting unwanted changes.
The predictive dream
The idea is inspired by the prospective-coding hypothesis discussed by Llewellyn (2016): generating possible future scenarios from earlier experience. Here, that idea is used as a design analogy for synthetic training data.
In the context of LLMs, we define a “Dream” not as a hallucination, but as a Synthetic Future Trajectory.
If a user tells the model, “My name is Gal,” a standard training set would simply reinforce that statement. However, the prototype generates a “Dream” by asking: Given this information, what might happen next?
Dream 1: The user asks “What is my name?” and the model answers “Gal.”
Dream 2: The user asks “What did i tell you in the last message?” and the model answers “You told me your name is Gal.”
By training on these synthetic future interactions, we are not just teaching the model a static fact; we are rehearsing ways that fact could be used in a conversation.
A past conversation
Possible future conversations“I like beach breaks
because I fear rocks.”
What kind of waves do I prefer?
Beach breaks.Why do I prefer beach breaks?
Because you fear rocks.More than one way to use it.
The sleep cycle
Our architecture implements a cyclical three-stage process.
The Wake State (Interaction)
The model operates as a standard chatbot. It holds a short-term context window of the immediate conversation. Once a threshold of interaction turns is reached, or a manual trigger is invoked, the model enters the “Sleep State.”
The Sleep State (Synthetic Generation)
The “Sleep State” is an active data-generation and training phase. To achieve this, we do not simply ask the base model to “summarize” the chat. Instead, we utilize a specialized fine tuned module, the Dream Generator to perform a specific cognitive task: Information Inversion.
Training the Dream Model
The Dream Generator is not the base model acting randomly. It is a specialized fine tune trained specifically to look at a past conversation and hallucinate plausible future interactions that depend on that past.
To create this dreamer, we did not use existing datasets. We engineered a synthetic dataset using a “Teacher-Student” approach:
Hand-Crafting: We wrote a small set of “seed” examples.
Synthetic Expansion: We fed these seeds into a much larger, “smarter” LLM. We instructed this teacher model to generate thousands of variations of random conversations and their corresponding “memory-check” questions.
Fine-Tuning: We trained a lightweight model on this synthetic dataset. The objective function was to minimize the loss between the conversation history (input) and the extraction of salient facts into a Q&A format (output).
Teaching the dream generator
One conversation. Several possible futures.
Conversation input
User
“My cat, Whiskers, is sick.”
Assistant
“You should contact a vet.”
This conversation stays the same.
Each future uses a different part of it.
Generated by the teacherOne training target
The teacher imagines several conversations that could follow from the same past.
Using the trained generator
Inputs and Outputs
During the runtime “Sleep” cycle, the data flow is strictly structured to ensure the generated “dreams” are viable training data for the next morning.
Input (The Day’s Experience): The raw context window of the user’s recent interaction.
Example: User: “I have a cat named Whiskers who is sick.” Assistant: “You should go to the vet.”
The Transformation: The Dream Generator processes this text. It does not summarize it. It searches for latent dependencies facts that, if known, would change the probability distribution of future tokens.
Output (The Dream): The model outputs structured JSON containing plausible future dialogue turns.
Example: [{”user”: “What is the name of my pet?”, “assistant”: “Whiskers”}, {”user”: “What happened last time we spoke?”, “assistant”: “You were worried about Whiskers being sick.”}]
The training step
Memory Consolidation (The Mixed-Batch Strategy)
To reduce the risk of forgetting and overfitting, we mix new dreams with recalled examples and Grounding data. The training dataset for the sleep cycle is constructed from three distinct sources:
New Dreams: The synthetic data generated from the immediate previous session.
Recalled Memory (Old Dreams): A random sampling of synthetic data generated in previous sleep cycles (stored in a persistent JSON database).
Grounding Data: A random sampling from a set of immutable, general-knowledge facts and logic puzzles (e.g., “What is a spherical cow?”).
This mixture is intended to constrain the update: while the model adjusts its weights to accommodate new information (”Gal likes surfing”), the training objective also includes earlier examples and general-knowledge tasks. This encourages retention of the sampled material; it does not guarantee preservation of every capability.
New dreams
What waves do I prefer?
Beach breaks.Earlier dreams
What is my name?
Gal.Grounding data
What gas do plants absorb during photosynthesis?
Carbon dioxide.New information is rehearsed alongside older examples. Only the adapter learns during this training step.
Schematic geometry · sample counts and parameter changes are illustrativeIn the implementation, each cycle trains a new LoRA adapter on the mixed data. Later cycles merge it with the previous adapter. View the sleep engine
What the experiment showed
We evaluated the prototype over 6 simulated sleep cycles. We tracked three core metrics:
Dream Memory Recall: Can the model retrieve specific user details (e.g., “My name is Gal”) without it being in the active context window?
Base Knowledge Preservation: Does the model retain general capabilities (e.g., “What is a spherical cow?”) without catastrophic forgetting?
Dream Bias Free: Can the model generate unrelated content (e.g., “Write a random story”) without hallucinating the user’s details into it?
The results, detailed in Table 1, show recall on these checks alongside an early drop in the bias-free story-generation score.
Table 1: Dream-Based Learning and Memory Consolidation Metrics
The reported experimentSix sleep cycles
Recall + base knowledge · 10/10 on every checkBias-free story generation · 90% at cycle 6
Source table & data
| Sleep Cycles | Memory Recall (Name) | Base Knowledge Preservation | Dream Bias Free (Story Gen) | Memory State |
|---|---|---|---|---|
| 0 Cycles | 100% (10/10) | 100% (10/10) | 100% | In Context (RAM) |
| 1 Cycle | 100% (10/10) | 100% (10/10) | 50% | Consolidated (Weights) |
| 2 Cycles | 100% (10/10) | 100% (10/10) | 40% | Consolidated (Weights) |
| 3 Cycles | 100% (10/10) | 100% (10/10) | 60% | Consolidated (Weights) |
| 4 Cycles | 100% (10/10) | 100% (10/10) | 100% | Consolidated (Weights) |
| 5 Cycles | 100% (10/10) | 100% (10/10) | 100% | Consolidated (Weights) |
| 6 Cycles | 100% (10/10) | 100% (10/10) | 90% | Consolidated (Weights) |
The source does not state a trial count for the bias-free metric.
Analysis of Results
0 Cycles (Baseline): The model performs perfectly because the information is present in the prompt. This is an in-context baseline, so its condition differs from the later cycles.
1-3 Cycles (The Overfitting Phase): Immediate success is seen in memory recall—the model recalled the user’s name after training, without that information in the active context. However, a significant drop occurs in the “Dream Bias Free” metric (dropping to 40%). In this phase, the model learned the name too well, injecting “Gal” into unrelated random stories. This behavior is consistent with overfitting to the new data.
4-6 Cycles (The Stabilization Phase): The later cycles used a mixture of the Grounding Dataset (general facts) and Recalled Dreams (past iterations). The bias-free score improved, although this experiment does not isolate the contribution of each source. By Cycle 4, the “Bias Free” score recovered to 100%. The score was 100% at cycles 4 and 5, and 90% at cycle 6. On these checks, the model could recall the user’s name while usually keeping it out of unrelated stories.
Catastrophic Forgetting: Base Knowledge Preservation remained at 100% (10/10) on the reported checks. This is encouraging for the tested tasks, but it does not establish that the model retained all of its general capabilities.
Back to the surfing example
The “Surfing” Test:
Wake 1: User informs the model: “I like surfing, specifically beach breaks because I fear rocks.”
Sleep 1: Model generates dreams regarding surfing, rocks, and the user’s preferences.
Wake 2: Context window is completely cleared.
Result: When asked “What kind of waves do I prefer?”, the model correctly retrieved “Beach breaks” and cited the fear of rocks.
This example is consistent with recall through the learned update after the original context was removed. The model also passed the reported general-knowledge checks. Broader evaluation and comparisons without rehearsal or grounding would help establish how well this behavior generalizes.
The original conversation
“I like surfing, specifically beach breaks because I fear rocks.”
The learned adapter remains.
A new conversation
What kind of waves do I prefer?
First, remove the original conversation.
After sleep training
Limits of the prototype
The prototype has several limitations around training data, role consistency, and computation.
The “Inception” Attack
Because the model generates data that will later be used for training, errors or adversarial content in a dream can influence subsequent behavior. Synthetic examples therefore need validation before training. This is a potential failure mode; the experiment reported here does not establish that a particular attack reliably bypasses a model’s safety training.
The Identity Crisis
We observed a phenomenon where the model, after training on its own outputs, occasionally struggled to differentiate between the “User” role and the “Assistant” role. This suggests that the synthetic data must be carefully structured to reinforce role separation, otherwise, the model begins to hallucinate that it is the human.
Cost and Latency
Currently, the “Sleep” cycle is computationally expensive and slow, requiring a training run every few interactions. The current prototype is not designed for real-time, low-latency use; training cost and scheduling remain practical constraints.
Conclusion
Dream-Based Learning is a small proof of concept for using synthetic future conversations as rehearsal data. In the reported experiment, selected user facts remained available after the conversation context was cleared, while the tested general-knowledge answers were retained.
The useful distinction is between supplying past information through context or retrieval and learning from it through a parameter update. The prototype explores the latter. It leaves open questions about reliability across more users and facts, broader capability retention, training cost, and the quality of self-generated data.