Build Daily
Tinley Park · October 21, 2026Why Nobody Has Solved Continual Learning
Charlie O'Neill of Baseten explains why the train step in the own-your-intelligence loop remains unsolved when a model is updated continually rather than once, walking through the trade-offs between SFT, RL, and distillation. He covers how each approach handles new knowledge, forgetting, and general capability, and offers guidance on tuning the SFT-to-RL ratio for your own task.
The outline
Tap a timestamp to jump there in the video.
The learning loop and its unsolved train box
Charlie O'Neill frames the own-your-intelligence learning loop and identifies the train box as the still-unsolved piece when models are updated continually.
What SFT can do that RL can't
SFT can install new knowledge into a model in ways that RL fundamentally cannot, because RL can only reinforce behaviors the model already produces.
Why off-policy SFT breaks on production edge cases
Off-policy SFT breaks down on production edge cases because the training distribution diverges from the live distribution the model encounters.
RL stays on-policy but can only upweight what it samples
RL remains on-policy and thus avoids distribution shift, but it can only upweight outputs the model already samples, limiting its ability to introduce novel behavior.
How open labs use distillation to warm start RL
Open labs use distillation from a stronger model to warm-start RL, giving the RL phase a higher-quality starting policy.
Distilled models start higher but plateau lower
Baseten replicated a result showing that distilled models begin training at a higher performance level but ultimately plateau lower than RL trained from scratch.
Continual learning: what breaks when you keep SFTing
Continual learning via repeated SFT causes catastrophic forgetting and degrades the model's general capabilities over successive updates.
Why RL forgets less but struggles to bake in knowledge
RL forgets less than SFT across updates but struggles to bake in genuinely new knowledge that the model has not yet sampled.
Self-distillation hints and the bias problem
Self-distillation and self-study hints can guide learning but introduce a bias problem because the model teaches itself from its own limited outputs.
Narrow tasks vs general agents like Cursor and Harvey
The trade-offs differ sharply between narrow single-task models and general agents like Cursor and Harvey that must handle diverse, open-ended work.
Frontier labs already run this loop, just slowly
Frontier labs already run this continual-learning loop in practice, but they do so slowly due to the cost and complexity of each iteration.
Most teams sit in the middle: tune your SFT to RL ratio
Most teams land in the middle and must tune the ratio of SFT and distillation to RL to balance knowledge installation against capability preservation.
The train step in the own-your-intelligence loop remains unsolved when a model updates continually rather than once, with SFT, RL, and distillation each trading new knowledge against forgetting and general capability. For a solo builder on a tight shipping cadence, that means the practical stack stays external—context, RAG, tool calls—rather than a model that rewrites its own weights. The SFT-to-RL ratio guidance is useful, but it is guidance for teams with compute budgets, not for a solo operator maintaining a product on a Tuesday.