Model collapse
The machine began eating its own output, and the output began to rot.
What happens when a generative system is trained on content that earlier versions of itself produced, rather than on the human record. Each generation copies the last generation's copies, and the small distortions compound. The tails of the distribution vanish first: the rare, the unusual, and the humanly odd get smoothed away, because the model learns the average of its own average. Over cycles the output narrows, flattens, and drifts from the world it was meant to describe. No one has to poison the well. The well poisons itself once the machine starts drinking from its own reflection.
Truth-adjacency
Truth-independent: the pattern works regardless of whether the claim is true
Where it shows up
Platforms and algorithms
What to watch for
The phrases and tells that mark this pattern in the wild:
generated content that feels averaged and strangely genericrare or unusual cases being systematically misseda system performing worse on the edges than the centersynthetic content flooding the sources the next model learns fromquality degrading across versions despite more compute How to recognize it
The tell is the loss of the unusual. A system trained on the human record captures the rare and the strange, because humans produce them. A collapsing system cannot, because it only ever sees its own smoothed-down averages. Watch for output that is fluent but generic, technically correct but stripped of the odd specifics that make something real. Also watch the trend across versions. Collapse is a process, not an event: quality erodes at the edges over successive generations even as the center looks fine. If the machine’s world is getting cleaner and duller than the one you live in, it may be learning from itself.
What to ask
- Is this system learning from the human record, or from prior machine output? The source of the training data determines whether the loop is open or closed.
- Are the rare cases disappearing? Collapse thins the tails first. If the unusual is systematically missed, the distribution is narrowing.
- Is quality drifting across generations despite more resources? Steady erosion at the edges, with a plausible-looking center, is the signature of recursive decay.
What it looks like when you’re wrong about it
You call “model collapse” on a single weak result from a system still trained mostly on human data. A bad answer is not a collapse. The pattern requires the degradation to come from recursive self-consumption across generations: the system learning from its own outputs, losing the tails of the distribution, and drifting from reality with each cycle. If the training data is still largely human and the failure is an isolated case, you are looking at an ordinary error, not a structural rot. The distinction matters, because collapse cannot be fixed by a better prompt. It can only be fixed by getting the human record back into the loop.
Recognized this online?
Misuse Guardrails
How this pattern gets misused
Someone blames model collapse for any mediocre AI output, including a single lazy result from a system trained mostly on real human data. The term becomes a catch-all for 'the machine was wrong,' which misses that collapse is specifically the compounding effect of self-consumption over generations. Used loosely, it sounds profound while explaining nothing about whether the failure was a one-off or a structural rot.
What it looks like when you're wrong about it
A model producing a weak or generic result on a hard prompt, while still trained primarily on the human record, is underperforming, not collapsing. The pattern requires the degradation to come from recursive self-consumption: the system learning from its own prior outputs across generations, losing the tails of the distribution, and drifting further from reality with each cycle. If the training data is still mostly human and the error is a single case, you are looking at an ordinary failure, not a collapse.
Not sure? Describe the situation to someone outside it. If they do not see the pattern, pause before you name it.
Related Patterns
The name is designed to spread. The hook is designed to stick. If you recognized something, share the name.