Large Universe Model/Essays/From corpus to scene to stream
EssayFrom corpus to scene to stream
Three nouns describe the whole sequence: corpus, scene, stream. Everything else follows from which one a model is built around.
Corpus
A corpus is a bounded collection assembled before training and unchanged after. It has an edge, and the edge is a date.
Building around a corpus produces particular properties. Knowledge is broad, because a corpus can be enormous. It is static, because the collection is closed. And it is unattributed, because the corpus is compressed into parameters where individual sources stop being separable.
Everything characteristic of the Large Language Model — the breadth, the cutoff, the inability to say where a claim came from — follows from the corpus being the unit.
Scene
A scene is bounded in a different way: not by date but by episode. It has extent and duration, and it ends.
Building around a scene gives you consequence. A model that has watched enough of them learns how things behave, which is what the Large World Model contributed. But the intake is still episodic, and the weights encoding the dynamics are still fixed at training time. The Large World Model widened what could be observed without changing when observation stops.
Stream
A stream has no edge. It has a beginning and no end, and the model is attached to it rather than trained on it.
This changes what the model must be. A corpus can be compressed into parameters. A scene can be encoded as dynamics. A stream cannot be compressed at all, because it is not finished — so the model needs somewhere to put a conclusion that is current, revisable, and outside the weights. That place is the belief state, and it is the structural signature of the Large Universe Model.
Why the order was forced
The sequence could not have run differently. Reading a stream of unstructured evidence requires understanding language, which required the corpus generation first. Interpreting sensed evidence requires a sense of consequence, which required the scene generation second.
The Large Universe Model is not a rejection of its predecessors. It is what they make possible once you stop closing the intake.
After the stream
There is no fourth noun. Corpus to scene to stream is a progression toward unboundedness, and the stream is already unbounded. See why there is no fourth model.