Try it in one click
It is not a chatbot: it has only read simple stories for young children, and it continues whatever you write. Next, open the Microscope to see why it wrote a word, or the Replay to watch it learn.
Sampling settings
How the next word is picked. Temperature: higher means more surprising choices; 0 always takes the most likely word. Top-k: choose only among the k most likely words (0 = no limit). Max new tokens: how long the story may get. Seed: the same seed and settings always give the same story.
Steering: push the story toward a concept
The microscope found concepts inside the model, such as one that fires on the word “cat”. While the model writes, the chosen concept is switched on artificially (its direction is added to the model's internal state at layer 2). Strength 0.5 is a good start; much higher and the text falls apart. How this was measured against random directions.
Name guard: hide names before they appear
Before each word is chosen, the guard reads the model's internal concepts and predicts whether a name is coming; if so, that word is hidden. Once a word is finished, a simple word check also hides known names and capitalized words in mid-sentence. It also hides some ordinary words, mostly at the start of a sentence, where a name could have come next. Hover or tap a hidden word to see why. Balanced makes fewer mistakes; strict catches more names but hides more ordinary words. How well it works, compared with a regex.
With Show how surprised the model was on, each token (a word or piece of a word) is outlined and shaded by how unexpected it was to the model (darker = less likely). Hover or tap a token to see its probability and the words the model almost chose.
The model was trained on TinyStories (simple English stories for young children, license CDLA-Sharing-1.0), so that is the only kind of text it knows. It is not a chatbot: it continues whatever you write.
How it was built and what was measured: the case study.