# Generating text Part 12 of the tutorial "tiny-gpt". Canonical: https://learn.welldun.ai/tiny-gpt/12-generating/ The weights are fourteen thousand trained numbers that have never said a word. The loop that makes them speak is the counting model's loop with one upgrade, one dial — and one trap that looks like a sensible policy. --- Training kept the weights from its best moment — held-out loss 1.6672, step 11,500. Those weights ship with this page. They are also, on their own, completely silent: fourteen and a half thousand numbers that score text without producing any. Producing text is a loop you have run before. ## The loop, with one upgrade The counting model generated by sampling a character, appending it, and going again. The trained model does exactly that, with one new line: it can only see sixteen characters, so the input is cropped to the last sixteen of whatever exists so far. ```python window = out[-16:] # the model sees this much, at most logits = forward(window) z = logits[last position] # one opinion: what comes next p = softmax(z / temperature) out.append(draw from p) ``` Everything in it is old: the forward pass, the last position's scores, softmax, the weighted die. The window slides one character at a time, the prompt falls out of sight after sixteen characters, and the model keeps writing from whatever it can still see. ## Temperature The one genuinely new thing is that division. Scale the scores down before softmax and the percentages sharpen toward the favourite; scale them up and they flatten toward even. You have seen this lever before, from the other side — it is what the attention divisor tames — but here it is a dial the user holds. At 0.3 the model plays safe: *"the model that was train the what a span in the text see string about a setence"* — mostly real words, mostly its favourite phrases. At 0.9 it takes chances. At 1.6 the bars flatten until digits and stray punctuation leak in. ## The greedy trap Push temperature to nothing and sampling becomes a policy: always take the single most likely character. It sounds like the sensible extreme. Here is what it actually produces: ``` the model that sees and the text in the same and the text in the same and the text in the same and the text in the same and ``` The most likely continuation of a phrase can be the phrase that led to it, and with the die gone there is no way out — the model orbits its own favourite sentence forever. A little randomness is not decoration. It is the exit. The model reports a distribution. Collapse it to its favourite and you also collapse the only mechanism that keeps it from repeating itself. ## The rematch The tutorial opened with a table of counted pairs writing `sathe` — every pair legal, the whole nonsense — because one character of memory forgets the word it is inside. Eleven parts later, both models exist in the same notebook, trained on the same text. Generating side by side: ``` bigram : thot fr tind d. nyory bo anevele - s i-od thisideitde in's transformer: t the when countion word a strayss tabger in codes to be stred add that character sure of when a districe ``` Neither is literature. But one is shattered glass and the other is trying to write this tutorial — words, spacing, phrases that almost parse. The difference between them is everything since the tally: rows instead of name-tags, sixteen characters of sight, heads that choose what to read, blocks that think about it, and eleven and a half thousand nudges. Same text, same die, same loop. The only difference is how much of what it wrote the model can still see — and what it learned to do with that. ## Where this leaves you You have built a language model from a tally of pairs to a transformer that beats every memory-free predictor on held-out text — with every number met on the way and every claim run in front of you. What this tiny version deliberately skips, real ones have: dropout, a projection after the heads are glued, weight tying between the two ends, tokens instead of characters, and corpora millions of times this size. None of them changes the shape of what you built. To rebuild it with the library everyone ships, the [PyTorch pages](/pytorch/) run the same claims against a real install — start with the calls you have already written by hand. ## Build it yourself Five cells: the weights re-scored as a receipt, the loop written out, temperature at three settings, the greedy trap sprung, and the rematch.