How does generative AI work?
It predicts the next chunk of text, over and over. Trained on enormous amounts of writing, the model learns the probability of what follows any sequence, then generates by sampling from those probabilities one piece at a time. The 2017 transformer made this work by letting every word attend to every other word at once.
Why — the first-principles explanation
Generative AI is built on a trick that sounds too simple to work: turn creation into prediction. Instead of teaching a machine to write, you teach it to guess what comes next. Show it "the cat sat on the" and have it predict "mat." Do this across trillions of words, and to get good at that one game the model is forced to learn grammar, facts, reasoning patterns, tone, and code — because all of those are needed to predict text well. Nobody taught it any of that directly. It's all a side effect of playing next-word.
Then generation is just prediction run in a loop. The model produces a probability for every possible next chunk, one gets picked, it's appended to the text, and the whole thing runs again with the longer input. Word by word, that loop writes an essay. This explains a detail people find baffling: why the same prompt gives different answers. The model doesn't always take the most likely word — if it did, output would be repetitive and lifeless. It samples from the distribution, so some randomness is deliberately injected. Generative AI is stochastic by design, not by accident.
What made this actually work was the transformer, published in June 2017 — an architecture based solely on attention, dispensing with recurrence and convolutions entirely. Attention lets every word look at every other word and weigh which ones matter for the current prediction. Older designs read strictly left to right, one word at a time, which was slow and forgetful. Reading everything at once means you can train on the internet with thousands of chips in parallel. Scale became possible, and scale turned out to be the whole story.
The same mechanism explains generative AI's characteristic flaws. It has no fact database — knowledge lives as statistical tendencies in the weights — so a fabricated citation is produced by the identical process as a true one, because both fit the pattern. And because it favors probable words, its output is measurably less surprising than human writing. That flatness is real and quantifiable: it's called perplexity, and AI-detection tools are built entirely on it.
An example that makes it click
Play a game. I say "peanut butter and…" and your brain fires jelly before you decide anything. Now imagine playing that game a trillion times, on every kind of text that exists — recipes, lawsuits, Python, poetry.
Eventually you're not just finishing clichés. To predict what comes after "the defendant argues that the contract is void because," you have to have absorbed how contracts and arguments work. You never studied law. You just got extremely good at what-comes-next, and law leaked in.
Now write by playing the game against yourself: guess a word, add it, guess again. That's it — that's an essay. And notice you don't always pick the most obvious word, or you'd sound like a broken toy. You pick something likely-ish. That's why asking twice gives two different answers. And it's why, when the pattern points at a citation-shaped thing, you'll produce a citation-shaped thing — whether or not it exists.
Key facts
- The transformer architecture underlying modern generative AI was published June 12, 2017 by Vaswani et al. in 'Attention Is All You Need'.
- The transformer is 'based solely on attention mechanisms, dispensing with recurrence and convolutions entirely' — enabling parallel training at internet scale.
- Generation works by repeatedly sampling the next token from a probability distribution and appending it, so identical prompts can produce different outputs by design.
- Models assign a probability to every possible next token; averaged surprise across a text is 'perplexity', and AI-generated text is measurably lower-perplexity than human writing (Liang et al., 2023).
- That perplexity gap is the entire basis of AI-detection tools — which flagged 61.22% of non-native English writers' essays as AI, showing the signal measures predictability rather than authorship.
- Knowledge is stored in weights as statistical tendencies rather than as retrievable records, which is why fluent output and fabricated facts come from the same mechanism.
▶ The 60-second explainer (script)
How does generative AI work? It plays one game: guess what comes next. That's it. Show it 'the cat sat on the' and have it predict 'mat.' Do that across trillions of words. And here's the beautiful part — to get good at that one game, it's forced to learn grammar, facts, reasoning, tone, code. Because you need all of that to predict text well. Nobody taught it law. But to predict what follows 'the defendant argues the contract is void because' — you have to have absorbed how contracts work. It leaked in, as a side effect. Then generating is just that game in a loop. Model gives a probability for every possible next word. One gets picked. It's added to the text. Run it again with the longer input. Word by word, that writes an essay. And that explains something people find weird: why the same prompt gives different answers. It doesn't always take the most likely word — that would sound like a broken toy. It samples. The randomness is deliberate. The unlock was the transformer, in 2017 — built entirely on attention, throwing out reading word-by-word in order. Attention lets every word look at every other word at once and decide what matters. Read everything simultaneously, and suddenly you can train on the internet with thousands of chips at once. Scale became possible. Scale was the whole story. And the same mechanism explains the flaws. There's no fact database inside. So a made-up citation gets produced exactly the same way as a real one — because both fit the pattern.
What authoritative sources say
People also ask
Why does the same prompt give different answers?
Because generation samples from a probability distribution rather than always taking the most likely word. The randomness is intentional — always picking the top choice produces repetitive, lifeless text.
Is generative AI copying from its training data?
Not usually retrieving verbatim. It stores statistical patterns in weights, not a searchable copy of the text. Exact regurgitation can happen with heavily repeated passages, but it's the exception.
What's a token?
The chunk the model actually predicts — often a word or word-piece rather than a whole word. 'Unbelievable' might be three tokens. The prediction loop runs on tokens, not letters.
How is generative AI different from other AI?
Older systems mostly classified things — spam or not spam. Generative models produce new content by predicting sequences. Same statistical foundation, but sampling from the distribution instead of just reading it.
Why does it sound so confident when it's wrong?
Because confidence is a property of the text pattern, not a report on its knowledge. Confident-sounding prose is statistically common in its training data, so it generates confident-sounding prose.