How does a coding AI model work?
A coding AI predicts the next chunk of text, one piece at a time, using patterns learned from billions of lines of public code. It doesn't run your program or understand your intent — it produces the most statistically plausible continuation. That's why it writes working code and confidently invents functions that don't exist.
Why — the first-principles explanation
Start with the surprising part: a coding model is a next-token predictor. Text gets chopped into tokens — roughly word- or symbol-sized pieces — and the model outputs a probability for every possible next token. `def get_user(` makes `self` or `id` overwhelmingly likely, and `banana` almost impossible. Generate one token, append it, feed the whole thing back, repeat. There's no separate 'reasoning module' bolted on. Everything the model appears to know about loops, types, and race conditions is compressed into the statistics of what tends to follow what.
The reason this produces real programs rather than gibberish is training scale plus structure. Code is unusually good training data: it's massively abundant on public repositories, it's highly patterned, and — crucially — it can be checked. You can run it. Later training stages exploit this by rewarding the model for code that actually compiles and passes tests, which is why modern coding models improved so violently. On SWE-Bench, a benchmark of real GitHub issues, systems went from solving 4.4% of problems in 2023 to 71.7% in 2024.
Under the hood the architecture is a transformer, and its key trick is attention: when predicting a token, the model weighs every other token in its context and decides which ones matter. That's how it 'remembers' that you defined `user_id` as a string forty lines up. Your context window — the file, the error, the instructions — is all it has. It has no memory of yesterday and no view of code you didn't show it, which is why the same model feels brilliant in a small file and lost in a large repository.
This mechanism explains the failure mode exactly. The model optimizes for plausibility, not truth. A function called `parse_datetime_safely()` is a very likely-looking thing to appear after your prompt — whether or not that function exists in the library you're using. That's a hallucination, and it isn't a bug to be patched out; it's what next-token prediction does when the plausible answer and the correct answer diverge. Which is why the tooling around the model — running the tests, reading the error, feeding it back — matters as much as the model.
An example that makes it click
It's your phone's autocomplete after a decade at the gym and a computer science degree. Your phone sees 'I'll be there in five' and suggests 'minutes' — not because it knows where you're going, but because 'minutes' follows that phrase constantly. A coding model does the same thing at enormous scale: it sees your function signature and your comment and suggests the body that most often follows patterns like those.
Now here's the catch, and it's the whole thing. Ask your phone's autocomplete for your friend's address and it will happily produce a street name that looks like an address and is completely wrong — because 'looks like an address' is all it was ever optimizing for. A coding model inventing `parse_datetime_safely()` is doing precisely that. The code looks right for the same reason the fake address looks right.
How to do it
- Tokenize: your prompt, code, and error messages are split into tokens the model can score.
- Attend: the transformer weighs which parts of the context matter for the next prediction — variable names, imports, the stack trace.
- Predict: the model outputs a probability distribution over the next token and samples one.
- Append and repeat: the chosen token joins the context and the loop runs again, token by token, until the code block is finished.
- Verify outside the model: the tool around it runs tests or a linter, and feeds failures back as new context — this loop, not the raw model, is what makes agentic coding work.
Key facts
- Coding models generate output one token at a time, each token conditioned on all previous tokens in the context window.
- AI performance on SWE-Bench, a benchmark of real software engineering problems, rose from 4.4% in 2023 to 71.7% in 2024 (Stanford AI Index, cited in Stanford Digital Economy Lab research, 2025).
- Shopify fine-tuned an open model, Qwen3-32B, to generate store automations — and found that having the model emit Python rather than the target JSON format improved syntactic correctness by 22 points and semantic correctness by 13 points.
- In a METR randomized trial (Feb–Jun 2025), 16 experienced developers using Cursor Pro with Claude 3.5/3.7 Sonnet were 19% slower on 246 real tasks, despite believing they were 20% faster.
- Models have no persistent memory between sessions — everything they 'know' about your project comes from the context window you provide.
▶ The 60-second explainer (script)
How does a coding AI actually work? Here's the surprising answer: it predicts the next chunk of text, one piece at a time. That's it. Your code gets chopped into tokens — word-sized pieces — and the model assigns a probability to every possible next token. After 'def get user, open paren', the token 'self' is overwhelmingly likely and the token 'banana' is basically impossible. Pick the likely one, stick it on the end, feed the whole thing back in, repeat. There's no separate reasoning module. Everything it seems to know about loops and types is baked into the statistics of what follows what. Why does this produce real working programs? Because code is fantastic training data. There's a mountain of it public, it's deeply patterned, and — this is the key — you can check it. You can run it. Training rewards code that actually compiles and passes tests. That's why these models improved so fast: on SWE-Bench, a benchmark of real GitHub issues, they went from four percent to seventy-two percent in a single year. But this also explains the failure mode perfectly. The model optimizes for plausible, not for true. So when it invents a function called parse-datetime-safely that doesn't exist in your library — that's not a bug. That's next-token prediction doing exactly what it does when the plausible answer and the right answer come apart. It's autocomplete with a computer science degree. Treat it like one.
What authoritative sources say
People also ask
Does a coding AI actually run my code?
The model itself doesn't. It only predicts text. Agentic tools built around it run your code, capture errors, and paste them back into the context — that verification loop is separate from the model and is what makes the difference.
Why does it invent functions that don't exist?
Because it's optimizing for what looks plausible, not what's true. A sensibly-named function is a highly likely continuation of your prompt regardless of whether that function is real.
Why is it great in a small file and useless in my big repo?
It only sees what's in its context window. Code it wasn't shown effectively doesn't exist to it, so cross-file conventions and hidden dependencies are invisible unless you surface them.
Do coding models remember my project between sessions?
No. Each session starts fresh. Anything persistent comes from tooling that re-reads your files and re-injects them into the prompt.
Is a bigger model always better for code?
No. Shopify got faster, cheaper results by fine-tuning a 32-billion-parameter open model for one narrow task — 2.2x faster and 68% cheaper than their previous approach. Fit to the task often beats raw size.