What is an AI chatbot?
An AI chatbot is a program that generates replies by predicting likely next words, rather than picking from scripted answers. Modern ones run on large language models trained on huge text collections. That's why they can answer anything — and why they can be fluently, confidently wrong.
Why — the first-principles explanation
The word "chatbot" covers two technologies that share nothing but a text box.
The old kind was a decision tree. A human wrote every branch: if the customer types "refund," show the refund menu. It could never say anything nobody wrote, which made it safe and useless. Ask it something off-script and it says "I didn't catch that."
The new kind writes every reply from scratch, one token at a time, by predicting what should come next given everything before it. Nobody wrote the answers. Nobody could — the number of possible conversations is unbounded. Instead the model absorbed the statistical structure of an enormous body of text during training, and generation is it drawing on that structure to continue your conversation plausibly.
Every important property falls out of that one design choice. It can discuss anything, because it isn't picking from a list. It's fluent, because fluency is exactly what next-word prediction optimizes. And it can be confidently wrong, because the objective was plausibility, not truth. A false sentence and a true sentence can look equally likely to the model. This isn't a bug someone will patch; it's the flip side of the mechanism that makes it work at all. Retrieval — feeding the model real documents at question time — helps a lot, which is why chatbots wired to search are more reliable than ones running on memory alone.
Two more consequences people trip over. It has no persistent memory unless a developer built one — each conversation is re-fed as text every turn, and when the window fills, the beginning falls off. And it isn't reasoning about you; it's continuing a document in which one character is a helpful assistant.
The adoption data tells the real story about where this sits. Stack Overflow's 2025 survey found 84% of developers use or plan to use AI tools — while trust in AI accuracy fell to 29%, and 66% named their top frustration as "AI solutions that are almost right, but not quite." That's the honest summary of what a chatbot is in 2026: enormously useful, and structurally unable to know when it's wrong.
An example that makes it click
Your phone's keyboard suggests the next word as you type. Say "I'm running" and it offers "late." It has no idea whether you're actually late. It knows that in the vast pile of texts people send, "late" follows "I'm running" constantly.
A chatbot is that, scaled up almost unimaginably — predicting not one word but paragraphs, having absorbed not your texts but a large share of everything written. And notice: your keyboard is never lying when it suggests "late." It's not tracking truth at all. Neither is the chatbot. It's suggesting what usually comes next, extremely well.
Key facts
- Modern AI chatbots generate each reply token by token via next-token prediction; rule-based chatbots select from human-written scripted responses.
- Fluency and factual error come from the same mechanism: the training objective rewards plausible continuations, not verified truth.
- Chatbots have no memory between sessions unless a developer implements one; conversation history is re-supplied as input text each turn.
- Stack Overflow's 2025 Developer Survey found 84% of respondents use or plan to use AI tools, up from 76% the prior year.
- In the same survey, trust in the accuracy of AI output fell to 29% from 40%, and 66% cited 'AI solutions that are almost right, but not quite' as their biggest frustration.
- The median Gemini Apps text prompt consumed 0.24 watt-hours of electricity as measured by Google in May 2025 — the physical cost of one chatbot reply.
▶ The 60-second explainer (script)
An AI chatbot is a program that writes its replies by predicting what word should come next — instead of picking from answers a human wrote in advance. That's the whole difference, and everything follows from it. The old chatbots were decision trees. A person wrote every branch. Type "refund," get the refund menu. Type anything off-script and it says "I didn't catch that." Safe. Useless. The new ones generate every reply from scratch, one word at a time. Nobody wrote the answers — nobody could, there are infinite possible conversations. The model absorbed the statistical shape of an enormous amount of text, and now it continues your conversation plausibly. It's your phone keyboard, scaled up beyond imagination. Type "I'm running" and your keyboard offers "late." It has no idea if you're late. It knows what usually follows. Now here's the part that matters. That same mechanism gives you the fluency *and* the errors. The model was trained to be plausible, not correct. A false sentence and a true sentence can look equally likely to it. That's not a bug someone will patch — it's the other side of the thing that makes it work. Stack Overflow's 2025 survey caught this perfectly: eighty-four percent of developers use AI tools, and trust in its accuracy fell to twenty-nine percent. Wildly useful. Structurally unable to know when it's wrong.
What authoritative sources say
People also ask
Does a chatbot understand me?
Genuinely contested. It builds internal representations that generalize far beyond memorized text, which isn't nothing. Whether that's understanding is an open scientific debate, not a settled question in either direction.
Why do chatbots make things up?
Because they're trained to produce plausible continuations, not verified ones. A false sentence can be just as statistically likely as a true one. Connecting the model to real documents at question time reduces this substantially.
Does it remember our past conversations?
Only if a developer built memory in. By default each session starts blank, and the conversation is re-fed as text every turn. When the context window fills, the earliest part drops off.
What's the difference between a chatbot and an AI agent?
A chatbot produces text. An agent uses a model to take actions — searching, running code, calling APIs — in a loop. The model is the same; the wiring around it differs.
Are AI chatbots safe to rely on for facts?
Not without verification. Developer survey data shows adoption at 84% while trust in accuracy sits at 29% — practitioners use them constantly and check their work. That's the correct posture.
The same question, asked other ways
- What is an AI bot?590/mo