How does AI work?

Why — the first-principles explanation

Start with the distinction between ordinary software and machine learning. A traditional program encodes rules a person wrote. A machine-learning system is given examples, an objective and an optimization procedure; its parameters are adjusted until its predictions reduce a loss or improve a measured score. Google’s machine-learning material describes gradient descent as repeatedly calculating loss, finding the direction that reduces it and moving the parameters a small amount. That describes a training loop, not a magical download of facts.

Training and using a model are different phases. During training, data is transformed into examples, the model makes a prediction, an objective measures error and an optimizer updates parameters. The data, objective, model architecture, filtering and evaluation set all shape what is learned. During inference, the parameters are normally fixed: an input is transformed, passed through the network and decoded into a prediction, classification, score or generated output. A product can update context or retrieve fresh documents without retraining the base model.

For a large language model, text is split into tokens—words, subwords or characters. Given the tokens already in context, the model estimates a distribution over possible next tokens. It then selects one according to the decoding settings and repeats until it reaches a stop condition. Google explains this probability-and-context view, but “next-token prediction” is not a complete product description: the network can learn rich internal representations of syntax, concepts and procedures, and an application can add files, search, code execution or other tools.

The transformer made long-context training practical at scale. Its attention mechanism lets representations weigh relationships among tokens instead of relying only on a one-token-at-a-time recurrence. That architecture is important, but it is not the definition of all AI: image classifiers, recommenders, speech systems, diffusion models and ordinary predictive models use different objectives and structures.

The last layer is the application. A chat interface may add a system instruction, conversation state, retrieval, a tool call, a deterministic workflow or an agent loop. A workflow follows paths written in code; an agent lets the model choose some next steps within permissions and stop conditions. These additions can improve task completion, but they also add latency, cost, privacy exposure and failure modes. NIST’s generative-AI profile treats confabulation, privacy, security, bias and information integrity as lifecycle risks, so a fluent answer is not proof that the underlying claim is true.

The practical mental model is therefore: data and objective shape parameters; inference produces a probabilistic output; application code supplies context and actions; evaluation and human review decide whether the result is fit for use.

An example that makes it click

How to do it

1. Define the task and the output to optimize: a label, forecast, ranking, generated text or an action are different problems.
2. Prepare representative data and a held-out evaluation set. Data quality, coverage and labeling affect the learned behavior as much as the architecture.
3. Train the model by computing predictions and loss, then updating parameters with an optimizer until the chosen metrics stabilize or meet the target.
4. At inference, transform the new input, run the fixed model and decode or threshold its output. Track uncertainty and preserve the evidence needed to check it.
5. Add retrieval, tools, memory or deterministic code only when the task needs current information or actions. Give each tool narrow permissions and explicit failure states.
6. Evaluate real tasks, edge cases, adversarial inputs and tool failures. For consequential work, add citations, logging, human review and a safe fallback instead of treating fluency as accuracy.

Key facts

• Machine-learning training adjusts parameters against an objective; inference normally runs new inputs through the resulting fixed parameters.
• Google describes a language model as estimating the probability of a token or sequence of tokens in context; tokens may be words, subwords or characters.
• Gradient descent lowers loss by repeatedly measuring error, finding a reducing direction and taking a small parameter step; the same idea generalizes beyond linear regression with different objectives and optimizers.
• The 2017 Transformer paper introduced an architecture based on attention rather than recurrence and convolutions, enabling highly parallel sequence processing at scale.
• Next-token prediction explains the decoding interface of a language model, not every component of a modern AI product or every kind of AI model.
• Retrieval can supply current documents and tools can take actions, but those additions are application components; they do not automatically change the base model’s learned parameters.
• A workflow uses predefined code paths, while an agent gives the model some control over tool selection or next steps inside permissions, budgets and stop conditions.
• NIST identifies generative-AI risks such as confabulation, privacy, harmful bias, information integrity and security; evaluation and oversight are part of responsible operation, not an optional afterthought.

What authoritative sources say

People also ask

The same question, asked other ways

This page answers one intent expressed in 4 phrasings. How the index is organized →

• What is AI and how does it work?
• How AI work demo video meme?
• How do AI work?

Related questions