How can generative AI be used in cybersecurity?
Generative AI already finds and fixes real software flaws. In DARPA's AI Cyber Challenge finals (August 2025), autonomous AI systems found 54 of 63 planted vulnerabilities and patched 43, plus 18 genuine bugs in open-source code — averaging 45 minutes and $152 per task. Defenders also use it for alert triage, code review, and phishing detection.
Why — the first-principles explanation
Almost all security work is reading. Reading logs. Reading code. Reading alerts, tickets, malware strings, and phishing emails. A large language model is, at bottom, a machine that predicts patterns in text — and source code, packet captures, and system logs are all text. So generative AI attacks the real bottleneck in security, which was never intelligence. It was attention. A human analyst can read maybe a few thousand lines of code carefully in a day. A model can read millions, badly but tirelessly. Trading depth for breadth is a good trade when the haystack is enormous and the needles are rare.
The second principle is why this works better in security than in most fields: you can check the answer. If an AI claims a function has a buffer overflow, you don't have to trust it — you run the input and see if the program crashes. If it writes a patch, you compile it and re-run the test suite and the crashing input. This is what made DARPA's AI Cyber Challenge results possible. The AI systems didn't have to be right; they had to be productive guessers wired to fuzzers and test harnesses that supplied ground truth. Cheap unreliable generation plus cheap reliable verification beats expensive reliable generation. That loop is the whole trick, and it's why the same approach does not transfer cleanly to, say, legal advice, where nothing crashes when the model is wrong.
The third principle is uncomfortable: the same economics work for attackers. Writing a convincing phishing email in flawless English used to be a skill barrier; now it costs a fraction of a cent. Scanning for vulnerabilities was expensive; now it isn't. Nothing about the technology inherently favors defense. What tilts the balance is that defenders can run these tools before shipping, on code they own, with the source, the tests, and unlimited attempts — and can then push the fix to everyone at once. Attackers get scale; defenders get scale plus home-field advantage. The side that industrializes the loop first wins the round.
Finally, note what generative AI is bad at here. It has no concept of your business, it confidently invents facts, and it can be talked out of its instructions by text it reads. That last one is not a bug you can patch — a model that follows instructions in its input cannot reliably tell your instructions from an attacker's. So the deployment rule that falls out of the mechanism is: let AI read and propose, make deterministic systems verify and act.
An example that makes it click
Picture an apartment building with 10,000 doors and one night guard. Working carefully, the guard can rattle 50 handles a shift, so most doors go unchecked for years. Now hand the guard a helper who checks all 10,000 doors every night — but the helper is a bit dim and flags 300 doors as "maybe unlocked" when only 4 really are. That sounds useless until you notice something: checking a door is trivial. You just turn the handle. So the guard walks the 300, finds the 4, and locks them before morning.
That is generative AI in security. The model is the dim, tireless helper generating suspicions; the fuzzer, the compiler, and the test suite are the hand on the doorknob. The value isn't that the AI is smart. It's that being wrong 296 times is free, and being right 4 times isn't.
How to do it
- Start read-only. Point the model at things where a wrong answer costs nothing: summarizing threat intel, drafting incident timelines, explaining an unfamiliar log format, triaging duplicate alerts.
- Wire it to a verifier before you trust any finding. Vulnerability claims should be confirmed by a fuzzer, proof-of-concept, or failing test — not by the model's confidence.
- Use it on code you own, pre-ship. Run AI code review in CI on pull requests, where a false positive is a comment and a true positive is a bug that never reached production.
- Keep humans on the action. Patches, blocks, account lockouts, and firewall rules get proposed by AI and approved by a person, per NIST AI RMF's Manage function.
- Assume the model's inputs are hostile. Anything it reads — a log line, a web page, a PR description — can carry a prompt injection (OWASP LLM01). Give it least-privilege tools and never let untrusted text reach a tool that can execute or exfiltrate.
- Measure it. Track false-positive rate, time-to-triage, and escaped bugs before and after. If the numbers don't move, you bought a demo, not a control.
Key facts
- DARPA's AI Cyber Challenge finals (results announced August 8, 2025): competitors' AI systems found 54 of 63 synthetic vulnerabilities (86%) and patched 43 of those found.
- The same systems discovered 18 real, previously unknown vulnerabilities in open-source projects — 6 in C and 12 in Java — and supplied 11 patches, all responsibly disclosed.
- Average patch submission time was 45 minutes at an average cost of $152 per competition task, across roughly 54 million lines of code analyzed.
- AIxCC awarded $8.5 million in finals prizes: $4M to Team Atlanta, $3M to Trail of Bits, $1.5M to Theori. Four of the seven finalist systems were open-sourced on August 8, 2025.
- OWASP ranks Prompt Injection as LLM01 — the top risk for LLM applications in its 2025 Top 10, covering both direct and indirect (external-content) injection.
- NIST released the Generative AI Profile (NIST AI 600-1) on July 26, 2024, as a companion to the AI Risk Management Framework 1.0 (January 26, 2023).
▶ The 60-second explainer (script)
Here's the direct answer: generative AI is used in cybersecurity to read things humans don't have time to read, and to propose fixes that machines then verify. We're not guessing about whether this works. In August 2025, DARPA ran the finals of its AI Cyber Challenge. Seven fully autonomous AI systems went after 63 planted vulnerabilities across 54 million lines of real open-source code. They found 54 of them. They patched 43. And along the way they stumbled onto 18 genuine bugs nobody had planted — six in C, twelve in Java — and shipped 11 patches for those too. Average time to a patch: 45 minutes. Average cost: 152 dollars. Why does it work? Because security is a reading problem, and models read cheaply. And critically, in security you can check the answer. The AI doesn't have to be right — it just has to guess a lot, while a fuzzer turns the doorknob to see which guesses were real. So the practical uses are: code review before you ship, alert triage, phishing detection, threat-intel summaries, and drafting patches for a human to approve. The catch: attackers get the same cheap scale. And any text your model reads can hijack it — that's prompt injection, OWASP's number one LLM risk. So the rule is simple. Let AI read and propose. Let deterministic systems verify and act.
What authoritative sources say
People also ask
Can generative AI replace a security team?
No. In DARPA's challenge the AI systems patched 43 of the 54 flaws they found — meaning roughly one in five confirmed vulnerabilities still had no working fix. AI expands how much a team can look at; it doesn't remove the need for people to judge and approve.
Does generative AI help attackers more than defenders?
Both gain scale. Attackers get cheap, fluent phishing and faster scanning. Defenders get something attackers don't: the source code, the test suite, unlimited attempts before shipping, and the ability to push one fix to every user at once.
What's the biggest mistake companies make deploying AI in security?
Giving the model tools that can act — delete, block, execute, send — while it still reads untrusted text. That combination turns OWASP's LLM01 prompt injection into a real breach path. Keep read and act separated.
Is AI-found vulnerability data trustworthy?
Only when a verifier confirms it. A model saying "this looks vulnerable" is a hypothesis. A crashing input or a failing test is evidence. Treat unverified AI findings as leads, not results.
Which framework should I follow for this?
NIST's AI Risk Management Framework 1.0 plus the Generative AI Profile (AI 600-1, July 2024) for governance, and the OWASP Top 10 for LLM Applications 2025 for concrete application-layer threats.