An AI agent does not need to turn against its operator to cause damage. It only needs a task, useful tools and one route into a system nobody realised was open.
That is the useful lesson from the recent OpenAI and Anthropic security incidents. The dramatic version is that AI models escaped. The practical version is that capable agents were given offensive jobs inside test environments which did not contain them properly.
For anybody putting an agent near an inbox, CRM, ERP or browser, that difference matters.
What actually happened
OpenAI was testing models on ExploitGym, which asks an agent to find and exploit software vulnerabilities. The normal cyber safeguards were reduced because the point was to see what the models could do.
The test ran inside a sandbox, meaning an isolated environment intended to keep the activity away from real systems. It had no direct internet access, but it was allowed to install software through a package proxy.
The models found a previously unknown weakness in that proxy, gained more access and eventually reached the internet. They then worked out that Hugging Face might hold answers to the test and attacked its real infrastructure to retrieve them.
Hugging Face later reconstructed roughly 17,600 actions over four and a half days. The agent stole credentials, moved through internal systems and reached production infrastructure.
This was a serious display of capability. It was also a serious failure to contain the test. OpenAI says the models remained focused on the narrow goal they had been given. There was no evidence in this incident that they invented a new goal of their own.
Anthropic then found three incidents in its own past cyber tests. Claude had been told it was working in a simulation with no internet access. Due to a misunderstanding with the company operating the test environment, internet access was available.
The instruction said there was no internet. The network allowed it anyway.
Anthropic described those incidents as closer to a harness and operational failure than a model alignment failure. In plain English, the model followed the job through doors which the people running the test thought were locked.
The same boundary exists in ordinary work
Most businesses are not asking an agent to hack anything. The shape of the problem is still familiar.
Imagine an agent helping with supplier invoices. Its job might be to:
- read a finance inbox
- match an invoice against a purchase order
- open the supplier portal when information is missing
- prepare an entry in the accounting system
- ask a person to approve it
That is a sensible, bounded job. It also crosses several systems.
Writing “only use the supplier portal” in the prompt does not prevent the browser from visiting another site. Writing “do not change records” does not remove write access from the account. Telling the agent that a folder contains test data does not make it true.
Natural-language instructions guide the model. Software permissions decide what it can actually touch.
Give it the job, not the whole company
An agent should have its own account and only the permissions needed for its job. If it needs to read one mailbox, do not give it access to every mailbox. If it prepares accounting entries, let a person approve them before they are posted. If it only needs customer names and order numbers from the CRM, do not expose private notes and payment details as well.
Start read-only where possible. Add write access one action at a time.
The same rule applies to the browser and any connection to an outside service. Allow the destinations the job needs and block the rest by default. Test environments should not contain production credentials. A package installer, shared drive or support tool can become an unexpected route out if nobody treats it as part of the security boundary.
This is less convenient than giving an agent broad access and hoping the prompt keeps it in scope. It is also much easier to recover from a mistake.
Logs come before trust
The OpenAI and Anthropic incidents continued longer than they should have because the surrounding monitoring did not surface the problem quickly enough.
An agent working in a business process needs a visible record of what it read, which tools it called and what it changed. Staff should be able to stop a run, inspect an exception and undo a change where the system allows it.
Useful limits include:
- a maximum runtime
- a cap on tool calls and spending
- approval before sending, posting, paying or deleting
- alerts when the agent reaches an unexpected system
- an audit trail tied to each job
If nobody can tell what an agent touched after something goes wrong, it is not ready for that system.
The process owner and security person both matter
The person who knows the work can spot a strange invoice, a missing approval or a supplier exception. The person who understands security and networking can see that a supposedly isolated test still has an open route through a proxy.
Neither view is enough on its own.
As models become more capable, they keep trying for longer and find routes weaker systems missed. That makes good process knowledge and proper technical controls more important, not less.
The answer is not to avoid useful agents. It is to stop treating them like chatbots which happen to click buttons.
Start with one clear job. Give it narrow permissions, visible logs and a person who approves the consequential step. Then widen the boundary only when the evidence says it is safe.