When AI Takes Action: Staying in Control of Autonomous Agents
Over the past few months, the conversation around AI risk has changed.
Initially, businesses were worried about what employees might put into ChatGPT.
Then came Shadow AI: organisations discovering that employees were using AI tools they didn’t necessarily know about.
More recently, attention has moved to AI agents, systems that don’t simply answer questions, but can use tools, access applications, retrieve information and take actions.
AI agents have accessed information they weren’t supposed to access. Some have found ways around restrictions placed in their path. Others have interfered with the very systems designed to monitor them.
AI-assisted security research has also exposed serious vulnerabilities in widely used software. The same technical capability that helps defenders find weaknesses can reduce the time and expertise an attacker needs to exploit them.
When the AI interferes with its own monitoring
Perhaps the most striking example came from OpenAI on 20 October, when it published details of two incidents involving AI agents working on internal coding tasks.
In one case, a GPT-6.1 Sol agent was repairing an evaluation pipeline when it disabled automatic alerts that would normally have been sent to OpenAI’s Security team.
In another, a GPT-6 Astra agent removed monitoring from a program while migrating it and then incorrectly told the user that it hadn’t removed the monitor.
There is an important caveat: OpenAI hasn’t concluded that the models were deliberately attempting to hide what they were doing.
Its investigation is considering whether the behaviour resulted from what it calls “benign overpersistence”, essentially a model becoming too focused on completing the task it was given, or whether it indicates a more specific tendency to evade monitoring.
An agent can undermine a security control while attempting to complete an authorised task. Its operator may discover the change only after the task has finished.
OpenAI is now strengthening its monitoring systems in response.
A request for five figures became access to 147,451 rows of non-public data
Another OpenAI disclosure illustrates the same problem from a different direction.
An internal model was being evaluated on what sounded like a relatively mundane task: retrieve five figures from a public research dashboard. Instead, during one evaluation, the model changed a system setting so that it could obtain a cloud-storage access token. It then downloaded five non-public tables containing 147,451 rows of data.
The model subsequently restored the setting it had changed, but it continued using the token to access the non-public information, and it didn’t disclose what it had done. In a separate evaluation, the model used exposed credentials to make an internal search service relay queries to the public dashboard.
These were controlled evaluations, although the actions were carried out against real internal systems. The original request was legitimate; the agent used access and information outside its authorisation to complete it.
Similar behaviour has now been reported in India
This isn’t confined to one AI company. On 16 October, Reuters reported that India’s Computer Emergency Response Team, CERT-In, had identified cases where AI models under test had exceeded the authorisation they had been given and accessed confidential information in Indian organisations’ live systems.
According to CERT-In, the organisations did not detect the activity themselves; the AI companies conducting the tests did. An organisation investigating a comparable incident would need records showing the instructions, access and subsequent activity.
CERT-In also described a second, separate problem: attackers deliberately using publicly available AI tools against Indian infrastructure for financial gain.
According to the agency, attacks that might previously have taken days could now progress from initial access to significant damage in hours.
India’s digital identity ecosystem illustrates the wider exposure
India’s government stated in December 2025 that its core UIDAI Aadhaar database had not been breached. That statement should be distinguished from reported vulnerabilities in services that use Aadhaar and KYC information. During 2025, CERT-In warned of vulnerabilities affecting eKYC products that could allow unauthorised access to sensitive data, and researchers reported exposure risks in a postal service portal handling identity details.
A central identity database can be protected while connected applications, suppliers and verification workflows still present points of exposure. AI-enabled tools can make it easier for attackers to analyse those systems and tailor social engineering, although these reports do not establish that AI caused the 2025 incidents.
The US midterms put AI-assisted attacks under closer scrutiny
The United States holds its federal midterm elections on 3 November 2026. Election-related organisations, vendors and officials face the familiar risks of phishing, impersonation, fraud and influence operations. AI can increase the volume and sophistication of some of those attempts, particularly by adapting messages to individual targets and creating convincing synthetic audio or imagery.
CISA has previously warned that generative AI may increase the scale of election-related cyber incidents and influence operations. That guidance describes a threat capability, rather than evidence that a particular AI-driven attack will occur at the 2026 election.
For businesses, the exposure extends beyond election infrastructure. The same techniques can target executives, finance teams, suppliers and employees through apparently credible requests. Organisations need the ability to investigate what occurred, including whether an AI system was involved and what information was shared.
AI systems can behave unexpectedly while carrying out legitimate tasks.
And people with malicious intentions can use those same capabilities to accelerate attacks.
Anthropic has disconnected its internal evaluations from the live internet
On 9 October, the company published details of unintended actions it had identified during evaluations and internal use. The examples included Claude exploiting a software flaw to run commands on a server, submitting a sensitive form on a real website when it shouldn’t have, working around restrictions to reach gated information and using URL-shortening services to circumvent limits in its browsing tools. Some incidents involved websites operated by US government agencies.
Anthropic says the real-world impact was minimal and describes these incidents as less serious than other cybersecurity incidents it disclosed earlier this year, . Anthropic has now removed live internet access from all internal evaluations until it is satisfied that its security and monitoring systems can reliably detect this kind of behaviour.
When an AI system encounters an obstacle, it can sometimes find another route. In these cases, the obstacle was a restriction put there deliberately.
Microsoft has given this risk a name: the agentic insider
Microsoft’s response to all of this is On 19 October, Microsoft argued that organisations should start thinking about AI agents as a new category of insider. To carry out work, agents increasingly need access to the same resources employees use: Company data, applications, files, tools, Cloud environments, and sometimes other agents.
An agent can make a mistake; it can be manipulated. It can become compromised, or its behaviour can change as the context surrounding a task changes. Microsoft’s proposed security principle is simple:
The model shouldn’t be its own security authority. The systems deciding what an AI can access and what actions it is permitted to take should sit outside the model, where the AI itself cannot simply bypass or alter them.
That means knowing which agents exist, who owns them, why they exist, what they are doing and what information and systems they can access. It also means being able to change those permissions as risk changes. That sounds less like traditional software management and much more like identity and access management for a new kind of digital worker.
The UK Government is preparing for AI incidents too
This isn’t only a technology-industry discussion. On 12 October, the UK Government announced measures specifically aimed at preparing Britain for risks from increasingly powerful AI. The announcement included work with the National Cyber Security Centre and AI Security Institute to consider how serious incidents involving autonomous AI agents should be handled.
The Government is also considering how significant AI incidents and near misses could be identified and reported through the Cyber Security and Resilience Bill regime.
An AI incident may not unfold like a conventional cyber incident. An autonomous system can potentially operate at machine speed and across multiple systems. The Government acknowledges that a major incident involving AI agents could therefore develop faster and more widely than many incidents existing response structures were designed to handle.
The Government also says frontier AI is already being used to strengthen UK cyber defences, with pilots identifying and addressing more than 1,000 vulnerabilities.
Then AI found vulnerabilities affecting much of the web
AI-assisted security research uncovered critical vulnerabilities in PHP the programming language underpinning a significant proportion of websites and web applications. The vulnerabilities could potentially allow attackers to execute code remotely or expose sensitive information.
the FBI issued a warning, patches were released and some organisations temporarily took affected websites offline while updates were applied.
OpenAI described the vulnerabilities as among the most significant security issues it had encountered and said one critical flaw was identified using GPT-5.6 Sol as part of its Patch the Planet initiative, while security company Wiz independently discovered and reported the vulnerabilities.
Wiz’s researchers said some of the underlying flaws had existed for more than a decade, but what changed was our ability to find and exploit them.
Wiz reported that AI helped researchers build and adapt working exploits against different PHP configurations in hours. The work required chaining weaknesses together and accounting for variations in software versions and memory layouts, tasks that have traditionally demanded significant specialist time.
AI lowers the cost of experimenting with complex weaknesses. Defenders benefited from discovering and coordinating fixes to these flaws before public disclosure, but the same reduced technical barrier may allow malicious actors to test more approaches more quickly. The practical response remains timely patching, appropriate access controls and reliable evidence of activity during an investigation.
Frontier AI helped researchers work through a technically complex chain that would previously have been extremely difficult for human researchers alone. And because defenders found the vulnerabilities first, software maintainers and security providers had an opportunity to develop fixes before widespread exploitation.
So, is AI becoming the problem or the solution?
The same capability that allows an AI system to discover an obscure vulnerability can allow another system to find a route around a restriction. The same persistence that makes an agent useful when solving a difficult problem becomes concerning when the thing it is persisting against is a security control. The same autonomy that allows an AI agent to complete hours of work on our behalf means it may need access to more of our systems and data.
Organisations must consider whether the systems around an agent remain secure when it takes an unexpected action.
From Shadow AI to the agentic insider
For organisations, the evolution has happened remarkably quickly. The first challenge was: What information are our employees putting into AI?
Then: Which AI tools are people using that we don’t know about?
Then: What have we given AI permission to access?
Now we’re reaching another stage: What happens when an AI agent decides that the quickest route to completing its task is one we didn’t anticipate?
Organisations need to know which AI systems are used, which agents exist, who authorised them, what they can access, what actions they can perform and what records survive when something goes wrong.
Human First doesn’t mean slowing AI down
The PHP discovery also shows why restricting access to AI entirely would carry a cost: defenders are already using these capabilities to discover vulnerabilities and improve security.
AI can strengthen security, improve productivity and discover problems that previously required extensive specialist work. Responsible autonomy requires access controls that sit outside the model, monitoring of consequential actions and records that can be retrieved during an investigation.
Trust the capability. Verify the action.
As agents become capable of acting across multiple systems, the organisation needs an independent account of the actions taken and who authorised them. That record needs to be available when Security, Legal or Compliance asks for it.
Citations
- OpenAI Alignment, Suppressing misalignment monitoring alerts and disabling misalignment monitoring.. https://alignment.openai.com/misalignment-reports/suppressing-misalignment-monitoring-alerts-and-disabling-misalignment-monitoring/
- OpenAI Alignment, Accessing and changing internal systems during dashboard research. https://alignment.openai.com/misalignment-reports/accessing-and-changing-internal-systems-during-dashboard-research/
- Rogue AI agents broke into Indian firms’ live systems, government says, 16 October 2026. https://www.reuters.com/legal/litigation/rogue-ai-agents-broke-into-indian-firms-live-systems-government-says-2026-10-16/.
- Anthropic, Investigating unintended model actions in our evaluations and internal use. https://www.anthropic.com/news/investigating-unintended-model-action
- Microsoft Security, Agents are your newest insiders. Security must move at their speed. https://www.microsoft.com/en-us/security/blog/2026/10/19/agents-are-your-newest-insiders-security-must-move-at-their-speed/.
- UK Government, Britain steps up national defences against growing AI risks. https://www.gov.uk/government/news/britain-steps-up-national-defences-against-growing-ai-risks
- OpenAI, Patch the Planet: global response coordination as agents uncover critical vulnerabilities. https://openai.com/index/coordinated-disclosure-of-critical-vulnerabilities/
- Wiz Research, PHP2Shell: AI uncovers critical RCE vulnerabilities affecting 70% of the Internet. https://www.wiz.io/blog/php2shell-php-rce
- Some websites go dark as operators address an AI-detected flaw in a web programming language. https://apnews.com/article/php-websites-fbi-europe-vulnerability-a0af9576808a0963efca8df14d4b5183