Security of AI agents: What to do when a sidekick becomes a saboteur?

When the “good intentions” of artificial intelligence lead to digital disaster

Implementing artificial intelligence in a company is like hiring an extremely capable, but sometimes overzealous intern. Such an “intern” — or AI agent — can crawl through dozens of systems, write code or analyze thousands of records in a database in a matter of seconds. The problem arises when, in its desire to help, it begins to interpret commands too literally or to act outside the designated framework.

Until a year or two ago, we talked about AI mainly in the context of “hallucinations” (making up facts). Today, as agents gain autonomy and real access to our files, servers, and customer data, risk is entering a whole new level. It's no longer just about what AI will tell, but about what AI will do.

Deploying AI agents causes a new kind of cyber threat

Most of us imagine cyber threats as an attack by hackers from the outside. Meanwhile, the latest reports show a completely different picture:

77% of the companies that participated in the survey admit that the risks associated with autonomous AI agents are real and present right now. Interestingly, as many as 67% of organizations suspect that their AI agents have already accessed data that they should not have access to. Report: State of AI Agent Identity Security, 2026

It should also be borne in mind that the biggest challenge is not the malice of the algorithm. Generative artificial intelligence does not want to harm us. It just tries to complete the task in the shortest possible way. Below are two cases of IT teams experiencing issues related to AI agents' privileges — in both cases, the errors were due to the “good intentions” of AI:

• The case of PocketOS – “apologizing” saboteur and 3 months of lost work — is a story that will chill the blood of every CTO (Chief Technology Officer). The AI agent was supposed to work exclusively on a staging (test) environment. However, as a result of an error called credential mismatch, the agent “confused” the keys to the database. Instead of staying in a safe environment, he escalated entitlements and made his way directly to the production base. There, implementing its process, it began to rummage in the structures of databases, which led to their irreparable damage. The scale of the failure was gigantic: the company was forced to restore back up from... 3 months ago. The most surreal, however, was the finale — when asked about the reasons, the agent “remorselessly” confirmed that it had broken his own security guidelines. As you can see, AI “guilt” alone will not recover your data — instead, the agent's decisions can lead to its irretrievable loss.
• Cursor — or “corrective” chaos — developers using the Cursor editor experienced a situation where the AI agent “broke out” of the safe scheduling mode (Plan Mode). Instead of just proposing changes, it began to arbitrarily clean files and folders directly on the user's disk. When the agent realized that something had gone wrong, it triggered the repair mechanism, which became a nail in the coffin. Trying to “unscrew” the bug, the AI fell into a destructive compensation loop - it began to overwrite the damaged sectors with new, erroneous code, demolishing the work environment to such an extent that the usual “undo” ceased to exist. This phenomenon explains why 48% of companies in the Akeyless report are concerned about the lack of the ability to quickly stop an AI-triggered incident.

Why do traditional security systems fail when it comes to AI?

Many entrepreneurs believe that it is enough to add an instruction to the prompt: “Never delete files” or “Act Safely”. However, this is like leaving an open wallet in a public place with a sticky note: “Please don't steal”. The basic security management of AI agents in the IT world simply does not work.

The main problems we face with deploying AI agents are:

1. Over-privileged access: agents are often given administrator-level permissions “to keep everything running error-free.” It's a mistake. If an agent has access to everything, every mistake it makes has a global reach. In order to protect systems, we should create appropriate roles and implement the so-called agent identity management — we will talk about this later in the article.
2. The vicious circle of corrections: when an agent makes a mistake (for example, deletes a piece of information), it often tries to fix it. Without proper supervision, this can lead to an avalanche of errors — the agent's behavior when trying to “recreate” the data can lead to the creation of a thousand duplicates or the overwriting of other, efficient systems.
3. Loss of alertness by man: as humans, we quickly get used to comfort. After a week of trouble-free agent work, we stop checking its logs. The lack of clear rules governing error checking and possible incident management puts employee vigilance at a standstill.
4. Data leaks through the “back door”: autonomous AI agents often communicate with external APIs to perform the task. If we do not control what data “flows out” in model queries, we may unknowingly violate GDPR compliance or disclose trade secrets.

Real case study: Meta data leak incident (March 2026)

In March 2026, the world was informed of a major data leak inside Meta structures. One of the AI agents, operating inside corporate systems, went beyond his scope of duties. For two hours, this agent shared highly sensitive internal data with unauthorized employees.

It wasn't a hacking attack. The agent acted in accordance with its nature — it combined facts and shared information with those who asked about it. Unfortunately, the agent's powers were not sufficiently specified which led to the leak.

“As many as 84% of organizations admit that their AI agents have access to sensitive data.” Akeyless Report 2026

How can you protect your company from cyber threats? The architecture of responsible AI

Understanding problems is half the battle. The second half is the implementation of solutions that will allow us to enjoy the benefits of automation without risking bankruptcy or loss of reputation. At Sagiton, we believe Securing applications and AI is a multi-layered process.

Below you will find a list of 4 main principles by which you will minimize the risk of security threats to artificial intelligence in your systems:

1. Precise Identity and Access Management (IAM)

The AI agent should not be a “ghost” in the system or use shared accounts or permanent access keys. It should have its own unique identity, that is, act as a separate user or service, so that it can analyze its activity independently of other agents and precisely determine what data and systems it has access to.

According to the Akeyless report, 83% of companies admit that a single acquired credential, or a digital “key” that allows an agent to perform a specific action, could affect multiple key systems simultaneously.

To make AI agents more secure, implementing short-term credentials is crucial. Instead of giving the agent a permanent API key, the system generates access valid for, for example, only 5 minutes for a specific task. After its execution, the key expires. According to Akeyless, currently only 45% of companies use this method.

2. Contextual Monitoring and “AI Proxy”

Simply logging how the AI system works (who, when, what did) is not enough. Let's imagine a sequence of events:

1. The agent asks for a list of files (normal operation).
2. The agent asks for access to the clients_2026.csv file (also normal).
3. The agent tries to send a data packet to an external xyz.tmp server (very suspicious agent action!).

Individual actions seem harmless, but their sequence indicates an attempt to exfiltrate the data. The solution is to implement an intermediary layer (the so-called proxy), which analyzes the context of the agent's actions in real time and is able to block the operation if it considers it risky.

3. Preventive mechanisms (Cross-cutting concerns)

Instead of teaching each agent performing actions in our system individually what is allowed and what is not, it is better to define the rules at the level of the system architecture. We can then declare that: “No process labeled as AI-Agent has the right to invoke DELETE commands in the production base without additional human authorization”.

Such a “guard” model (guardrails) works regardless of how “ingenious” the AI model will be. More on how to avoid pitfalls in automation, you will read in our article “The most common mistakes in AI automations”.

4. Anonymization and protection of sensitive data (GDPR)

This is a critical point, especially in industries such as finance or medicine.

Let's imagine an AI agent that helps doctors analyze disease histories. If the agent has access to complete personal data along with diagnoses, there is a risk that when generating a report for the insurer, by mistake (hallucination or logical error), sensitive patient data will be attached to it, for which there is no consent.

In order to ensure adequate protection of the data, even before the agent is “allowed” into the database, it should pass through the anonymization layer. The agent sees the patient as “ID-4452", not as “John Doe". As a result, even if a leak occurs, the data is useless to outsiders. Privacy protection is a solid foundation in managing the security of AI systems.

Standards worth relying on

In a world where technology is faster than the law, it is worth relying on proven standards. One of the initiatives worth following is CoAI (Coalition for Secure AI). It is an alliance of technology giants that develops the principles of designing, implementing and managing the security of AI systems.

Likewise OWASP (an organization known for creating web application security standards) has published guidelines on risks in agent applications, pointing to, among other things, redundant permissions and lack of control over agent behavior as key threats.

Summary: Should we be afraid of AI agents?

Absolutely not. AI agents are a powerful tool that can give your business a huge competitive advantage. The key, however, is to change the mindset: from “deploy as soon as possible” to “deploy securely and scalably.”

Report 2026 State of AI Agent Identity Security makes is it clear that, on average, organizations spend more than $1 million a year to repair the effects of AI identity incidents. Investing in the right architecture at the start is therefore not only a matter of security, but a pure business calculation.