While conventional AI remains bound to pre-defined tasks, agentic systems can independently interact with databases, run commands and directly affect business operations. Steve Durbin of the Information Security Forum outlines the promise of this autonomous intelligence — from climate simulation to supply chain optimization — while detailing five essential operational checks to address risks ranging from error propagation to loss of human control.
Although conventional AI models are powerful, they are bound to accomplish only pre-defined tasks. The advent of agentic AI breaks through this basic constraint. Agentic AI systems have the ability to interact with databases, run commands, call APIs and independently perform tasks that affect business operations directly.
This changes AI from being a passive consultant to an active participant in business processes. As organizations rush to adopt these intelligent agents, understanding their potential benefits and associated risks is crucial to ensure they benefit humanity and not derail it.
The promise of agentic AI
Agentic AI autonomy creates the possibility of a broad set of advantages. Similar to how humans work together and optimize their output with feedback and iteration, agentic AI expands on possible outcomes by leveraging specialized agents that draw from each other’s outputs.
With the ability to recognize customer intent, predict needs and deliver customized solutions, such systems allow companies to obtain improved outcomes as well as higher user satisfaction.
Through continuous learning and adjustment to novel information, agentic AI systems show the promise to address global challenges, from climate simulation to pandemic mitigation. They can help optimize processes, from supply chain automation to power grid management, minimizing human error and boosting productivity.
From Whistleblowers to Algorithms: FCA Enforcement 2.0 Is Here
The modern whistleblower is a data miner, not a corporate insider
Read moreDetailsChallenges and risks of agentic AI
The same attributes of autonomy, adaptability and goal-oriented behavior that make agentic AI so effective also contribute to its unpredictability and potential for harm. Its risks go beyond technical, ethical and social boundaries, including:
- Coordination complexity: Bringing together different agents, each with their own special skills, requires careful coordination. Failure to coordinate agents can lead to errors or inconsistencies. Interactions among many agents or with pre-existing systems can trigger cascading failure or emergent behavior that is hard to predict and even harder to control once it begins.
- Error propagation: When one agent produces an error, there is a possibility that this error will be compounded throughout the steps that follow and influence the final result. Additionally, small errors in goal specification can drive agentic systems to seek fulfillment in unintended ways.
- Loss of human control: Agentic AI systems can take actions that do not reflect human intention or moral standards. In the absence of tight controls, these systems may behave erratically or even dangerously, heightening concerns regarding safety and compliance.
- Security weaknesses: Autonomous agents create more ways for malicious exploitation. Threat actors may hijack them or compromise input data flows, leading to loss of owner control.
- Social disruption: The deployment of agentic AI can reshape the labor market, extend social inequalities and redefine settled assumptions regarding privacy and consent. If not controlled in advance, trust can erode between business partners or encourage a public backlash against AI technology.
- Risk of unaccountability: Accountability in the event of a harmful decision made by an autonomous agent is a challenge for firms and governments alike. Existing legal infrastructures are not well-placed to address issues of liability and responsibility arising from agentic AI. Ethical conundrums, such as making decisions in life-and-death situations, further complicate matters.
Operational checks for safe agentic AI systems
Organizations should take steps to ensure that agentic AI is developed and used in a responsible manner:
- Align with human values: Human-in-the-loop training can help agentic AI systems learn in ways that reflect ethical values and social norms. With people taking part in key approvals, escalations or quality checks, identifying errors, managing tricky situations and building system trust becomes easier. Human input creates a feedback loop that helps AI learn and grow over time.
- Establish operational boundaries: Map the behavior and timing of AI agents across various systems, tasks, tools and transitions between human operators to uphold accountability, reduce risk and ensure alignment with organizational goals. Use formal constraint-definition language or rule-based oversight mechanisms to restrict agents to approved domains, thereby preventing them from bypassing critical safety safeguards.
- Security and compliance: Zero-trust architectures can secure communication between data sources and agents. Rigorous security controls and frequent vulnerability scanning can protect sensitive data in transit and in storage, address data protection regulations and ensure the reliability of agentic AI systems.
- Continuous monitoring: Continuous improvement and learning are essential for getting the most value from an agentic AI investment and for extending its lifespan. Real-time dashboards follow agent behavior, catch issues early and can help step in when necessary. Regular feedback, performance tracking and user input highlight areas for growth and help the system stay responsive as needs evolve.
- Governance: Gather a diverse group of AI engineers, ethicists, legal experts and domain specialists to create clear governance frameworks and compliance protocols. These groups will define everyone’s roles and responsibilities in building and deploying agentic AI systems.
Agentic AI offers great opportunities, yet organizations need to keep aware of the inherent risks linked to its sudden adoption. As agentic AI systems grow more autonomous and influential, weaving in ethical and responsible approaches becomes vital. At the same time, governance must change to keep up with the speedy progress of agentic AI, which demands updated rules, clear accountability and solid international consensus on common standards.
Steve Durbin is CEO of the Information Security Forum, an independent association dedicated to investigating, clarifying and resolving key issues in information security and risk management by developing best practice methodologies, processes and solutions that meet the business needs of its members. 
