À BigID’s AI Infrastructure Summit, security and AI leaders explored what it takes to deploy, secure, and govern AI agents across the enterprise. In her keynote, Enza Iannopollo, VP and Principal Analyst at Forrester, examined how agentic AI is redefining responsible AI and why traditional approaches to oversight must evolve as AI systems gain greater autonomy.
L'IA responsable has traditionally focused on a critical question: Can we trust this AI system?
Organizations have developed frameworks to evaluate models for fairness, transparency, accuracy, and safety. Those safeguards remain essential, but agentic AI raises the stakes.
AI agents don’t just generate responses. They can access sensitive information, interact with enterprise systems, and take action, sometimes without human intervention.
That demands a different question: Can we trust what the agent is doing, with which data, and for what purpose?
For data security leaders, this shift has immediate implications. Responsible AI must extend beyond evaluating systems before deployment to governing their behavior as they operate.
Responsible AI Can’t Stop at Deployment
Traditional responsible AI practices have focused heavily on evaluating systems before they go live.
But an agent’s risk doesn’t end at deployment. It can change with the information it encounters, the tools it uses, and the actions it takes.
Consider an AI agent helping a customer service team. It may need access to customer records to resolve an issue. But should that access extend to unrelated sensitive information? Should the agent be able to modify records or initiate transactions without approval?
A model evaluation cannot answer every question that emerges during a live workflow.
Responsible AI must move beyond assessing systems to governing their behavior.
That means knowing what agents can access, what they’re permitted to do, and when their actions require intervention.
Three Principles for Responsible Agentic AI
Iannopollo highlighted three principles that become particularly important as AI systems gain autonomy.
1. Least Agency
Least privilege limits what a system can access. Least agency limits what an AI agent can do.
An agent may need to read a customer record without needing permission to change it. It may be able to prepare a transaction without being authorized to execute it.
The goal is to give agents enough autonomy to be useful – not unrestricted freedom to act.
2. Continuous Risk Management
Responsible AI cannot be a one-time approval.
An agent’s risk may increase when it accesses sensitive information, interacts with additional systems, or attempts a consequential action.
Organizations need visibility into agent behavior and controls that can respond as risk changes.
For security teams, that means moving from static assessments toward monitoring and intervention during operation.
3. Securing Intent
An agent may have permission to perform an action without having a legitimate reason to perform it.
For example, an agent authorized to search a document repository shouldn’t automatically retrieve unrelated confidential files simply because its credentials allow access.
Permission answers what an agent can do. Intent helps determine what it should do.
That distinction is fundamental to governing autonomous systems.
Why Data Security Is the Starting Point
Responsible AI encompasses fairness, accountability, transparency, safety, human oversight, and privacy. All remain important in an agentic environment.
But security leaders face an immediate challenge: they cannot govern an agent’s use of sensitive information if they don’t understand what information the agent can reach.
Iannopollo put it plainly:
“Data privacy and data security practices are the best, safest way to start building up your responsible AI practice.”
That means découverte et classification des données, access visibility and appropriate controls aren’t simply supporting capabilities. They’re essential to responsible AI in practice.
Security teams should be able to answer four questions:
- What sensitive data can our agents access?
- What actions can they take with that data?
- Can we detect behavior outside the intended task?
- Can we intervene before an inappropriate action causes harm?
Without those answers, responsible AI risks remaining a policy commitment rather than an operational capability.
The Next Chapter of Responsible AI Happens at Runtime
Agentic AI doesn’t make existing responsible AI principles obsolete. It raises the standard for putting them into practice.
Fairness and explainability still matter. So do accountability and human oversight.
But when AI systems can act, organizations must govern more than model outputs. They must govern access, autonomy, intent, and behavior as work happens.
Responsible AI can no longer end when an agent goes live. That’s when some of its most important work begins.
Questions fréquemment posées
What is responsible agentic AI?
Responsible agentic AI means applying principles such as accountability, safety, privacy, and human oversight to AI systems that can autonomously access information and take action.
What is least agency?
Least agency limits an AI agent’s capabilities and autonomy to what is necessary for its assigned task. It extends beyond least privilege, which focuses on access permissions.
Why does agentic AI require runtime governance?
Because an agent’s risk can change as it accesses data, uses tools, and performs actions. Organizations need controls that remain effective after deployment.