Governing agentic AI when machines begin to decide at scale

As AI agents gain autonomy across enterprise workflows, SAP outlines how accountability, auditability, and human oversight must be engineered by design.

- Shrikanth G

As enterprises move from Al-assisted workflows to autonomous decision-making systems, the focus is shifting sharply from capability to control. Sudhakar Singh, VP and Head of Responsible AI at SAP, explains how organisations must rethink governance, auditability, and human oversight as agentic AI becomes embedded across core business processes.

Drawing on SAP's experience of deploying agentic intelligence inside enterprise systems, Singh outlines why risk thresholds, reversibility, clean data foundations, and human-in-the-loop models are essential to ensuring that autonomy does not outpace accountability. Excerpts from the conversation.

As AI agents begin making autonomous decisions across finance, HR, and supply chain, how should enterprises rethink accountability and auditability?

Autonomous agents operate within industry-specific and organisation-specific risk thresholds. For instance, different organisations may define varying limits for automatic fund transfers. Any transaction exceeding the configured threshold must trigger human intervention and require explicit approval before the agent proceeds.

For actions approved automatically, enterprises must maintain comprehensive audit logs that capture the complete decision tree. This enables retrospective human review when required. Reversibility is another critical risk management concept. In systems where actions cannot be reversed, such as medical procedures, the risk level is significantly higher, and autonomous agent usage should be restricted or prohibited in such high-risk scenarios.

What new governance frameworks are needed when multiple agents collaborate and create chained decisions without human oversight?