When AI misfires Unpacking the invisible guardrails

AI doesn't always play by the rules. That's why digital identity, privilege management, and human oversight are becoming essential to prevent chaos when autonomous systems act on their own

- By Harsh Sharma [e-mail]

In the evolving landscape of enterprise AI, safety isn't just about patching bugs. It's about ensuring that intelligent systems know exactly who they are, what they're allowed to do, and when to stop. At the forefront of this challenge is Sudhakar Singh, Vice President and Head of Responsible AI, SAP. Singh's work focuses on ensuring AI behaves responsibly, within ethical and operational boundaries, without compromising on innovation.

From identity controls to explainability, he offers a clear-eyed look at the inner workings of responsible AI deployment, where trust isn't assumed but continuously engineered.

Why Al identity isn't optional anymore

Al is no longer just another application layer. It's a living part of the enterprise fabric, interacting with critical data and making decisions, sometimes autonomously. But with that autonomy comes risk. So how do you control whothe Al is?

Certificate-based authentication is emerging as the bedrock for Al identity enforcement. While most users still rely on passwords and multi-factor authentication (MFA), Al operates in a system-to-system context. In these environments, digital certificates provide a reliable way to trace and verify the origin of traffic: what system is calling which model or agent, and whether it has the right privileges.

Unlike passwords, certificates aren't easy to fake. That makes them especially useful in segregating human users from Al agents. You know which user called an app, and which system executed the Al operation. It becomes a foundational control that creates a virtual boundary between human and machine identities.

Rebuilding the who-did-what chain

When something goes wrong, say an Al takes an action it shouldn't, the most urgent task is reconstructing what happened. Who was involved? What triggered the action? Was it authorized?