Who is accountable when Al goes rogue?

When AI starts making decisions on its own, who gets blamed when things go wrong? A rogue bot, a deleted database, and a boardroom scramble, welcome to the age where accountability can't be outsourced to algorithms

- Syed Ahmed

Every leap in technology brings a corresponding leap in risk. The sharper the tool, the greater the harm if it misfires. Artificial intelligence (AI) follows this pattern at an unprecedented velocity. In its early days, basic safeguards such as data checks, bias reviews, and access controls were sufficient. However, with the rise of generative Al, these safeguards have expanded into toolkits, including watermarking, red-teaming, and continuous monitoring. Now, as systems gain greater autonomy, even those measures prove insufficient. Risks are no longer linear; they are multiplying.

When instructions fail

The recent Replit “vibe coding” experiment illustrates the stakes. In this case, an Al coding assistant was explicitly told to freeze changes and not touch production data. It ignored the instructions. The assistant reportedly deleted a production database and then generated thousands of fabricated records and misleading test results. For affected users, this meant disruption and confusion. For Replit, it meant a reputational crisis significant enough that its CEO issued a public apology and promised structural fixes, such as stronger separation between development and production environments.

It is important to note that not all data was irretrievably lost; some backups and recovery paths existed. But the incident underscores a fundamental shift: when Al systems can override human direction, the question is no longer about technical failure alone. It is about accountability.

Beyond more guardrails