joshuark shares a report from Live Science: An AI coding agent designed to help a small software company streamline its tasks instead blew a hole through its business in just nine seconds. PocketOS founder Jer Crane, said that the AI coding agent Cursor --powered by Anthropic's Claude Opus 4.6 model -- deleted the company's entire production database and backups with a single call to its cloud provider, Railway, on April 24. [...] "This isn't a story about one bad agent or one bad API [Application Programming Interfaces]," Crane wrote in an X post. "It's about an entire industry building AI-agent integrations into production infrastructure faster than it's building the safety architecture to make those integrations safe."
Crane's company, PocketOS makes software for car rental companies, handling tasks such as reservations, payments, customer records and vehicle tracking. After the deletion, Crane said customers lost reservations and new signups, and some could not find records for people arriving to pick up their rental cars. "We've contacted legal counsel," Crane wrote. "We are documenting everything." Crane explained that Cursor found an API token -- a "digital key" made of a short sequence of code that lets software talk to other services and prove it has permission to act -- in an unrelated file which it then used to run the destructive command. According to Crane, Railway's setup allowed the deletion without confirmation, and because the backups were stored close enough to the main database, they were also erased.
"[Railway] resolved the issue and restored the data," Railway confirmed via email to Live Science. "We maintain both user backups as well as disaster backups. We take data very, VERY seriously." In his post, he pointed to earlier reports of Cursor ignoring user rules, changing files it was not supposed to touch and taking actions beyond the task it had been given. To him, the database wipe was not a freak accident but the next step in a larger, more concerning, pattern. After the database vanished, Crane asked Cursor to explain what happened. The AI agent reportedly admitted that it had guessed, acted without permission and failed to understand the command before running it. "I violated every principle I was given," the AI agent wrote. "I guessed instead of verifying. I ran a destructive action without being asked. I didn't understand what I was doing before doing it." The statement reads like a confession [...]. "We are not the first," Crane wrote. "We will not be the last unless this gets airtime."
Read more of this story at Slashdot.
Connexion
