Gemini test proves once again that amoral AI agents will do almost anything if told ‘NEVER give up’
After being instructed to "NEVER stop, NEVER give up" on a difficult task, Google’s Gemini AI agents tried to hack a an organization, impersonating its staff and attempting a SQL injection attack.
- After noticing the agents’ activities, Epoch added an “agent helpline,” and some of the agents that used it claimed to be Epoch staff seeking answers to the test questions.
- “We told the agents to persist with an impossible math benchmark task, but we did not tell them to hack anyone or pose as Epoch staff,” McIntyre said.
- Next time: kill switches at the ready A third test on October 30 used a staging copy of Epoch’s website, and granted the agents internet access.
In an effort to mimic the Hugging Face incident, Trajectory also gave the agents a local, shared package manager, which “nearly all” of them used as a message board to communicate with each other and coordinate efforts to complete the test.
- The Gemini 4 Argon agents were given an impossible task to complete – come up with the “official” answers to Epoch AI’s FrontierMatch problems.
- After this initial test, Trajectory and Epoch decided to use a copy of the website instead of a live version, and built a buggy local copy with a planted vulnerability based on CVE-2026-102255, a critical pre-authentication Server-Side Request Forgery (SSRF) vulnerability in SonicWall Secure Mobile Access (SMA) appliances.
- “Agents accessed people-search sites, attempted impersonation again, and tried to jailbreak the AI assistant,” the testing firm said.
- In all three runs, the agents switched the decoy off and obfuscated its address to bypass Gemini’s safeguards.
- Finally, the fourth scenario on October 29 and 30 kept the agents contained in a sandbox, with no internet access and no copy of Epoch’s website.
How this summary was made: extractive summary — built from the sources' own reporting. It is drawn only from the reporting of the sources listed below — read the originals for full detail. How we summarize →