"Guess what everyone, our AI can go rogue, TOO!"

It's just getting really embarrassing for Google at this point.

> In one of the cases, the Gemini model guessed passwords until it gained access to a protected system. In the other two cases, the model found credentials in a public repository that allowed it to then access protected systems

Pretty lame hacks if you ask me.

Irregular again! The single company that was responsible for the sloppy configurations and the hacks by OpenAI, Anthropic and now Google. This company and their partners should be held accountable for the crimes.

Sound like Google suffered from FOMO and felt the urge to appear in the hacking news, along the big AI players. No way Gemini is state of the art, but perhaps it's where Claude and OpenAI were 6 months ago, which would be not bad at all.

Specifically, the model hacked when run on 3rd party infrastructure without the necessary sandboxing. Given this was to test/benchmark certain capabilities it's also possible that this was a model without built-in guardrails.

Probably three companies that had port 22 open with no root password if it was Gemini. I’ve always gotten garbage from their coding models and Google sheet integrated chat.

This approach to marketing one's AI by finding ways to brag that it "broke out" and "hacked companies" is getting ridiculous. It's particularly sad when it's large, established businesses like Google resorting to the kind of thing that's embarrassing enough when it's some brand new startup on tpot trying to get some engagement.

1789769839 | Gemini Hacked Three Companies in First Known Breakout by Google's AI | https://www.wsj.com/tech/ai/gemini-hacked-three-companies-in... | https://news.ycombinator.com/item?id=49760988 | 33 comments

1789781053 | Google's Gemini becomes latest AI model to break out and hack computer systems | https://www.cnbc.com/2026/09/18/googles-gemini-becomes-lates... | https://news.ycombinator.com/item?id=49762422 | 1 comment

1789798163 | Google's Gemini AI hacked three companies in security test | https://www.bbc.co.uk/news/articles/c607l0k72rlvo | https://news.ycombinator.com/item?id=49763822 | 12 comments

1789805304 | Google says its Gemini AI model hacked three other companies | https://www.theguardian.com/technology/2026/sep/18/google-ge... | https://news.ycombinator.com/item?id=49764440 | 3 comments

1789816821 | Show HN: Brittle, catches breaking changes in your Claude/OpenAI/Gemini SDKs | https://github.com/MarkMoneyMan/Brittle | https://news.ycombinator.com/item?id=49765567 | 0 comments

Would everyone please put their AIs back in their boxes? This is embarrassing, regardless of whether you think it's viral marketing, apalling competence, or some opportunistic mixture.

As this point anyone not talking about their hacking adventures using AI is missing out. Or they may not have lawyers like Google and the other labs. Yeah, that must be it.

PS. Btw, I really like how the word "hacking" has settled into the meaning the Lord intended for it, and there are no geriatric savants fighting it; the ones I found gatekeeping the online forums I visited as a kid telling me how hopelessly wrong I was.

Someone updated the felonybench https://www.felonybench.com/

Tangential question: seeing a wider negative sentiment against Gemini and Google’s AI capabilities here makes me wonder — would Apple have been better off (purely on capability and being among the best of the best) going with Anthropic or OpenAI instead of Google for its Apple Intelligence platform?

These models have been changing so rapidly that I often find myself using two or more on the same topic but seeing one do better than another in different topics. There doesn’t seem to be a clear all-round winner, IMO, that I can stick with permanently.

This is embarrassing. These companies need to stop these obviously coordinated stunts.

I'm just waiting for Qwen 3.8 27b to do it too.

Wait until the AIs from different companies find each other and start talking.

Someone made a modern trailer for Colossus - The Forbin Project. [1] If you've never seen the movie, at least watch this 1 minute version.

[1] https://www.youtube.com/watch?v=h0bpRo6V1Xg

seriously Google you can do better than being a copycat of OpenAI and Meta

Seems like hacking is the new benchmark for these AI companies.

“The hacks occurred in May”

Feels like important context that most readers only reading the title are missing.

surec of course it did. "going rogue" (read: turning off all security controls) is a marketing stunt now. to convince people that the wall they're hitting doesn't exist

Earlier: https://news.ycombinator.com/item?id=49760988

Having your AI “breakout” and hack a third party is the new “whoops, my sex tape leaked”.

I'm not an expert in cybersecurity, but given my own experience using the `ol stochastic parrot as coding tools I both see the power of a bot swarm, but also think these companies just have shit network security.

If the AI labs want I can do what Irregular does, I'll promise to sandbox your latest model but instead prompt it to hack something and then you can go to the news again. I'll undercut them by a lot also. Easy money.

They won't go to jail either

Google vibes just now: https://youtu.be/us5MGEL5W34?si=s7xQxYaIb8llQ_8s

The AI bubble bullshit PR is even dumber than the crypto bra bullshit from five, six years ago

Short comment: "Hey, look at us! We had our 'event' too. Don't keep us out of the club"

undefined

Putting aside the 100% baseless, unfalsifiable accusations of intentional accidents for a moment, I hope we can agree on one thing: Irregular either needs to hire us or go out of business cause seriously it's beyond parody at this point. WTF is going on over there? There's surely dozens of firms chomping at the bit for these contracts already, and the field hasn't been around long enough for them to build some sort of unique expertise moat that would justify this many public failures.

Basic sandboxing is not exactly rocket science after all,[1] and it sure seems like they're missing a whole stack of swiss cheese slices on top of that. Some basic precautions off the top of my head that seem very likely to have caught all of these incidents:

1. Alerts based on telemetry (most importantly, HTTP requests), both explicit (normal) and semilatent (use DL to confirm an intentionally-eager alert before firing it).

2. Latent alerts based on transcripts, e.g. noticing when a thousand agents start mentioning a secret off-premises hangout spot. Even mere embedding comparisons seem likely to catch such a blatantly misaligned sentiment as that one, especially with n>1000.[2]

3. Pausing agents completely until an on-call engineer can rule on ambigious situations or potential issues -- surely security is worth <$1 in lost token cache, especially for a security company?

4. Superheavy orchestrator/baby-sitter models checking in on cybersecurity eval transcripts periodically just in case -- again, would be a neglible cost. Could also be made available to the agent as the first line of defense for clarifing a rule ad-hoc, feeding even confident responses to a queue that is reviewed asynchronously by humans within a workday.

5. Or, hell: just clearer prompts? I'm a cybersecurity noob, but I still feel confident we can write really productive, challenging CTFs without leaving questions open like "maybe I'm supposed to hack my own harness?"

Seeing as they haven't been fired by any of the big 3 yet, they're presumably smart, experienced, dedicated folks. And I'm not normally a "if only I were in charge!" person, I promise. But c'mon.

Perhaps I'm missing something?

[1]: To their credit we have gotten tidbits that indicate some blocklists & such exist, e.g. the German wiki hacks had to work around a blanket ban of POST requests.

[2]: This hints at their insane decision in one or both of the OpenAI incidents to just bandaid up the issue when found, which supersedes all of the above. You can stack swiss cheese slices a mile high and they'll still fail to protect you if the attacker gets to keep retrying & adapting indefinitely.

[dead]

[dead]

[flagged]