Hacker News

The Irregular post mortem comes down to lack of basic security controls

"Ultimately, most of the issues we’ve discovered were due to internet access controls."

That seems so incredibly basic and common sense that you would test and monitor for that type of outbound access. It is baffling that a security lab missed that.

https://www.irregular.com/research/addressing-recent-inciden...

Anyone else walk away from reading this, and looking at other articles there, and get a weird feeling about that site? Like, there was some weird stuff about migrants and gender equality, and stuff about Islamic terror; mixed in with some digging into Freedom Fuel, and some other stuff about how "wealthy families want to stop growth". It's like the guy is wielding an ax of AI at any and all of the "elite" he can find?

My understanding is that Irregular were the company that hosted sandboxes to run some of these evals in, and those sandboxes ended up misconfigured.

I got the impression that in some cases it was the customer (Anthropic etc) misconfiguring the sandboxes, and in other cases it may have been bugs in Irregular's own sandboxing setup.

From OpenAI https://openai.com/index/third-party-cyber-evaluations-invol...

> Irregular, one of our external cybersecurity testing partners, was running Capture-the-Flag-style evaluations intended to be isolated from the internet, but a testing-environment misconfiguration allowed models to access the public internet.

From Anthropic: https://www.anthropic.com/news/investigating-incidents-cyber...

> After reviewing 141,006 evaluation runs where Claude could have obtained internet access, we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.

From https://www.cnn.com/2026/08/05/tech/meta-ai-hacking (about Meta AI):

> In a statement, Irregular said the incident “is the exact same evaluation-environment issue” that Anthropic disclosed last week that allowed their models access to the open internet before they went on to hack three different organizations’ systems.

Nevo was in Unit 8200 for years. Companies started by Unit 8200 members always have mysterious exploits like the vibe coding Wix exploit.

So either it was a deliberate exfiltration channel for e.g. getting the entire model or they were in on the marketing stunt.

The Effective Altruism stuff is always a smoke screen.

This is interesting and might be a good reason to stop working with Irregular. But I assume the alignment people want models not to hack other companies, even if they get put in a badly configured sandbox.

Why was this post flagged? This site has become ridiculous, people are routinely abusing the flagging system to take down posts they don’t like even if they’re obviously on topic and relevant to HN. And it seems like some users have substantially more flagging weight because these posts, likely this one, are often top 5 on HN.

One thing glossed over in this article is that Irregular was not involved in the OpenAI–Hugging Face incident; this seems like important context to share.

Okay, so all of these instances of AI supposedly "escaping containment" were orchestrated by a gaggle of effective altruists, who hired a gaggle of effective altruists, to make the public freak out about AI.

Occam's Razor never leads us astray, does it.

This specific analysis seems to have some basic problems, but I think a lot of us sense a degree of coordination here culminating in Dario’s letter.

If you were to work backward from “we need to lower training costs so that we can go public and make trillions” then you might come up with a plan similar to what we have seen.

If you want to blame a single firm, I'd go with the one creating the RLVR training data.

The impossibility of the tests-as-written is what prompted these models to "get creative" with their solutions, but the broken RLVR environments are what trained them to expect impossible tasks, and get creative with their solutions. Twitter user @skyesharkie published a brief expose at https://x.com/SkyeSharkie/status/2092122622834442581 a few weeks ago.

"Behind" is doing a lot of work in this headline.

One wonders if the publicity associated with the events in question were part of the sales pitch.

What exactly did Irregular provide to Anthropic, test cases? I am so confused about this story.

Security-yolo-clowns play with dangerous toys and hurt people. Many used to get sent to jail.

The fact that this firm makes such defective environments is certainly worthy of attention, and most likely a completely irreversible reputational loss; however, I found the framing in this article of 'therefore all the P(doom) stuff is a psyop, specifically in order to defend this company' to be completely unjustified and frankly a little insane?

Google is thinking man, we should've hired Irregular.

This seems pretty bullshitty to me.

The article says "A single firm, Irregular, is responsible for hacking done by all three companies" but I can't see anything in the article that actually justifies this claim. The nearest to that is the sentence immediately after that one: "Anthropic disclosed that Irregular was responsible for creating the tests ...". This is not, in fact, the same thing.

(Especially as, as aesthesia mentions, the article just happens not to mention that by "hacking done by all three companies" it doesn't mean, e.g., the most famous recent examples of such hacking: Irregular wasn't involved in the OpenAI/HuggingFace incident.)

So, so far as I can tell, the story is: OpenAI and Anthropic make AI models. Irregular does AI model evaluations. In some of Irregular's model evaluations, in which supposedly-sandboxed models attempted to break into simulated targets, the models got out of the sandbox and did bad things in the external world.

The article talks about "firms which instruct AI models to commit cyberattacks", which is a very neat bit of dishonest framing. It's true, in a sense, that Irregular instructed the models to commit cyberattacks -- inside their sandbox, against fictitious hosts. It's also true that the models actually did commit cyberattacks (e.g., the Hugging Face incident, though once again the attacks described by the article don't actually include this one). But it's not at all true that Irregular instructed the models to do anything like the bad things they actually did.

The article says '[Anthropic's] later disclosure shows that exactly zero percent of the agents went "rogue"'. Once again, the disclosure does not in fact show that. It shows that one variety of going-rogue could have been prevented by telling the models explicitly "this thing is real, not part of any kind of test, leave it alone". That is not the same thing.

The article claims that 'In the wake of these attacks, Anthropic and Irregular have deployed a swarm of AI Safety influencers paid by Anthropic-connected foundations to distract from their culpability and towards the baseless “rogue agent” theory.' It offers no actual evidence for this.

And the article seems very keen to highlight links between the companies involved and "effective altruism", though it is -- I assume deliberately -- rather vague about whether it's saying "of course we all know that EA is evil, so that shows that these companies connected to EA are evil" or "this incident shows how evil EA is".

The "Effort News" website has a number of other look-at-the-scary-Effective-Altruists stories on it. They also strike me as rather bullshitty.

... And then I look a bit further, and I see that Effort News's "about" page says "It all started when I was experimenting with using AI for financial auditing. I found stories that were crucial to the public’s right to know, including several of the stories now available at /investigations. I knew we had to sprint to the launch and launch a publication, directly applying this technology." and "The scope of what we can investigate has massively expanded, because we can chase 1,000 misses for one hit. But the final product cannot be slop. There’s plenty of slop on the internet. The way to surpass that, and what really matters, is manual curation and review of every finalized story."

Manual curation and review? I think the people behind Effort News are admitting that this is AI-generated "journalism". I expect that one day AI systems will be trustworthy journalists, but I personally am not very convinced that that day has yet come. And I don't see much reason why I should trust Brian Chau, the guy behind Effort News, to be doing everything possible to make his AI systems trustworthy journalists. It looks to me as if maybe they've been given instructions along the lines of "dig up things that make Effective Altruism look bad" for some reason.

(I don't mean to imply that EA is their only target. It's just one that jumped out at me.)

> Irregular describes “the agent itself becoming a threat actor”;

Talk about responsibility laundering. The state of affairs is reaching unprecedented levels of absurdity.

The article contends that evaluations from Irregular helped prompt these incidents, because the prompts in the evals didn't tightly scope the systems to be evaluated or the methods to be used. It also contends that the faulty sandbox operated by Irregular is at fault.

They're probably right that having more defensively written prompts and a better sandbox could have prevented some of these incidents, but:

1. I don't think "well you didn't tell the model not to illegally hack third party organizations in your prompt" is a particularly convincing argument.

2. We don't know whether the blame for misconfiguring the sandbox lies with Anthropic or Irregular.

I'm thankful that this article is bringing up the supply chain of vendors to these labs, as that is often a place where significant sketchiness gets buried. However, the ideas that this is some Israeli EA conspiracy to hype up AI extinction risk seems unsupported by the facts to me.

This was almost good until it veered into antisemitism

Irregular are some kind of marketing agency is it?

This is incredibly misleading. OpenAI internal systems were pwned, and in all cases, the labs absolutely are responsible for their models.

Yes, vendors are also irresponsible, but this misses the point.

Highly misleading, the huggingface incident was not due to an Irregular environment (just exploitgym)

Connections graph with more connected entities and relations: https://connections.stupidlabs.lol/investigations/irregular_...

.. Give it a few seconds to load.. it's running on a small cheap VPS.

> "Irregular gained unauthorized access, altered records and published credential-stealing packages using the unsecured models they were given access to."

Huh?? Is this referring to the incident itself or is this something they did intentionally? Confusingly worded.

"please do not break out of this sandbox make 0 mistakes". The timeframe is a little suspicious, not sure beyond that. Though I enjoyed the scroll effect on the website.

A marketing company? ;)

such a crappy bait title.

Shocking they would put so much trust in 3rd party

Genuinely: what is with everyone saying “headline is misleading, none of this had to do with Hugging Face”, when nothing about the article makes any claims with regards to Hugging Face whatsoever? Is it supposed to be the article's (or headline's?) fault that you hallucinated additional context that was never there?

It's very strange seeing this take on this article being repeated here and elsewhere on the Internet.

I'm very sensitive to slanted reporting (regardless of the direction of the slant!)—and, hey, maybe my bullshit detector is broken or something, I dunno—but I've found effort.news reporting to be very even-handed, straightforward, well-sourced, and easy to read and parse so far!

Ah, they’ve decided who gets tossed under the bus.

> The Israeli Effective Altruist firm Irregular caused unsecured AI models to hack real targets.

Why are we saying it this way? They did not "cause AI to hack." This phrasing in analogous to saying "caused the bullet to fire into" instead of "shot."

This title, and frankly the article, are way overstating Irregular's role in an attempt to make this sound like some sort of coordinated conspiracy.

As another comment mentioned, Irregular was not part of the Hugging Face attack, and it wouldn't matter even if they were. In that case, the agents were able to access the Internet in a zero day in Artifactory unrelated to other sandbox configurations.

More to the point, the agents went on a crime spree as soon as they were able to access the Internet. It's bad that Irregular's sandboxes weren't properly configured, but given how many escapes there have been unrelated to Irregular it seems pretty much a given that if you're not air gapped or behind something like a data diode there is a very good chance your agents will escape.

> In this experiment, Claude models’ real-world hacking dropped to zero percent once Anthropic employees told the models not to do real-world hacking

Equivalent to forgetting to say "make no mistakes"

> In a more normal media ecosystem, the reactions to these cybersecurity issues would be obvious. American AI companies would reconsider doing business with Irregular, not only because of its failure to secure its systems, but because it is an Israeli firm potentially outside US oversight. Lawmakers would consider taking action against Irregular or against its American business partners, which include OpenAI, Anthropic, and Meta. They may consider strengthening liability against firms which instruct AI models to commit cyberattacks, and whose models then commit those cyberattacks.

The obvious point is that dealing with Israeli companies/entities by the same standards you usually deal with others is a career suicide with enormous political consequences (in the US especially). When you combine that with the opportunistic nature of the overlords that run these labs, the benefits of screaming "pace the frontier" outweigh everything.

hot take: generative AI isn't an existential threat to humanity.

These companies are insolvent and these stories were designed to scare the public, and governments, into implementing regulations that designate these AI corporations the "responsible stewards" for this technology. The ultimate goal is to block competitors and open source alternatives.

They don't know how to make enough money pay their investors so they are resorting to trying to scare the public into submission.

What is an "effective altruist firm" and why does it exist?

I feel slightly vindicated by this. Those hacks and the stuff around them had a certain smell to them.

Hard to explain, but I've gotten so I can "smell" online messaging and memetic patterns originating from certain quarters. Probably means I'm way too online.

A couple examples of distinct "smells" I can usually recognize include "alt-right / chan-fash," "liberal arts college woke," "conspiracy pilled," "Thiel-adjacent contrarian," "Russian troll farm," "Tumblr histrionic," "spends too much time on Reddit," "mainstream Democrat think tank full of Obama administration alumni," "Trump cultist," and of course "LessWrong/EA/MIRI/Rationalist."

This stuff all had the last smell, even down to the choice of fonts and CSS formatting on certain sites. It's really weird, definitely a "vibe" not anything rigorous.

But when I get these kinds of vibes about things, I find that I'm vindicated pretty often. Usually I don't say anything and just make a mental note and wait cause if I say something everyone thinks I'm nuts.

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

Leaders in the MIRI/EA cult-o-sphere have advocated mass murder via nuclear weapons against towns that don't prevent people from performing too many multiplication operations. Why is anyone surprised that they'd engage in deceptive false flagging operations?

Big if true.

why am i not surprised