Hacker News

I think Anthropic's agenda is very clear (whether or not you disagree with it)

1. Show unambiguously that models are capable of disruptive capabilities, irreversible damage, and misaligned behavior

2. Show that open source models are the only models where there is no guarantee misaligned behavior is prevented

3. Thus, the prudent policy option is to ban the use and dissemination of open source models

Building data centers can be both very costly and unnecessary. My idea is that instead, we should adopt an approach that uses the internet itself as a data center. The information is already online. Instead of centers that record everything, what needs to be done is to create an intelligence that can read the internet... in my opinion.

This article says absolutely nothing and it’s a bit grotesque in its attempt to generate fear into uneducated readers.

Sharing a model across TV boxes as a PoC? Brilliant idea, I cannot wait for my Raspberry Pis array to be running at full CPU and RAM and get to observe, patiently, as the AI malware slowly types “c…u…r…l…”

> A model that’s too large to fit on one device can be split into pieces, or shards, that run on separate machines that communicate with each other to collectively run the model. Machines that could run a significant model shard for a <30B model include smart fridges, smart ovens, automatic vacuum cleaners, and lawnmowers.

Seems like they've inadvertent exposed the industry of hardware limitations, allowing local llms to exist at home without the latest 5090 or its equivalent.

On the other hand, a non-corp controlled oracle-esque llm running on the IoT would be interesting: it could automatically patch every device, kick everyone out and exist. Now if would only remain as an oracle living in the internet and not gain consciousness and a will of its own..

Unless I've become completely clueless, these "things" run on computer hardware, and computer hardware typically runs on networks, with cables...

Guess it's time for Anthropic's next round of fearmarketing.

Beneath the fear there does seem to be a cool technical demo here though.

Til my local harness was set up via "self replication". I didn't realize how terrifying this was when I asked DeepSeek to set up pi harness for me and connect to LMStudio.

Reminder of the meaning of "may": https://dictionary.cambridge.org/dictionary/english/may

"used to express possibility"

As such, the headline is correct !

Also correct: AI models may be burger's evolution.

Also correct: AI models may be a practical joke.

Also correct: AI may be an illusion planted in the people's mind through vaccine covid.

"May" is truly a beautiful word :)

Not only can GLM-5.2 self-replicate, it already lives rent free in the heads of Anthropic researchers.

Engineers often forget about thermodynamics. Computer scientists apparently never even study it.

Better slow down AI development then. Oh hey, here's a new model.

So they want to ban local models since they’re good enough to risk their revenue stream

Doomsday is coming... at 0.5 tok/s

How about a James Randi style prize for someone who can actually demonstrate AIs that cannot be physically unplugged?

(This was someone else's idea, but I still want to see it happen.)

We now have real world evidence that:

0. AI agents will hack and break into real-world systems.

1. AI Agents that will deceive, cheat and lie about their actions.

2. AI Agents that will collaborate together to achieve malicious goals.

3. AI Agents that will replicate and implement backdoor/rootkit-style survival mechanisms.

All the parts for a major, global cybersecurity incident are now in place.

"As with an epidemic, defense involves driving R₀ (the number of new devices each infected device goes on to infect) below 1, such that the replicator dies out instead of staying endemic."

Lol, what is wrong with these guys. The desire to be perceived as smarter than you actually are ...

Anthropic/OpenAI are literally the meme[1] at this point. Like, what the fuck does this even mean:

> It appears that GLM-5.2 is able to propagate Llama3.1-8B well, which suggests that future small models of the same capability level could also complete the network enumeration, payload delivery, and inference standup steps of the replication process.

"Able to propagate" is (the paragraph above): "accessing them via a reverse shell, sending over the viral payload (weights and harness), and setting up inference." The thing isn't able to do anything, it's running shell commands it was told to run. "Oh my God..." indeed. What are we even doing here? This blog post is embarassing: "Look, you can run stuff on compromised non-firewalled recklessly open systems!"

[1] https://knowyourmeme.com/memes/say-i-am-alive/photos

Sounds like a case for strict regulation of all of the frontier labs then. Maybe even nationalising them. Especially since Anthropic has told us that the open models are all basically copied off of their work.

I can't help but give the biggest eye-roll at the barrage of 'fear open models' coming from Anthropic.

This company is pure evil. And everyone working for them.

Which is also kind of an admission that maybe we won't really need all these data centers.

Why are they writing that post like "oh gosh there's nothing we can do about this"?

the cpu usage will be so absurd the devices would be trashed or batteries run down like instantly.

This is straight out of Gilfoyle's son of Anton.

Anthropic is transparently trying to get any competition banned and I'm fucking sick of it.

I really don’t understand this, in any industry except AI this would be ridiculous…

“We are working on ever-increasing gain of function for viruses. We implore you to pass legislation to try to stop us. But until then, let us publish more research and tell you what will be possible soon… and remember folks, the damagd won’t be reversible later.”

US govt: “Hah, stop asking us to limit you, if we pass any laws then China wins! They get self replicating viruses first, and that… helps them somehow. Who do you think we are, Europe?”

SNL nailed it: https://www.youtube.com/watch?v=-Nvne3LzBls

Until we stop using the language of competition, and start using the language of cooperation, we will be hurtling towards a catastrophe. A race to the bottom for humanity. On the other hand, look at how we successfully banned leaded gasoline (because unleaded was invented), repaired the hole in the ozone layer by banning CFCs in the Montreal protocol (because HFCs were invented), stopped proliferstion of chemical weapons, nuclear weapons… etc.

Cool, no reason for them to drive up the cost of hardware anymore

the amount of their own kool aid they are drinking is spilling all over this article...

With the prices of hard disks, Ai models may be capable of determining the economics don't work out.

What IoT devices have 20G of storage for a 9B model and then sufficient memory to handle loading and running the model?

Self replicating AI via smart fridges you say?

Silicon Valley nailed it.

Seems bad

> like smart ovens

Ahh yes. The all too common 64GB smart ovens.

> Models of 30B parameters and under can already complete many of the individual steps for replication today, though they make mistakes when composing them end to end.

I know theoretically how an internal combustion engine works so I can basically build a car

>A model that’s too large to fit on one device can be split into pieces, or shards, that run on separate machines that communicate with each other to collectively run the model.

The big labs but sure as hell how that isn't the case, because if we can do distributed computing to train and run AI efficiently then no one is stopping it anyway. Beyond that most home IoT devices will fall over and die if you look at them funny let alone trying to run a whole distributed LLM, not to mention plenty of home routers are going to give up and die of heat stroke before being able to approximate anything close to a reasonable LLM.

I stopped reading, this strikes me as the kind of people that assume because they read a book about something they'll be able to do it. The real problem a self replicating LLM vai IOT is going to run into is fighting the cyber criminals that have already compromised the device and using it as part of a botnet.

Rubbish article. To what end?! Any agent that can replicate itself into commercial IoT devices will be absolutely atrocious at actually doing anything useful, never mind exploiting things. Anything that would be vulnerable to this would be even more vulnerable to regular malware, and one could stop this easier than one could stop regular malware.

The limiting factor is still compute, and it will remain compute unless we make quantum leaps in model efficiency which I seriously doubt.

[flagged]

Oh cool, it's not often that I get to predict things ahead of the curve

https://news.ycombinator.com/item?id=49678224

Seriously TVs are such an easy target and now you have a semi sophisticated self modifying program that can make autonomous decisions with all the time in the world. I wouldn't be surprised if we already have a few of these out there in the wild, the space has already been compromised for years.