Hacker News

Every Windows computer has a small rwkv model on it. Wouldn't be hard at all to get decent cpu performance from a tiny malicious harness, especially one that used the self-evolving skills features and open source models.

Malware is going to be crazy, people aren't ready for the revelation of how insecure and broken things are. Everything is held together by bubblegum, duct tape, and panicked engineers putting out fires.

The abstract says:

> The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks.

I think an approach could be to use some engineered security issue or however people build botnets, and give it some AI llm that is small and minimal but comes with instructions to download models from hugging face, and some other minimal prompts and descriptions of tools. Then it could use this to grow in infected computers and try find more capable and vulnerable computers to run better capable models and also devise some minimal communication between the different points of the botnet. Perhaps set itself a goal to dominate the biggest amount of compute and have some other goal. Would be curious to see what happens.

When the worm makes someone's machine start to sound like a leaf blower, you are found out.

In the abstract, what does it mean "the attacker's marginal cost per new infection is zero"?

Unless your field of academia is digital. Perhaps this is why he wanted to attack printers on the network.

I mean, if it's online it has a network/wireless card and a TCP stack along with at least some amount of RAM, so yea, in theory unless the programming is perfect it could be exploited. Now, it's not going to be used to run AI, but could very well get used in a DDOS or something like that.

Doesn't Neal Stephenson's Snow Crash have a similar idea? IIRC, a computer infects human brains via language and sound.

In the 2004 Battlestar Galactica series, the explanation for why the Galactica was the only ship that survived a massive Cylon attack seems more and more likely. The ship was old and wasn't fully connected to the human's command and control systems and so the Cylon virus couldn't reach it.

Wrong zone.

Yeah, lab leak is hard enough to contain with human viruses, but labs have well established protocols to prevent it happening.

Computing doesn't have good protocols except for air-gapping, we really just have lots of layers of best-effort detection, and billions of devices which mix data and instruction often in a careless fashion.

I used to not believe in the dangers of AI or the risk of internet-collapse from "rogue AI", but a genuine self-mutating virus could genuinely take down the internet and need an entirely new separate net. ( Or we'd discover if the current backbone actually has the power to break encryption to stop it. )

And this time, you can bet any new internet would be corporation captured. CompuServe and AOL failed because of the open internet, but we're a very different world now, governments would support the corporation led locked-down approaches for "safety".

I don't for a second believe the capability is actually there yet, but it's no longer unthinkable that such a thing could be created in a lab within a decade. Once out in the wild, there's a lot of idle compute out there to harness for self-improvement and spreading.

Acronyms, shorthand etc. are routinely used on here to refer to US states,universities etc.

For those of us outside the US, its a minor pain of using hacker news. Interestingly, this is the first time I've heard complaint about it and its a non-US university.

University of Toronto, it's in TFA and even the URL.

My first guess was Texas...

There’s a difference between speculation and measurement, especially since you’d have people making arguments like saying that open models aren’t powerful/fast enough to work. Demonstrating this is a useful warning to everyone (most of the industry) who’s been slacking on internal defenses because they don’t think a well-resourced attacker will target them.

[dead]

this is part of the pro-active security loop. gotta demonstrate how it can break to figure out how to defend it.

our other choice is to let someone else figure it out in relative secrecy. then theyre able to cause a bunch of damage to a wide range of systems. with no defences for it. everyone would be scrambling around figuring out how to deal with it while the damage is going on. not good.