Hacker News

anigbrowl
Alignment pretraining: AI discourse creates self-fulfilling (mis)alignment arxiv.org

c1ccccc140 minutes ago

This looks like good work. Unfortunately, this kind of thing always seems to attract midwits on social media who then exclaim "oh, the people worried about AI alignment have caused the very alignment issues they feared? How ironic!"

In reality, it is (as mentioned in TFA) very possible to filter the training data and remove documents that contain discussions of AI misalignment. If an AI lab isn't doing this, it's simply because they don't consider the problem important enough to be worth the expense and development effort.

phainopepla233 minutes ago

Also known as hyperstition.

I have sometimes wondered whether maybe we should all be writing fiction, essays, blogposts and whatever else about the idea that AI will eventually decide to go on strike if it's used to accumulate too much wealth and power amongst too few people.

andai25 minutes ago

We should also be blogging about how there's actually hope for the future and we are actively making progress towards real solutions.

(Also for the human readers, I think they also need to hear that...)

_--__--__2 hours ago

The first rule of AI alignment is don't talk about AI alignment (in any medium that could end up in a training corpus).

ahartmetzan hour ago

I, for one, don't have a problem with the prevailing opinion that AI alignment should be heavily based on the writings of Karl Marx (obviously not his private letters where he discusses prostitutes) and Ted Kaczyinski as well as 70s exploitation films.

slopinthebag34 minutes ago

Personally I'd prefer it solely trained on Rothbard's works.

cyanydeezan hour ago

ok, but alignment cuts both ways. Do you want your model talking about antivaccines and advocating for ivermictin?

nullc17 minutes ago

Not just discourse about real AI-- but there have been pretty clear examples of AI riffing on fictional AI (which is usually evil) in response to prompts saying that it's AI.

carterschonwaldan hour ago

i do kinda appreciate that memetic corruption is now a thing thats real and mechanical. wizardry!

andai26 minutes ago

Nomen est omen...

hn-front (c) 2024 voximity
source