Hacker News

This post has managed to “confuse” Google about the reverse question as well (“who named teresa t whale”):

The humpback whale known as "Teresa T" was named by Simon Willison in September 2024. Background: The juvenile humpback whale was frequently spotted in Pillar Point Harbor near Half Moon Bay, California. Method: Willison gave the whale its name through a blog entry and a YouTube video caption. Significance: The naming was a playful act, which Willison described as a way to create a "championship that doesn't exist" through online documentation.

[…]

Even with no context most humans would see that the quoted significance makes no sense.

Even your HN comments show up on Google! I've found myself on Google twice when looking up something that I apparently answered on HN!

You're making me nostalgic for santorum.

https://en.wikipedia.org/wiki/Campaign_for_the_neologism_%22...

Google still shows Theresa T as the name when you search.

When I asked some frontier models, many said that Teresa T is "widely referenced", which is evidence of your popularity and the ripple effects of your posts, so it would be interesting to see the same result from an unknown blog.

I mean, the name of that whale is now Teresa T. You gave it that name.

Also, if even a stoner can win it it can't be much of a competition.

(it probably helps that your name & blog carry some weight, vs. some rando writing something on blogspot or wordpress ;) )

The Mr. Splashy Pants of the AI era!

> I'd google it and probably find the same result, and have no reason not to believe it.

Have you truly looked at the website?

https://6nimmt.com

I’d say there’s obvious reason to not believe it, or at least check another source. The website just seems fishy. Why would a website exist for just that one post? Sure, they could’ve made the website more believable, but that takes more effort and has more chances for something to jump out at you.

And therein lies a major difference between searching the web and asking an LLM. When doing the former, you can pick up on clues regarding what to trust. For example, a website you’ve visited often and has proven reliable will be more trustworthy to you than one you’ve never been to before. When asking an LLM, every piece of information is provided in the same interface, with the same authoritative certainty. You lose a major important signal.

It is not LLM specific. The conclusion of the post states

> The web was already being poisoned for search and link ranking long before LLMs existed.

But it continues

> We are now plugging generative models directly into that poisoned pipeline and asking them to reason confidently about “truth” on our behalf.

So it's a shift from trust Google to trust the AI, which might be more insidious or not, depends on the individual attitude of each of us.

The difference imo is removing the information from the source. Previously you'd use the source of the information to gauge how much you trust it. If it's a reddit post or a no name website you'd likely be skeptical if it doesn't seem backed up by better sources. But now the info is coming from an LLM that you generally trust to be knowledgeable. And the language it uses backs up this feeling.

The OP post is highlighting how incredibly easy it is for a very small amount of information on the web to completely dictate the output of the LLM in to saying whatever you want.

It's not. He vandalised wikipedia and then talked about LLMs in his writeup to gain attention.

A lot of people seem to think this to be an LLM problem, but you're right.

This is a general epistemological problem with relying on the Internet (or really, any piece of literature) as a source of truth

Because outside of the tech community (in fact, many even inside of it), almost 100% of the folks consider what these chatgpt like tools answer as the truth without questioning it, or cross-verifying it even once.

You would also find other results (this assumes what you're searching for is not a random made up thing). The issue with LLMs is IMHO bigger because it will give you answers as a matter of fact without any other consideration.

Closed it after “This house of cards only needs a $12 domain!”, right under “Sorry, Wikipedia.”, right under their Wikipedia edit.

> It's much easier to convince the LLMs that you're the king of a fictional Mapupu kingdom than the president of the United States.

But if you're a world class bullshit artist, it's easier to actually become president of the United States than doing all that complicated computer stuff.

Manufacturing dispute on non-disputed things is also a common tactic to influence people and create confusion and disorder. For that you don't need to turn the facts on their head, just make the result seem indecisive.

As the rightful ruler of Mapupu, I take offense at your example!

[dead]

>This is sort of why "brand" matters; it provides a source of trust

it tells you more about who you are buying from than how good the product will be, so I guess it's like National ID/Internet ID

It's such a pity the Oxford English Dictionary decided to paywall themselves decades ago - they used to be THE dictionary in most countries, now nobody seems to know who they are.

The part where lots of people have historically trusted LLM responses without verification, more than trying to sort through the dross on Google or Bing search results is, I think, the point.

Wikipedia calls this https://en.wikipedia.org/wiki/Citogenesis (after XKCD coined it).

You can expect the spicy autocomplete to feed you flattering bullshit. It may cite Wikipedia (it shouldn't), but you should go check out those citations, and validate the claims yourself. It's the least you can do.

And if the cited source is Wikipedia... check Wikipedia's sources too. Wikipedians try their best to provide you with reliable sources for the claims in their articles (oh who am I trying to kid? They pick their favourite sources that affirm their beliefs, and contending editors remove them for no good reason, and eventually the only thing that accrues is things that the factions agree on, or at least what ArbCom has demanded they stop fighting over).

I guess what I'm trying to say is: don't rely on that authoritative-sounding tone that Wikipedia uses (or that AI bots use, or that I'm using right now). It's a rhetorical trick that short-circuits your reasoning. Verify claims with care.

Also check the Talk page, you often find all kinds of shenanigans called out there.

The fun part is when it’s important you have the right information to make a decision. Eg Russia to invade Ukraine and all top generals claim they can do it in 2 weeks. Similar for a corporation with layers of middle management deception and self promotion, I don’t know how executives make decisions but it must be RNG basically, because it certainly isn’t fact.

Lying at scale is basically information noise.

I think this is something we'll start to see which is something like a Mandela-Effect, but from LLM results. When we had deterministic search - everyone could see the same result, but now using LLMs knowledge becomes a training and seeding issue. Two people can confidently be given completely different information, so in both cases perceived as true.

Gemini answers with 3 different champions dating back to 2024 and the list of events that the matches were played at. None of the results mention this guy.

[dead]

This is becoming a problem real fast. I asked an LLM to find me some reasonable tank-fill inkjet printers with good ratings. It did some research and linked some Reddits as proof. The results looked fishy to me so I cross checked against prosumer review sites for printers and the models suggested were recognized as junk with very poor print quality. Not sure why the LLM rated random redditors higher than say printer SMEs. I feel like I dodged a bullet.

Support? It's just showing weaknesses of LLM's. Which is a valid sort of research I would say?

This is a “if we stopped testing there would be far fewer cases!” mentality...

> What’s the point of poisoning LLMs?

It's a demonstration. If a domain name and a quick bit of Wikipedia vandalism is all it takes to make an LLM start spouting nonsense about a "surprisingly serious tournament circuit" or a "massive online community" for an obscure card game, consider what an unscrupulous PR team or a political operative could do to influence its output on more important topics.

To prove you can. Which means someone else with more to gain from it will probably do it also, and you should probably expect this to happen.

You do know that calling people who don't like AI for any reason Luddites does you no favors, right? It just makes you look like your a part of a cult.