KolmogorovComp19 minutes ago
I'm sympatic to open3dlab but I think the focus on adult-content is just trying to spark outcry for cheap.
It has been shown times and times against that removing adult content from the training source (at least for diffusion models), results in generally lower-quality output, even outside of adult-content, particularly in terms of representing human bodies.
hypfer31 minutes ago
Only tangentially related, but I've recently realized that unless you're some sort of business with a valid requirement of that, you can just block incoming traffic from all of AWS, DigitalOcean, Azure, etc.
I think the spark came from some other HN comment I saw? So thank you for that, whoever wrote that.
That alone is not a sufficient strategy of course, but you can get rid of a lot of bots by just refusing to talk to where they tend to live.
Of course, if everyone were to do that, they'd eventually pivot to residential proxies. But I don't see that happening.
jeremyjh9 minutes ago
There have been a number of articles on the frontpage of HN about the residential proxy traffic. It is happening. This was one of the most popular:
hypfer7 minutes ago
I know, but it's an extra step, more expensive, and less legally solid.
None of these measures alone are a full solution, but if you stack a bunch of cheese slices, eventually, no holes line up anymore.
stopsayingsorry8 minutes ago
>But I don't see that happening.
Then you are not looking. Already have to switch to resi proxies from time to time when encountering yet another smartass who decided to play security theater and block datacenter ips on his shitpile he calls website, because "OUR USERS MUST BE GOOD BOYS WHO LOVE TO LIVE UNDER SURVILANCE AND ALLOW ISP TO TRACK THEM 24/7" (and before you say something like "others can track you too!", your ISP has the ability to knock on your door, they should be given as little information as possible, let them do metadata analysis on per-person basis if they really want it, but no 1-click mass surveillance shit).
If you are valuable, you will be be scrapped. Turning user hostile in a hopeless endeavor to "protect yourself" is not the solution.
hypfer4 minutes ago
I can see that you're having strong emotions about this, but that rant is just hard to read and serves primarily as venting and not really to convince anyone of anything.
Unless one zooms out and thinks "okay, these guys seem to react negatively if we do exactly that. We should do more of it", which is probably not what you want.
sampullman3 minutes ago
It's hostile to you personally, but I wouldn't call it "user hostile".
Blocking "scrapping [sic]" is also not security theater, you misunderstand the intent of GP.
cube0016 minutes ago
I initially thought this was a good idea too until another comment pointed out you'll lose most business or personal customers in offices because they connect through VPN gateways running on those cloud services (especially Azure)
hypfer8 minutes ago
True but the question is if that matters.
I would argue that more often than not it might not. Really depends on the specifics though.
A lot of stuff on the web actually isn't really relevant for all audiences everywhere I'd say. It just happened to be accessible by all, because academics back in the day dreamt of a better open world that didn't really manifest :(
gunalx17 minutes ago
Allmost all the bottraffic I see is from random residential proxies though.
timpera10 minutes ago
Same, I just checked and this week the vast majority of the bot traffic I get is random residential IPs that all pretend to be the exact same Chrome 155 on Linux.
sgerenser2 minutes ago
Probably all people that fell for the “superbox” TV streamer scam.
tommica27 minutes ago
true! but how does one find all those ips?
Xylakant26 minutes ago
That’s pretty easy - all major providers have public lists of IP ranges, usually as json files.
hypfer27 minutes ago
Just ask RIPE
bsenftner2 minutes ago
If that data is at all important, why is is downloadable?
bijowo167612 minutes ago
if you put stuff available to download, dont be surprised that someone will download it
What is the actual news here, can anyone explain ????
freecodeioa minute ago
honestly for me is the hypocrisy
we've spent the entire 2010s with lots of talk whether creating a scraper for google or linkedin and packaging that data to sell is illegal just for the entire thing to turn around where large companies are basically burning small servers down with requests to do the exact same thing
GL2626 minutes ago
maybe stupid question but how is it possible to not be protected against supercrawlers like this, when you own large amounts of data, specially if all requests are coming from the same server ? Aren't cyberprotection services like CloudFlare work against this kind of "attacks" ?
Steve1638411 minutes ago
Yeah, I can imagine it being a problem if it was from different IP addresses, but surely putting a limit on requests from a single IP would be a straightforward fix? Although maybe until it happens, you don't think about it.
semiquaver2 minutes ago
So what? You made your content available and someone downloaded it. Why is this even notable?
j-pb10 minutes ago
3$ in outbound traffic on a public service, how is this news?
dofm5 minutes ago
I mean, it's not the size of the download, it's the unauthorised scraping of community assets strictly against terms and ignoring robots.txt by an American corporation. Resource of a kind that used to get kids threatened with spending most of their life in prison.
But I agree that this kind of theft by US corporations is no longer news.
(Yes, it's theft: they were aware they may not do it and according to the article attempted to get around further restrictions. I mean it's literally even theft of the $3 you mention — you don't have to be in possession the $3 to illegally deprive its owner of it under common theft statutes)
swiftcoder18 minutes ago
I wonder at what point the Roblox kids all reach 18 years old, and Roblox can pivot them all to a separate adult version thereof...
peridotintellig14 minutes ago
I'm not even impressed..
caaqil24 minutes ago
Who actually cares? Isn't the whole premise of the AI revolution just gorging ourselves with other people's data so we can have AGI? I was under the impression that we have accepted this sacrifice with OAI and friends leading the way.
Steve1638411 minutes ago
Who's "we"?
tempfile20 minutes ago
"we" in this sentence is really a handful of encloistered silicon valley weirdos. Most people do not like it and have no real say in the matter.