Hacker News

Or, we just need to get this over with and declare any digital data findable via the internet to just be public property of everyone. Everything becomes public, besides stuff you keep locally, and there is no difference anymore, it's all just data anyone can use for whatever. A 1 year grace period for everyone to pull stuff off they don't want to be a part of this bright new open era, then we just scrap everything related to intellectual property, copyright and similar stupid stuff, and slap UBI on top of all of it for good measure.

You could sidestep it by running non-permissibly licensed training data that you purchased through an LLM. Legal attitude so far seems to be that this is transformative as long as it's not 1:1. The question on whether or not the end result is copyrightable of course remains controversial and inconsistent, but that question is also fairly irrelevent. You don't get more libre than public domain.

That's a fair amount of computational and labor overhead mind you, as you'll need to verify and prune the quality of your mountain of synthetic data, but certainly possible.

Though this assumes the legal system is a rational actor playing by the set of rules it claims to. In fact, I highly suspect you could get very unlucky and get an unfavorable ruling against you, because you stepped on a big pile of money's toes in the process of doing this.

Hear me out.

Decentralized unstoppable storage, combined with decentralized unstoppable training, sorta like SETI for AI training. The seed of this tech already exists with IPFS and others like it.

We know (some? all?) of the big labs have skirted copyright laws at one point or another. Truly open models would just build on what is publicly available.

UAE's IFM / LLM360 MO is indeed "fully open source" LLMs: https://www.llm360.ai/reports/LLM360-Towards-Fully-Transpare...

Eventually we'll just construct 100% synthetic training data that can reliably reproduce pretrains and fine tunes.

The first broadly useful fully open source models will do this.

We already have open data / open code / open weights for some domain-specific cases, such as audio models trained on large open datasets, eg. Tacotron / LJSpeech from waaay back in the day, though that is certainly not SOTA anymore.

Distillation could possibly be considered an early case of this as raw AI outputs are themselves not copyrightable unless humans enrich, filter, or transform them. Granted, that does not handle the cases where the outputs are sufficiently similar to copyrighted original works.

Why do you make the worse choice and not use what you would prefer to use? You have been able to "get shit done" on Linux for nearly 30 years. Have the courage of your convictions.

And that's why companies shouldn't fear opening up, but having both is still a net benefit.

Without open-source, there'd be no macOS.. So good thing, it exists.

which is why everyone runs docker on mac, to get shit done.

we get shit done on the cloud with the former rather than the later

I personally find the analogy unconvincing, the UX dimension is completely different as I can use the same harness with any model; and the year of the linux desktop is coming soon (tm)

I'm sure anthropic didn't want to fund the extra "safety" guardrails they put into fable, but they were forced to, else they couldn't release it.

Sure there are all kinds of problems with that situation. But it still demonstrates that they can be coerced: play nice or don't play at all.

In this case, the fully open source pipeline is probably as valuable or more so than the weights, so releasing early has some justification.

32B performs worse than the 7B model so I'm sure they will improve it

It's actually just a coding interview test that I liked to ask in the past. You can find it and others on leetcode.

The reason I personally like my question is because it's pretty close to some of the real world work we do. It's mostly mundane and easy to bang out, but really easy for someone to do a n log n solution where an n solution exists.

A good example (but not my question) would be something like

"I have a list of People objects with a `first` and `last` name. Write a function which groups together all the People with the same last name in `your language of choice`"

To be clear, the question wasn't a complex one. It was more on the level of "could I use this for a fast inline coder" IE, single somewhat simple function question.

I wouldn't have dreamed to use this as an agent model.

7B models of the past have been able to pass this question. I've not tested it on a 4B model until now.

Same for smartphones. There was a time of unlimited hype and secrery around new iphones. Who remembers the story of an iphone 5 prototype left a bar. Journalists were going crazy, people were signing petitions for Apple to not hunt down but instead forgive the employee that made such a grave mistake. People were offering millions to buy the prototype so they can brag they got the new phone 2 weeks before everyone else did.

Or who remembers the dancing disease of 1518, were people would stop what they are doing and start randomly doing the same dance. The lords? Out of their minds. The priests? Terrified the devil had taken hold of the flock! I have come to believe that it was probably some tik-tok like hype trend of doing a fortnite dance while waiting in line for bread and communion. And the energy back then, like now, was off the charts.

Hype and memetic trend seeking encoded deep in human psyche.

Repetitive Strain Injury inverts this statement

In fact, automated a11y checks and tooling is quite advanced nowadays and tragically underutilized by web developers. Now that we have llms to scale all the shitty code of front-end devs at startups, I feel increasingly hopeless about things ever improving

But it's missing the all-important charts that the blog had before it started asking for authentication.

fair, I stand corrected on the data being there. the part I'm still holding judgment on is whether the full training recipe ships too, not just the raw bytes.

[dead]