Hacker News

I was there at the start of the smartphone boom. I built a demo Android device that was capable of telephony/data, 3D rendering, etc. all before Google open-sourced the OS, for a SoC vendor that wasn't in Google's inner circle. Yet the industry was not interested in my junior profile during the subprime crisis.

Good points, but I think we can expect the AI space to be more tumultuous.

What most people want from mobile technology is for it to work, not too expensively, and for it to get out of their way.

What most people want out of AI is for no leader to emerge and wield supremacy against the rest of us. People are afraid of it in ways they weren't afraid of mobile, so they're more willing to work together against whoever is in the lead.

Its more like an arms race and less like a utility. The disadvantage I face when my competition has better mobile coverage and bandwidth is minor. The disadvantage I face when my competition has better intelligence on tap is much more significant.

This is super interesting because I moved from mobile telephony into ML and data science, and information theory and working with data in statistically correct way was what helped me! This was 10 years ago though.

Also, in a few years, LLMs will be building the next generation of LLMs anyway, probably autonomously to a high degree.

It's like looking at the early internet and then suggesting teenagers should write browsers as their projects instead of webpages.

How far into the future can you see?

That seems like a perfectly good use of time for a kid in the 70’s! You’d learn a lot of engineering skills and demonstrate a tenaciousness that most people don’t have. I don’t think the goal is to predict what will be the important technology in 10 years. The goal is to challenge oneself with hard tasks and learn interesting stuff. A lot of the “AI” people today were compilers people yesterday

There are more career opportunities building rockets than designing rockets. Lots of welders, machinists, electrical engineers etc build rockets, and those skills transfer.

The same is not so different for AI. A few people design novel AI, but there are a lot of people training AI (especially if you include fine tuning) and implementing AI, even as a hobby.

Building an LLM covers a lot of CS fundamentals and forces you to do lots of research in order to implement one, especially on more modest hardware.

When I was 17 and still witnessing Apollo Moon launches, I wanted to build an AI that would handily outperform LLMs as we know them today.

But that was way back in the early 1970's and all I had to work with was a mainframe.

Well the mainframe itself wasn't bad, the real show-stopper was that I didn't own the computer outright, no strings attached, no debt, etc.

>I'd probably try to make an LLM that I could use on some specific problem.

I thought so too back then, still do so I guess this is one of those things that could stand the test of time. I always wanted to start with something a lot simpler than a Moon mission myself. At 17 I already had a significant breakthrough in the chem labs and it was from alternatives to a single processing step plus everything that descended from that, rather than trying to tackle a much more complex detailed multi-step synthesis. I was only 17 but I was not trying to be a slouch, I don't think pg was either at that age but his advice is not for just anybody. I couldn't have done it if I hadn't made major progress since being 16, and it really emphasized at the time how much maturity can make a difference. My imagination ran wild as I extrapolated :)

In a reply from LeCun to pg:

>>I'll figure out a set of methods and architectures beyond LLMs that can quickly learn to perform physical tasks as efficiently as humans and animals. That last item is also what I would if I were 30, 40, 50, or 66 years old

I see no reason to stop at 66 either ;)

But I figured that people owning more computer power than I could ever afford were going to be doing something like this as soon as they could, without having to wait for something like an LLM to arrive before getting peoples' attention.

It did seem like things were going to take longer than you expect, so it's pretty good to have a lifetime of concentrating on the specialized natural science domain expertise, focused now for 50 full years on how it would combine if AI ever got good enough.

Both the natural science and the AI need to be a major cut above, I still see dramatic room for improvement in my own work. If I'm going to have to rely on "other peoples' AI" then that natural science component is going to have to pull a lot of weight to keep up with the kind of computers that only rich-as-hell high-rollers have access to.

But he’s not saying that it’s a career opportunity.

It seems to me like he’s saying that doing this thing would be 1) fun and 2) a great way to become employable in the future. I don’t believe he’s saying that this project would be some kind of job training exercise.

Game cheating and reverse engineering MMO backends taught me a lot: databases, networking, securing a backend (and frontend), limitations of simpler languages when comparing them to more native options for building backends.

You haven't actually refuted their premise.

I don’t think that’s correct, the data is not that special either, and getting a similar dataset is significantly easier than getting the compute capacity to use it, even if they are both relatively hard.

Probably this also is too cynical and simplistic, but: really what’s special is the ability to get this kind of capital, with the freedom to burn it on mad moonshots, with long enough leeway to actually get to see a few of the moonshots come true.

No wonder that the head of YC made this happen, this is exactly what they are world-leading at.

> AI is the subtrate the future runs on.

We don't know this. So many people are simply claiming this confidently, and a lot of them are betting their careers on it, but nobody has a crystal ball. Whenever someone tells you confidently, and without any doubt or qualifications, that something "is the future," be skeptical.

I remember when the Segway was definitely going to change urban planning worldwide.

I don't really agree, I think the future should run on humans.

AI is a great solution in niche areas but generally doesn't make much money. All the large companies are in the negative.

The steam engine was less of a bubble and was much more revolutionary and had a greater impact.

> AI is the subtrate the future runs on.

Current AI can automate significant amounts of grunt work in programming and math. It's good at running web searches and writing summaries. There are a few other niches where it is currently successful. But other than that, many corporate AI projects are spectacular failures.

So just given what we have in hand, assuming no further breakthroughs, then we're maybe looking at AI being somewhat bigger than the Internet. Which would make it a revolutionary technology, sure.

But to get from "a revolutionary technology" to "the substrate the future runs on", then you need to assume more breakthroughs: long-context operation over weeks or months, displacing human workers 100% instead of 75%, and the ability to directly economically compete with actual humans. And people are investing literal trillions of dollars to make that future come true, without really thinking through what truly competitive-with-human AI would actually mean. We might be looking at massive job loss, centralization of power, fully automated "companies" with no humans dominating markets, and other dystopian scenarios.

And in those worlds, it's unclear that being good at CUDA and matrix math will be all that helpful, careerwise. The AIs are already pretty good at that stuff. Data scientists get paid OK when they actually get hired, but it's not everything college students were promised in the 2010s, either.

We can't yet build a fully-general competitor for the human mind. But we're getting closer. And if we ever do build one, the consequences will be really weird in any number of ways. So I worry about visions of the future that assume AI keeps improving significantly, but that also assume it still somehow remains a "normal" technology that doesn't, for example, render most humans fundamentally uncompetitive.

How did that edge manifest?

"I'd build the foundation of knowledge to base a startup on later"

what kinds of startups ?

You can do plenty of "real engineering" under normal conditions. But specifically when it comes to LLM engineering, no there's really not much you can do, they are called "large" for a reason. You can play around at small scale, but those lessons you learn will not be very relevant to the real problems in the market.

Sure you can gradually climb the ladder by demonstrating your skills bit by bit and getting access to more resources. It has very good prospects if you do manage to push through. But it's a hard and risky path, and you will not be able to get any interesting results for the longest time.

For a young middle-class student, it just doesn't make much sense. You can do much more impressive and impactful things with your time without getting into that black hole.

I know how to build an LLM, I know plenty of fellow young engineers that do too. It's really not that complex. But they can't do much with it without capital or access.

Good engineering has never been a bottleneck in this field, it's been all about having access to capital and taking smart but dangerous risks burning it on compute, without much idea of how long you need to keep burning for. There's still no end in sight, some are still managing to convince investors and keep burning, and we are seeing progress, but the business case is still unclear. If you want to get in that game, go ahead, but it's not something I would advice the average young engineer.

But can you _sell_ that? If you can't you can't get a job doing it.

Interesting/capable diffusion models are much smaller than similarly interesting language models. But yes you could always scale things down to learn the fundamentals.

A lot of startups succeed by lying to themselves about the quality of their solutions and focusing on selling a marketing story.

I find the fine tune approach more interesting than straight to RAG and MCP.

End of the day they're all customized data stores and protocols to interact with them. May as well stick to a uniform toolkit with fine-tunes.

Not that other tools aren't useful. But reaching straight for a bunch of infrastructure reliant services is like jumping in with k8s when you're still at a stage where basic mocks in code are sufficient.

I won't roll my own encryption or UI lib but want to stay focused on the incompleteness of the project I have to ship not all the buttons and knobs of some dependency or framework. Same old manage context switch problem.

Both of you are right. There is demand for tailored (fine-tuned) models; almost every enterprise would theoretically benefit from them.

But there are also a lot of prerequisites, namely does the enterprise have its sh*t together on a technical level. Does it have the processes and data pipelines available to train and benefit from these models? Probably not!

Applied ML is at the crown of a tech pyramid whereas most enterprises are still struggling at ground level. Being able to build from be ground is likely a safer skillset than only knowing how to work at the (non-existent) apex.

This is kind of different though isn't it? Doing an assembly or compiler class has pretty clear benefits in this regard.

But LLMs are tools. Does a great engineer need to know how vscode works? Might be helpful to understand how extensions work, LSPs, and project configurations.

Usually when working with any tools, you need to understand how to get the most out of your tool for your needs and that's about it. Core fundamentals about how software and hardware works in general seems like it would be MUCH more useful than LLM core knowledge.

This is roughly how GPUs for neural networks got started: after Andrew Ng left Google Brain, he no longer had access to a 10,000-CPU cluster used to train the original DistBelief system. But his Stanford students could buy a GPU...

> Current models are clearly nowhere near the efficiency limit (the brain does vastly more with far less power).

I think this is disingenuous. One could say that drones are nowhere the efficiency limit either: a bee can fly for hours on the energy contained in just a few milligrams of honey, while our best battery-powered drones can't stay airborne for more than 30 minutes. But comparing energy efficiency of electric/mechanical devices to their biological counterparts is not an apples-to-apples comparison. There's a world of difference between the energy storage and delivery mechanisms.

And as many have pointed out already in the siblings, it's not just about the compute but the access to petabytes of training data.

It never was native "too hard" it was always "too expensive".

That's the same case finding companies that will actually pay for hand made LLM instead of using something from big providers will be hard because most companies won't be able to afford it.

Yeah if you find a company that will do that stuff directly, good for you, but you will have to be very lucky and you will have to compete with other people who followed PG advice.

So I would rather learn all there is about properly using LLMs and integrating them with existing systems, that will most likely by useful for 90% of companies out there.

Building business niche harnesses is in my opinion much better direction. Knowing what will work best in specific cases is it FTS or vector search, optimisation of usage, getting best results while using cheaper models, knowing how to use tools to run models on the servers, and all the tooling around that like various MCP or just tooling that will be provided to models.

That is what I am currently busy with and I already have customers for that knowledge.

Oh yes I agree, LLMs are not that complex in principle, most engineers could build a toy version completely from scratch without too much difficulty.

But that’s the tip of the iceberg. If you have any ambitions of doing this professionally, it quickly becomes clear that all it’s all about knowing how to deal with problems that are only present at massive scale, when an LLM is actually L and becomes AI.

The mundane details about how to build a tiny autocomplete model and the maths behind it you can learn in a couple weeks easily. It’s not black magic, there are much harder areas of computers science.

> +1, I have a friend, math PhD that's been working on ML research 5+ years in London yet he has not been able to find any position.

Maybe people simply don't want math PhDs but something else? Since 1-2 years ago I started doing consulting/freelancing in the ML space, but more on the infrastructure, deployments and similar stuff, as a general purpose developer, and I have a waiting list of clients interested in more work, some of them even trying to recruit me to work for them full-time as well. I'm based in continental Europe, fwiw.

As always and everywhere, it's who you know (and who knows you) that matters.

I mean it makes sense right? Anthropic for instance has like, a couple hundred staff in London with plans to expand to somewhere just shy of a thousand. There are far far more ML/Maths/CS/Stats PhD's than there are openings. Especially in London there is no shortage of suitable candidates given Cambridge/Oxford/Imperial/UCL are surrounding it. 2% of the UK population has a PhD alone...

That’s great but the industry has moved on to code generation and editing, not code completion.

Can you tell me what specs your machine has? There is a difference between a few hours and a few days

The argument is sound, but the maths don't math for now, and it's unclear when/if they will.

For the time being, unless you truly have millions, the outcome from training will be very net negative, while focusing on building on top of existing AI will yield amazing things if you apply the same talent and effort.

When it does get cheaper, then it will be easier to acquire the skills and experience too, and the struggle you went through by trying to do it now will be somewhat wasted.

Besides, I am well versed in this field, and it is not rocket science. There are plenty of software engineering domains that are a lot more challenging, like high-end graphics, large-scale data engineering or kernel programming. People will learn to train LLMs when people want them to.

Right, just like companies don't use SAAS.

In reality, enterprises are happy to offload even risky tasks to others as long as they get some contractual guarantees about their data. Would they like more choice in who to buy from? Yes, but not enough to in-house such a specific discipline.

The cost of training a model from scratch is going to be cost prohibitive for the vast majority of companies (even if renting the hardware needed for the 1-2 month training time). It's an interesting learning exercise, and some of the things learned can be applied to other parts of the process. There's also the issue of needing a huge amount of data needed to get decent weights.

Fine-tuning a model or LoRA based on the companies data set is more feasible but you're likely going to need several runs as you test/try out different base models, parameters, etc. This is why there are a lot of fine-tuned models on huggingface based on base or instruction-trained models from the larger AI companies that have released open weight models (Microsoft, Google, IBM, Mistral, DeepSeek, Qwen, etc.).

Training is limited on memory first (storing training data and weights) and computation second. Realistically you need to own or rent 2-8 H100/B100 devices or Google's TPUs.

The majority of workflows for a company providing AI capabilities are likely best solved by tailoring a system prompt for the chosen model, evaluating the prompt and model with tools like promptfoo, and then running it on a compute cloud provider (including AWS Bedrock). If the company is big/financially well off enough they could look at buying the hardware needed to run it on their own servers.

For other uses like agentic software development you'd need to spin up a suitable model on a compute cloud provider (or local hardware if the model is small enough) and then tell your IDE/editor to use that model. You would need some way of benchmarking and evaluating the models to see if they are capable of doing the tasks you need. -- There have been some tests done by people on YouTube that suggests that Qwen 3.8 27B is a decent model, but your needs may vary.

Most companies that build physical goods don't care for one second about their IT department other than how much money they can save per month, starting by outsourcing whole of it, thus they have little use for internal LLMs.

> You’re missing the point. Understanding how Unity works fundamentally makes you a better Unity dev.

Writing your own game engine makes you realize that the Unity engine is not really that well written....

Except knowing how LLMs work don’t actually provide much understanding for using them. People don’t use LLMs the way we’ve built on most other tons or platforms. It’s more learning Unity hoping to be a better gamer.

The big question is whether companies hold enough proprietary data to do useful things that for e.g. Anthropic, etc. can't easily replicate.

For some very niche cases I think this is probably the case but for the vast majority, the company's data isn't as useful as they think it is or anywhere near the size needed.

Everyone says they want to do training, because it's sexy and an easy way to justify raising mad funding rounds. Some manage, most don't.

I don't know where you are located, but in EU, in China, and yes even in Silicon Valley, the vast majority of companies do not do any real AI engineering. There's nothing wrong with it, it's just not a smart path for most purposes. You can do amazing things without training, and if you try to train, you cannot get anything amazing unless you burn millions.

Very few people can afford to play the long game and cross that dessert. And, sure, you will not get far without good engineering, but good engineering is definitely not sufficient and is not the primary bottleneck.

I disagree. It's absolutely possible to develop an intuition about extremely complex mathematical ideas, including llms or high dimensional systems. Learning to build an llm is a great way to start building that intuition.

I'd have to push back, though not on the part you'd expect. Your description of human researchers is roughly right: a lot of the field is try-things-and-narrativize-after.

But the load-bearing assumption is that intuition has to be human-shaped intuition. Humans can't intuit thousands of orthogonal directions because we project everything down into a 3D metaphor and hope it holds. That's a fact about our hardware, not about the systems.

And the reason why is the most interesting part: nothing requires the compression step. A model or an agent can operate over the actual objects, holding thousands of runs and ablations in context and noticing regularities in the native dimensionality, without translating them into a picture of a ball rolling down a hill. No bottleneck at "can you visualize it."

So the narrower claim: it's not that intuition here is impossible full-stop, it's that human intuition is unreliable. Your post-hoc rationalization point is evidence for that, not against it. The story exists because a person needs something to hold in their head. Drop that requirement and the failure mode goes with it.

[dead]

As a 17 year old, I agree with this. Ofc I'm against all the hate directed at PG, I believe that all knowledge has value regardless of its economic utility, but I understand where the hate is coming from. Personally, I find LLMs boring for now, and I'm more focused on CS and electrical engineering.

It's best not to take advice on direction of careers from anyone. It's better to find and work on things that interest you, and then take advice from people that are amazing in that specific field. In mid 2000s in Australia all the "top people" were telling me not to get into a software engineering career because it was dead. It's certainly challenged right now, but it took off during those 15+ years.

>What's the alternative really, seek advice from unsuccessful people?

Seek advice from the averagely successful people, since that is statistically what you're most likely to be.

> seek advice from unsuccessful people

Intuitively, I would guess that they have a better grasp of what made them fail than successful people have of what made them succeed.

The better alternative is to always ignore unsolicited advice.

Kids don’t even know what it is, even less actually feel what it is. At 17 you think that it won’t hit you, that you will be the one to survive until you don’t.

> What's the alternative really, seek advice from unsuccessful people? That seems worse :)

Learning from their mistakes (where it did go wrong) can be as valuable as.

> What's the alternative really, seek advice from unsuccessful people?

Do both. Get advice from successful and unsuccessful people and take the diff

I would seek advice from people who have a theory of why or why not they were successful. A lot of those results were happening in very specific contexts and usually should not be regarded as a blueprint, but as inspiration to whatever I do.

The unsuccessful person is much more likely to be actually connected to the daily needs and struggles of a normal person. Paul Graham hasn't seen the inside of a grocery store in the past twenty years.

I know who the 17 years old is closest to.

>Personally I don't see the problem, as long as you're aware there is survivorship bias involved here.

Pretty tall ask, especially if this is targeted at 17 year olds...

> Personally I do both

That’s what you get from listening to “successful people”. You get to learn about all the things they tried that failed, then the things that did work on that 24th try, which was successful.

The “survivorship bias” people always seem to assume that the “survivor” lucked into his fortune on his first try ever, so he can’t have learned anything, so we don’t have to listen to him. But that’s seldom the case.

I’ve written about this before:

https://expatsoftware.com/Articles/survivorship-bias.html

> seek advice from unsuccessful people? That seems worse :)

Not sure why would you think so.

Inverse reasoning is very powerful, and unsuccessful people can give you plenty of "don't do this mistake", which the survivors would not even think about.

Seek advice from people who have had a normal level of success. Not a one-in-a-million level.

Hey, I'm 19 CS undergrad but i don't really know what to do but I really wanna build something that can shape the world. Can you tell me more about why you would advice someone to learn about transistors? what are the possible career paths?

I kind of want to blanket recommend but that's not very helpful

I would suggest starting with Andrej Karpathy's YouTube video: https://youtu.be/kCc8FmEb1nY?is=oiDsrBYJg_MUUmoD

This video is excellent. I'm a huge fan. Also the video is zero commitment and instantly available which makes it a good way to check you are interested.

The book by Sebastian Raschka is slightly less accessible but very reasonably priced and the experience of working through a book is a lot nicer than skipping back and forth in a video (for me). Sebastian's blog posts on recent architectures are absolutely great too.

I've read Raschka's "Build Your Own LLM from Scratch" book and really enjoyed it. I haven't tried the code yet, but the code from his previous Python ML book worked great.

actually just use chatgpt. There is a new mode of learning thats now avaiable that doesnt require you to read about things that are already discovered leaving you with a shallow knowledge.

you can now play the inventor and start with question "i want to build next token prediction software" and go as far as you can with your current knowledge while brainstroming with chatgpt as a rubber duck.

dont not start with a course on probabality , linear algebra or calculus . do not watch 3 hr videos or 3blue animations .

there is agood video on this way to learn.

https://www.youtube.com/watch?v=cbiyPOn-__M&t=380s

Imo it's a breakdown of trust of the startup ecosystem as a whole. Repeatedly startups have enshittified and it's become undeniable that the investment apparatus around startups is partly responsible. We have seen a great driver of uncreative destruction, industries undermined, small businesses undermined just to drive masses of money into few pockets - less fairness for the people working in what is now the gig economy and ultimately prices and other costs that end up as high or higher than they were before for consumers. Not to mention the whole AI/OpenAI situation which many perceive as threatening their skillset per se, essentially tearing up the social contract that existed on this site.

The sycophancy on HN is starting to break down because there is a higher proportion of users sceptical towards the outputs of the VC and wider investment world than ones who believe they're potential beneficiaries of it.

Tech industry people are becoming less interested in HN as a warm handshake into the startup world because, frequently, they're disgusted by it. And this reflects on the sentiments people post on PG's articles.

Increasingly if those at YC want the same kind of low-bar praise they got before, they will need to get it from machines.

Because at some point in life everyone gets tired of fairytales. He started mending the anecdotes to his content which always rubs people the wrong way.

> I can't recall or point out exactly when, but there is a stark before/after moment where the opinions of anything pg went from "Interesting and maybe true in some ways" to what we see today, lots of knee-jerk reactions and hardly any comments about the actual content.

Hard disagree. This submission is still being highly upvoted, while another recent post[1] on the harms caused by Graham’s fellows[2], with a fairly tame comment section, has been flagged. That is a constant on HN. It’s not a fluke, it’s as predictable as the sunrise and getting more pronounced.

I’m sure we’re both biased in our perceptions. Mine is that HN in general (certainly more than any other website) used to worship[3] everything he wrote, together with others like Musk, until things started to really go to shit and many eyes have been opened to the effects of the unfettered greed of rich tech guys out of touch with reality.[4]

[1]: https://news.ycombinator.com/item?id=49411762

[2]: A better English word is escaping me.

[3]: That word I choose hyperbolically but deliberately. It definitely was not “interesting and maybe true in some ways”, it was much more hardcore than that.

[4]: That is not “knee-jerk” but a slow realisation still ongoing.

How exactly does one go about "tinkering" with an LLM? Any architectural change you introduce needs fine tuning. That needs data and compute

I tried to modify the embedding output of bert to make it generate box embeddings instead of point ones. At the time I had access to university provided A100 gpus but even with all that a training run took half a day. Models these days I don't think I can train it in any reasonable time with that much compute.

> Now I'm curious, do people actually tried to hack vacuum tubes or other big servers that's barely 1MB RAM?

"Barely?"

It's insane to lump vacuum tubes together with servers with 1 MB RAM. My first PC, which I used for a decade, had 640KB RAM. And that was an upgrade from 512 KB RAM. My other PC had only 128KB RAM. None of these were considered the equivalent of (by then long dead) vacuum tubes in their day.

You can get a lot done in 1 MB.

Yeah, there are retrocomputing hobbyists who mess around with sometimes-physically-large computers that were important many decades ago. I don't know if anyone is hacking on vacuum tubes of the kind that you could in principle build a computer with - there's a reason they became obsolete for digital computation almost as soon as the transistor was invented. On the other hand, I personally think it would be neat to try to build a CRT in a garage, which is of course a type of vacuum tube. I don't think this would be an easy garage project, but it does seem like might be tractable for someone who understand physical manufacturing and electronics well, has access to glassblowing equipment, etc.

Depends on where you live (lots of countries have no tuition fees or far lower than the US), what funding you have, how good a university you go it, what you want to do (some careers require a degree), whether you will enjoy it, and whether you are there just for financial benefits or more than that.

Its not good advice for everybody, but it is good advice for a lot of people. What if you want to be a doctor? What if you want to work in R & D? Not everyone enjoys working in a shop or a farm. Also, how old is your friend group? If they are mid twenties you are ignoring the greater scope for advancement in a lot of white collar careers.

> my 25 years old friend with her own shop knows more about real life, business and economy than ivy league MBAs.

Within the narrow limits relevant to her business. How much does she know about macro-economics or financial economics, or scaling up a business? I also suspect you are comparing her to people who went straight on from bachelors to MBA (which is a bad path - study business after having some experience IMO) and lack experience. How will she compare in 10 years time when those people also have real world experience?

If you’re talking purely financial benefit: yes, it is still good advice, as most white collar jobs still require a degree.

If you’re talking about a place to mature, around others who are at a similar phase of life, also yes.

It is where most people meet their cofounders, for example, even if they don’t found anything until much later.

It depends on the university and your goals, I suppose.

Many people are surprised to learn how affordable elite colleges are if you genuinely need financial aid. I had no idea -- was pleasantly surprised when my alma mater took over 80% off of my tuition.

> Is going to university really that good of advice nowadays?

Just some anecdata, but every single one of my university professors was quite bad, but they think that since they are the professor, that means they are smart and the expert.

[dead]

As opposed to what, neglecting the human experience to grind yourself to the bone for those who own capital, and then voting to uphold that capital? Makes no sense.

This also somewhat accurately describes what a VC backed startup does.

> Then vote for someone who will make the debts go away?

As opposed to vote for someone who gives even more power and money to the oligarchs? You bet your last dollar that people will choose the former over the latter.

yes?

I mean, have you seen the options for people graduating right now? How people are behaving?

Or forget the data, look at how the story of the new future technology is being told. The people making it recognize that it has the potential to put swathes of white collar workers out of jobs, and they are openly talking/warning/PR-ing about it.

People in tech and SV, the places which have a underlying culture of near delusional optimism, are talking about trying to avoid being part of "the permanent underclass".

Gambling is up, and prediction markets are being treated as financial investments. Wall street bets is a thing, and outright speculative investments are the hope people have to get ahead.

This is happening in the USA, forget the weaker or smaller economies.

When people see the future as one massive zero sum game, with no way to win by building, then they are going to change how they plan their future.

[dead]

I think the answer then is to get a Phd? I mean, one should learn it to satisfy their curiosity and to build things but I don't think it's going to help you career in any significant way.

Throw the computer and the smartphone out of the window.

Or more reasonably, the same old thing : use Linux, hack a little, why not learn programming basics. But learn to own your technology, fight against centralization of technology. The same old RMS story.

IDK where the tech industry is going, if there will be jobs anymore or not, but what I'm sure (and what have been the case for the last 10-15 years anyway) is that for most tech jobs, having good technical knowledge beyond the basics is pretty useless and will probably not be recognized.

If you can, stay a computer geek if that's your thing, but don't make it your career choice, the Eldorado is behind us.

Stay in school, get good grades.

"it doesn't matter what you choose to study because no job is safe from being outsourced overseas, handed over to an indentured servant, or made redundant by a machine. you will spend your whole life surviving while the cannibalistic pedophiles who own everything invent new ways to make you own nothing. be frugal, don't get married, don't have children, do everything you can to stay healthy and independent, and you just might live a reasonably comfortable life."

Stay away from the internet and learn how to build a local business instead.

Learn math

This seems like a much more interesting question to me.

Telling other people's children what to do is easy and basically doesn't have any downside to being wrong. With your own children things are a bit different.

So: what are people here with school age children telling their own kids about the future? If their kids ask, what kind of careers would they encourage them to pursue, assuming they have the skills and interest?

When I was last in the Bay Area, maybe about a decade ago the bookshops were full of titles like "Python for Preschoolers" (I exaggerate, but only slightly). Clearly at the time a lot of people working in tech thought that cultivating an interest in programming was going to be the path to being a successful (by some metric) adult. Is that still the case?

The real reason to recommend this is that they have an excellent chance of avoiding the "nerd-to-incel" pipeline that tech all-but-guarantees for its best nerds. Post GenAI boom, the social cost of working in tech combined with the coming collapse in high paying jobs, means that unironically coders should be learning a bit about coal mines.

In this regard, Healthcare is a polar opposite. It's pretty hard as a male nurse to not "accidentally" become a home wrecker.

> Meet the farmer's daughter, and marry her.

There’s no such thing as destiny.

You could also walk out your front door and get hit be a meteor tomorrow.

This only changes whether you are naive enough to believe “honest” business means anything in today’s age. If anything, I worry being honest is holding back smart people who try to compete in a rigged game.

“only make $115k”

This right here is why nobody is shedding tears for the massive employment crisis in tech.

You make double that shilling ai slopware while they work nights saving lives.

I would tell a 17 year old that the world will always need nurses, same can’t be said for guys sitting in their pajamas burning tokens.

What is happening now in tech has been a long time coming, and it can’t happen fast enough.

I second 1. I'm a newbie in neural networks and I think it's an excellent book! One of my barriers in ML is the resources, I find them overcomplicated or too simplistic without a mid term. It's not the case of this book, everything is well-explained. Neural Networks aren't fun for me, but this book makes it very interesting.

I think this is the best structured class out there that teaches how to scale LLMs . Hope the 17 year old knows linear algebra. Building an intuition for the shape of the matrices is important. A lot of understanding the 'building from scratch' means understanding choices like why RoPE instead of the original frequency based positional encoding. Start with Karpathy and then go to CS 336

Have you been living under a rock? We might not get "super-intelligences" (if such a thing even exists) from LLMs but they've proven incredibly useful in basically anything relating to text, including code.

No wonder the state of everything when 17 year olds are getting pressured to be “enterprising” and “get a competitive edge”. How about learning to be empathetic, respecting your fellow humans, caring for the place you live in, enhancing the lives of others? We shouldn’t be teaching 17 year olds to be greedy, selfish, self-aggrandising blowhards like Zuckerberg, Musk, and Graham. They are not good role models for the future of humanity.

he does mention that it would later on be about building a startup, which is about making money

Did you write anything for Symbian or PalmOS before WP?

God I almost invested in Xamarin after Windows Phone got aborted, I did spend a lil time on UWP, but thank god Flutter came out not so long after that. After all these years I learned to stay away from Microsoft tech stack.

Okay sure it’s not a “time machine” in that sense. He wouldn’t go back forty years or whatever. And he’s still saying the same thing that I was alluding to.

Agree with this - started programming through learning scripting in ROBLOX when I was like 12 (this was back like 17 or 18 years ago) and it developed into a life-long passion for software engineering. I am thankful I had people around me (my parents), who were aware enough to realize I wasn't just playing video games and gave me the time I needed on the computer to learn and experiment with programming...

This also meant that by the time I was actually offered to take a programming class in school (junior year of HS), I had already been able to self-teach myself well beyond what that class was covering, thanks to just working on random projects that scratched an itch I had at the time, looking up anything I didn't know or understand, and internalizing those concepts over time.

In short though, I definitely agree, young kids and teens (and also, frankly, adults too!) should be encouraged to explore things that they have a passion for, without being told 'you need to go to school for this' or 'you cant understand this at your age'

I just voiced my opinion. I just think buiding an LLM from the scratch for 17 y.o. is pointless exercise, advising a teenager to do so is borderline irresponsible, and frankly PG is simply virtue signalling here, as LLMs are still trendy, esp. in his circles.

There still will be varyy small number of outliers among youngsters who'd be able to extract tremensous value from such an excercise, but for most that'd be _IMO_ waste of of time, with illusion of understanding w/o actually having any.

> Why would you tell people that the correct order

Because I can?. JK. Because that was my experience, of someone who is 2.5 older than 17?

> For some (many?) people, a 'proper' understanding develops _after_ the exploration.

I am afraid you have a too confrontational attitude here, but I'll answer anyway: because I do not believe you can simply "explore" such complex topics like building an LLMs. You'd simply be unable to build LLM drom scratch, unless you'd call cargo-cult chaining magic numpy incantations you've taken from Karpathy's tutorials "exploring".

If I were in "exploratory" state of mins, I'd rather go from entirely different side - I'd try playing with LoRA-ing existing small LLMs, such as venerable 2 y.o. Mistral Nemo, to get "feeling" for what training is and how hyperameters influence the process.

That'd would be a terrible advice if there weren't a plenty of other things "you are not equipped for", but far less daunting both theoretically and practically. Such as, say, convolutional neural networks, or some older ML tech. Or even something totally unrelated to ML.

Transformers are difficult to understand even to people with strong ML background, let alone a teenager.

How would you filter out garbage from your training data, for example? If you are trying to use someone elses corpus, would it be "from the scratch" then?