HarHarVeryFunnyan hour ago
On a related note, I was reading yesterday that apparently the real bottleneck for Chinese production of AI accelerators is HBM production, not processors or ASML equipment.
The lack of ASML EUV machines certainly hurts, and pushing DUV so hard results in abysmal yields of good chips, but you can compensate by running more wafers or making smaller chips, and the net result is that Huawei's Ascend production volume is limited by CXMT's HBM capacity not processor dies.
The problem is that HBM manufacture requires many steps (die thinning, via drilling, plating, alignment) where the equipment used by everyone else (Samsung, SK Hynix, Micron) is also blocked by sanctions, so the Chinese are having to develop all of this themselves too, which they have, but yields are currently low, even when using shorter HBM stacks.
andy_pppan hour ago
Tokens per second is almost entirely memory bandwidth at inference time, training obviously needs more compute but you can add more chips for that.
vatsachak32 minutes ago
China should invest in an analog inference chip. It's a hail mary but why not.
6510an hour ago
I read HBM yields are 25-30% (vs 80-90%) making them 3 to 5 times as expensive. They are 4-5 years behind, that probably means 1-2 in Chinese time.
throwaway85825an hour ago
Does that include yield from packaging?
fooker3 hours ago
What's the main blocker (other than the current inflated cost) for using HBM instead of DRAM as the primary memory for consumer electronics?
bob10293 hours ago
It's not that there's a blocker. It's that it takes roughly 3x the manufacturing capacity to produce an HBM package at the same storage capacity as DRAM. We are sacrificing total bytes for bandwidth.
rkagereran hour ago
I don't fully understand the source of the "total bytes" constraint, but a major factor may be because HBM4 / HBM4E can only make use of the footprint directly above the processor/logic die (or in direct vicinity of its interconnect), while traditional DRAM can be placed further away where there's lots of real estate on the motherboard.
I gather a practical max ceiling today is a stack of 16 chips in height yielding 64GB?
These chips have a massive bus size of 2048 bits, instead of the 64 or 128 bits (dual channel) used by DDR5. That's what gives them their order-of-magnitude bandwidth speedup. But even though they technically pack in more capacity per square millimeter of motherboard, I gather they take up more space than older technologies once you account for the vias and interconnects to route all those signals.
threecheesean hour ago
Thanks for that, just went down an interesting rabbit hole. Many of us were hoping this re-tooling would eventually trickle some fast RAM down to DRAM-exhausted PCs, but given it would require a rearchitecture of the motherboard it's unlikely.
tliltocatl3 hours ago
How so? FEOL is pretty much the same, BEOL is almost the same save TSVs, the packaging tech is different and more advanced, but not exactly 1:1 comparable. Do TSVs really occupy 3x the area of DDR IO's?
Const-me3 hours ago
See remark on the slide 11: https://www.servethehome.com/micron-evolving-memory-architec... That presentation is by Micron.
Karliss2 hours ago
It doesn't really say that it needs to use 3x more area, but that 3x more gets consumed due to "advanced packaging and manfuacturing complexity". Which doesn't properly explain why it consumes 3x more and could simply mean they have a bad yield and 2/3 produced is garbage.
bob10292 hours ago
> 2/3 produced is garbage.
This might not be far off the mark. You are irreversibly linking the fates of these devices after a certain stage of manufacturing. If something goes wrong at final packaging time, you lose all dies instead of one.
tliltocatl2 hours ago
Yea, that's the question. Yield situation can improve. Area overhead would not improve short of a completely new and incompatible tech.
skavi3 hours ago
tliltocatl3 hours ago
Yuck. Time to build an xSPI/HyperRAM workstation (if only these had multi-bank chips).
buildbot2 hours ago
Sadly the $ per byte of xSPI and HyperRAM quite high
tliltocatlan hour ago
Yes, but enough to run a text editor, or even a mechanical CAD. Not an LLM, but that's the point!
monster_truck3 hours ago
3x is a reasonable figure. They are not literally that large, though.
torginus21 minutes ago
Mainly bus width. Afaik HBM is like 1024 bits vs DDRs 64 so you need lots of transfers in parallel to saturate the bus, and CPUs kinda want 64 bytes of data as that's the size of a cache line ASAP. So you need a ton of in flight transfers which isn't a thing CPUs provide, maybe multicore workloads.
Buy the way you win with CPUs is with latency, and not bandwidth, which is why Apple M series actually uses DDR with lower latency because of the stacking.
phkahler3 hours ago
>> What's the main blocker (other than the current inflated cost) for using HBM instead of DRAM as the primary memory for consumer electronics?
HBM is meant to be integrated into the same package as the CPU, so no more DIMM sockets. It also has higher latency apparently.
chessgecko3 hours ago
I think people might prefer the lower idle power consumption from lpddr over the better bandwidth in hbm in battery powered stuff. That said right now the price is definitely preventing us from finding out.
fooker3 hours ago
Idle yes, but HBM energy consumption / memory operations seems to be a bit better than DRAM.
vlovich1233 hours ago
Only if you’re running at 100%. Consumers generally do not.
monster_truck3 hours ago
That hasn't been true since early HBM2 days, before the controllers standardized on power/voltage management and did things like leave them in P0 to ship on time
vlovich12343 minutes ago
LPDDR/DDR/GDDR generally still win over HBM when there’s no data being transferred. HBM is primarily better in watts/byte transferred. Consumer electronics spend most of their time idle.
Zagitta17 minutes ago
Racing to idle is a very common power optimization technique
HarHarVeryFunny2 hours ago
Why would you want/need to?
The advantage of HBM over regular non-stacked DRAM is memory bandwidth, which also requires a super-wide memory bus - 2048 bits wide for HBM4. Compare that to the 128 bit wide bus of a modern CPU.
So to take advantage of it on the desktop, or anywhere else, you need that 2048 bit wide bus, and a processor capable of consuming 2-3 TB of data per second!
These are not normal requirements, other than for a GPU.
fooker2 hours ago
SIMD (and especially the modern matrix extensions) can use as much bandwidth you can throw at it.
b112an hour ago
Without further clarification, that statement seems impossible. "As much" being unbounded and all. You should expand what you mean.
fooker44 minutes ago
This will blow your mind, but it actually is pretty close to being unbounded. :)
Consider the 'MMA N matrices' primitive modern CPUs are starting to support. For the current generation of CPUs, N is a constant like 16 or 32, but there's nothing preventing it from being 1024 or larger if we have more memory bandwidth.
All this with a single instruction.
XorNotan hour ago
Right but if HBM memory is all that people want to produce, then building a CPU which can use it use it would be useful on it's own merits.
But in reality we also already have unified memory architecture systems, integrated graphics etc.
p1eskan hour ago
People want to produce hbm because it’s more expensive and more profitable than regular memory.
XorNot43 minutes ago
There is a world where scale and experience means it's about the same though, is the thing.
And memory is already expensive. It's downright hard to even get it though - you frequently would prefer not what's cheapest, but whatever is in largest scale production.
refulgentis3 hours ago
In one sense, nothing, in another, everything. It is DRAM, but the bandwidth requirements mean it’s paired to a processor, i.e. no DIMMs. Not 100% sure but things like MacBooks and the Framework tower, where you have fixed RAM for the device lifetime, have ~0 tradeoff.
reliabilityguy3 hours ago
HBM is a stack of DRAMs, so there is no “instead”.
buckle80173 hours ago
The vias to enable stacking is a significant amount of the die area.
saltcured2 hours ago
If they're talking about production capacity, that is some product of die area and process steps, right? It doesn't have to be 3x die area, just 3x lower factory throughput for the same number of functioning memory bits.
nutjob23 hours ago
Nothing except CPU manufacturer choices. Mac laptops use it and they're consumer products.
People will have to get used to buying a fixed amount of RAM with their CPU but thats unlikely to be a problem.
fooker3 hours ago
Apple's "unified memory" marketing is so strong that even tech literate people seem to have this misconception!
They have managed to pull this sort of thing off many many times. https://en.wikipedia.org/wiki/Reality_distortion_field
Rohansi3 hours ago
Yup, the only reason Macs have higher memory bandwidth is because they use more memory channels, which gives them a wider bus. Both Intel and AMD only allow more than dual channel memory on server class processors these days.
kstrauser3 hours ago
The “Apple only does X better because they do Y” thing has been a meme for ages. I remember dismissals like “PowerPC is only faster at math because it has more integer units” or something along those lines, and thinking, uh, isn’t that a good thing?
pdpian hour ago
There's a qualitative difference between "they're doing a different thing" and "they're doing the same thing, tuned differently". GP is saying that this is a case of "they just tuned it differently".
This distinction doesn't change what the performance numbers look like today, but it does inform what changes would be necessary for those numbers to look different tomorrow. E.g. Apple Silicon isn't fundamentally orders of magnitude more efficient than x86, they just used smaller features. Newer Intel and AMD chips made on equivalent processes _also_ get similar efficiency gains.
pixl972 hours ago
Depends on the expense trade off.
If I get 10% more performance for 50% more cost it really depends on one's needs, for example.
Rohansi2 hours ago
I am just saying it's not magic and x86 is capable of doing the same. Quad channel memory used to be more common in consumer hardware but now it looks like you don't even have the option anymore for desktops. AMD's Strix Halo was the first sign to reversing that (it has quad channel memory!) and hopefully we see more of that in the future.
colejohnson66an hour ago
AMD does market segmentation and limits consumer chips to dual-channel. You need to cough up the dough and get Threadripper for quad-channel. Or even more for Threadripper Pro to get octa-channel.
torginus17 minutes ago
Afaik steam deck is quad channel, despite it being a pretty low end chip (with a decent GPU though)
Rohansian hour ago
As I mentioned above AMD's Strix Halo has quad channel memory and is consumer level. But yes, other than that everything is segmented away.
Kon5ole2 hours ago
The memory bandwidth is a small thing compared to the massive win you get by not having to move data between two memory pools at all.
fooker2 hours ago
Often you don't move memory around as a programmer, but that's exactly what happens in the background.
It's the address space that's unified, not always the physical hardware.
The data movement (when needed) is handled transparently in the background by page faults and other tricks.
Rohansi2 hours ago
Depends on your workload. And AMD has supported unified memory long before Apple Silicon existed anyway.
throwaway85825an hour ago
Except strix halo.
Kon5ole2 hours ago
Having unified memory is a real advantage though, it's not a reality distortion.
davrosthedalek2 hours ago
The price is that you essentially glue CPU and GPU together, which limits total compute, from a size and thermal perspective.
This is really not a limit because of unified memory -- in principle, PCIe GPUs could read/write main memory without the CPU. But it's a limit for /fast/ unified memory, because fast means close.
So unified memory is great as long as the integrated GPU is strong enough. Then it has two advantages: a) probably faster transfer CPU<->GPU (but that's an implementation choice for the non-unified case b) If you either need a lot of memory for the CPU or the GPU, but not for both at the same time, you pay for memory only once.
nvme0n1p12 hours ago
Agreed. That's why it's a good thing all computers made in the past 15 years have unified memory, not just macs.
stymaar2 hours ago
Is that Marcan's vtuber persona (Asahi Lina) that changed name since Marcan doesn't work on Asahi Linux anymore?
fooker2 hours ago
It is a real advantage.
The reality distortion is that people seem to believe it's HBM, or somehow it gives you extraordinary amounts of vram. Neither are really true.
KeplerBoy3 hours ago
MacBooks use regular soldered lpddr5(x) RAM. Same RAM as every other laptop manufacturer, they just use more lanes to achieve a higher bandwidth.
bunderbunder3 hours ago
Perhaps more noteworthy for general home and business computing, doesn’t it also allow for lower latency?
Rohansi3 hours ago
More channels or soldered memory? Channels are basically RAID 0 so it depends what you're measuring. Soldering memory down was the only way to use LPDDR5X so if you wanted the best memory you had to solder it down. LPCAMM2 exists though so newer devices can use that instead of soldering them down, but not all devices would be able to fit the required LPCAMM2 slots.
throwaway8582544 minutes ago
SOCAMM2 allows for removable ram in nearly 0 added space.
nomorewords3 hours ago
The normal non-tech-savvy person already does this. They simply don't know that their ram is upgradeable or something else breaks first, before having to touch ram.
dylan6042 hours ago
what is this too much RAM thing you mention? I thought the only valid RAM situation you could find yourself is not enough RAM. Too much? That's just fantasy
addaon3 hours ago
> Mac laptops use it and they're consumer products
No, Mac laptops use LPDDR, currently LPDDR5X.
chessgecko3 hours ago
pretty sure its lpddr not hbm.
glub9 minutes ago
tl;dr; This is existing production being redirected to HBM. Which means consumer market is going to get a lot worse.
And the worst thing is that is the best possible strategy for them. It's essentially win-win for everyone but consumers.
- Buyer (AI) has crazy money, so will pay whatever
- Seller doesn't have to build anything new, as the buyer is willing to pay whatever
- Generate ridiculous profits from crazy money
- No oversupply risk in case of reversal
- Return from producing HBM to DDR5 in a single quarter if reversal does happen.
Hold on to your existing hardware, people, and be on lookout for your local deals.
gs174 hours ago
A shame that this should if anything, lead to consumer DRAM prices getting even worse.
nicoburns4 hours ago
Why would more RAM supply lead to higher consumer prices?
devy3 hours ago
HBM4 and HBM4E DRAM are NOT the DDR4/5 that consumer markets need. Capacity allocation is leaning more to data center grade HBMs so less to produce dedicated DDR4/5 DRAMs. Supply demand will further drive up the consumer DRAM price! Note, the article mentions NO of new fabs is being constructed (all semiconductor manufacturers know that constructing more fabs means the boom/burst cycle will eventually kill them, so no one create more fabs) Perhaps the federal government need to step in here - the market doesn't fit the issue.
tipsytoad3 hours ago
Plus it takes 3x the wafer capacity for hbm than the same byte capacity in dram, so we’ll likely see the consumer market be decimated here
bnjms2 hours ago
HBM can be used for GPU though. I wish it was more common.
sgt3 hours ago
Might lower the prices of Mac Minis, Studios etc though
airspresso3 hours ago
No, those use LPDDR5(x), not HBM.
matja3 hours ago
Depends on what proportion of Samsung's output is current HBM4 and HBM4E DRAM.
Everything in their statement can be true and it be a bad thing for consumers of non-HBM RAM.
"HBM capacity to expand to 250,000 wafers a month"
So let's say their current HBM capacity is 100k wafers/month (pure speculation/random number for illustration), and their total RAM capacity (including HBM( is 300k wafers/month, then non-HBM capacity reduces from 200k to 50k.
gs173 hours ago
And it's worse than that, Micron said that converting capacity to HBM was at a 3:1 ratio, so that 250k HBM could be up to 750k in non-HBM. Fortunately, some of the increase is due to improved processes.
EgregiousCube3 hours ago
Keep in mind that total manufacturing capacity is increasing as well. Perhaps not enough to fully offset, but it's incorrect to assume that supply capacity is flat.
cogman10an hour ago
It'll take a lot of time.
Micron has been working on expanding in Boise since around 2023. They are predicting that the first new chips will start rolling out around 2027.
I don't think we'll see any relief until at least 2028 or even later. And even then, probably not without antitrust laws being enforced. Because you know memory manufacturers won't just simply drop prices, they have a long history of price fixing.
pixl972 hours ago
Question is what does the demand curve for regular ram look like also? If regular production is on a 10% increase curve but a 40% demand curve than we've been at the samw place.
mattstir3 hours ago
Part of the implication is that factories that could be producing consumer-facing DRAM like DDR5 would be retooled to produce HBM instead, leading to even less total consumer RAM production.
zargon3 hours ago
Memory allocated for HBM is memory taken away from DDR production.
kenny114 hours ago
If Samsung is limited in the number of wafers they can process per month and they use more of those to produce HBM they necessarily have less of them left to make other products, like consumer DRAM.
stevefan19993 hours ago
Gamma squeezed by the AI business, that's why. The more you add the fuel, the further it will pump up.
deagle504 hours ago
Presumably DRAM production capacity gets reallocated to HBM, not sure.
shevy-java4 hours ago
Depends on how quickly they are sold.
robotnikmanan hour ago
Maybe this leads to device manufacturers using HBM memory instead.
[deleted]3 hours agocollapsed
amelius4 hours ago
Will that be enough for AI's hunger?
mixedbit3 hours ago
During the Covid global chip shortage Intel announced new factories to address the production bottleneck, before the factories even started to be constructed, the shortage was long over and the projects were eventually canceled.
GoToRO3 hours ago
It will be just in time for when AI will run very well on consumer hardware and the need for data centers will collapse.
amelius2 hours ago
What if the AI companies start selling the hardware with their models?
I.e., you get a locked down device with access to their AI and maybe a way to run third party apps. Of course the developers of those apps will have to pay a percentage of their revenue.
Sound familiar?
throwaway8582541 minutes ago
More likely they would sell a chip with the model etched in silicon for the dramatically higher token/s.
konaraddi3 hours ago
There will be demand for both
IshKebab3 hours ago
That's never going to happen. By the time you can run current frontier models on your $10k desktop the frontier will have massively advanced and people will want those models instead.
blurbleblurble3 hours ago
It's going to happen very soon, which is why these frontier labs are scrambling to shut down open source language models. There's an existential risk threatening their obscene returns.
mrlonglongan hour ago
It is for that reason they are being archived and torrented as a very large middle finger.
IshKebab2 hours ago
> It's going to happen very soon
Why? You can't just assert it. There are very good reasons to think it won't happen soon, and you've given no reasons to think it will happen soon.
dwedgean hour ago
You asserted that it was never going to happen first
bunderbunder3 hours ago
I’m not so sure about that. Already AI vendors are back to cutting prices to try and keep customers from cutting back on their usage. My own employer is working hard at pivoting to much smaller fine-tuned models for established use cases, and seeing model performance improvement in addition to large inference cost reductions. Being able to run them locally hasn’t exactly been a disaster for devex, either.
It may turn out that demand for SOTA frontier models isn’t so limitless after all.
pixl972 hours ago
Every product follows demand curves. At a price of 0 you could find infinite usage. This has nearly zero relation to how much it costs to provide the product.
bunderbunder2 hours ago
Except of course it relates. All else being equal, we will prefer $X COGS over $2X COGS because that helps us with both profit margins and price competition.
pixl972 hours ago
It relates in the sense there's a minimum cost of production without losses, not the actual price people are willing to pay.
bunderbunderan hour ago
Framing it in terms of the price people might be willing to pay for a single product in isolation frames the point I was making, which was about price competition, right out of the picture.
Maybe I'd be willing to pay $10 for product A if I had other options. But if there's a product B for $3 that's not quite as nice but still ticks all my boxes, then product instantly becomes a lot less attractive.
kingleopold3 hours ago
This + your ROI in $10k device will be always lower than busy datacenter, its literally math. They sell free compute to others when you dont use it, you will never sell at that level or even you magically sell home compute, you will not compete at price
hypfer3 hours ago
I'm not sure if this prediction will hold true.
We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now.
Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.
gehsty2 hours ago
It’s a constant tension in computing that has been around since mainframes and clients… Neither is going to disappear. My general feeling is normal people care more about how thin and light something is than their privacy, so if data center powered LLMs will have a strong future.
hypfer2 hours ago
Hmm I'm not 100% sure about that, given that edge is very viable, and the geopolitical climate has changed quite significantly.
I agree that datacenters are not going to go away, but I have doubts that the buildup that has happened is really going to pay off for most operators.
airspresso3 hours ago
We are certainly not in the diminishing returns phase for LLM progress. No sign of that yet.
bunderbunder2 hours ago
I’ll grant that for specialized applications like coding agents and mathematics, but even there I suspect that most the real gains are actually taking place in the harness.
But I suspect returns may have already diminished into negative territory for at least some other use cases. One of my least favorite job responsibilities in this brave new era is figuring out how to avoid performance and behavior regressions when an older model were using for some application reaches end of life. It’s getting uncommon for me to look at our benchmark results and say, “Oh, good, it does better on one of the newer models!”
pixl972 hours ago
>suspect that most the real gains are actually taking place in the harness.
Part of the reason harnesses work well is you can run a lot of agents in parallel. That doesn't slow down demand.
hypfer2 hours ago
That is true, but the eventual realization that more machines doing more coin flips in parallel does not mean "more work gets done" might.
LLMs are amazing tech, but they're terrible without oversight. More agents faster just makes reality collapse on them quicker.
But yeah, you're right, temporarily, this will still push demand. But the topic was about "diminishing returns" as in "tech getting better". Not as in "customer spending".
pixl972 hours ago
It's kind of weird because more machines working together does mean more work gets done. Coin flips and weighted coin flips are totally different things. Any biases weights towards reality push you closer to reality when you use them.
New models keep being able to use more and more agents on longer time frames. Your hypothesis doesn't look like what we're measuring.
hypfer2 hours ago
Who is we?
pixl972 hours ago
The people mapping AI capabilities.
hypfer2 hours ago
Oh cool, so that we includes me! :)
pixl9732 minutes ago
Maybe turn on your light when you use a ruler? Not sure what else to say.
bunderbunder2 hours ago
I had actually been thinking more about all the non-LLM functionality that go into the harnesses. I'm not going to name names and I haven't done any rigorous testing, but my general impression is that choice of harness matters more than choice of model. In terms of basic task completion success specifically, not code aesthetics.
pixl972 hours ago
A perfect harness will not extract gold from a dumb model. It's a system that builds on each other, though we've not probed that frontier much to have a good intuition on what effects what.
3eb7988a16632 hours ago
One thing that I really want to know - the better models from today vs a year ago - what has changed. They have already pre-trained on all available public data. Scooping up the last percentage of archaic texts which were never digitized is not going to move the needle.
Is it just that the providers are generating tons of synthetic datasets on coding tasks so that the models get more exposure to the right thing to do? Every time someone points out an LLM stupidity they add some training data to patch over the weakness (trivial to generate "there are two 'l's in llama")?
hypfer3 hours ago
Well I mean if I wanted to be extra pedantic, I would argue that we've been in that phase since LLMs were first introduced.
Before that, we had 0. After that, we had more than 1.
A leap as far as that is hard to recreate.
But that wasn't my point. That's just trolling.
The actual point is that LLMs aren't gaining new capabilities anymore. They just get more reliable at the ones they already have; turning what was a coin flip to some higher probability.
That's (intuitively speaking, not strictly mathematically speaking) kinda the mathematical definition of diminishing returns.
484884483 hours ago
they really dont want to hear this bro lol
hypfer2 hours ago
I can see that by those reddit-style vote swings, but who are "they", exactly?
Who is so emotionally invested into random comment sections being purely positive about their pet.. uuuuuuuh.. tech?
Very weird.
EA-31673 hours ago
Or even more amusing in time for the bubble to burst, I hope the big RAM makers end up holding the whole bag for that. Greedy bastards.
glub20 minutes ago
Nearly the entire reason we're in this mess is because RAM/SSD/HDD makers are terrified of AI bubble bursting.
Long-term contracts, not building new fabs - they're in full on hedging mode right now.
bethekidyouwant3 hours ago
Since you’re not greedy when there is a memory glut I’m sure you’ll be willing to pay extra to make it fair.
EA-31673 hours ago
Not at all, I’m going to relax with a cooling drink and watch the consequences of this unbelievably destructive and wasteful venture implode. I also adore the idea that it's on the customer to be "fair" to a company that dumped us a group in favor of chasing B2B money.
Their choice, their consequences.
pixl972 hours ago
Internet is a fad, it will implode at any moment.
EA-31672 hours ago
In fact as a business opportunity it did just that because the massive investment in the mid-late 1990's was without anything like a connection to profitability or sustainable demand. It took years after that crash for the concept of e-commerce to begin both a recovery and evolution into what we see today.
I expect something similar for AI. It's useful tech... just not very profitable tech, and certainly not to the tune of trillions of dollars worth of public demand. The promises of superintelligence, replacing everyone, and the rest of the hype will die with the companies who made the promises, but the tech will survive and thrive.
pixl972 hours ago
>The promises of superintelligence, replacing everyone, and the rest of the hype will die with the companies who made the promises
The hype will die but there are many reasons why super intelligence and robots are a separate entity from said hype.
Look, back a few decades and tell someone our gdp would be in the trillions and it's likely they'd have a hard time believing you. A huge portion of our products that we use day to day would be complete science fiction to them.
All we're negotiating at this point is the timescale it will take.
card_zeroan hour ago
Unpredictable things happen routinely, therefore these specific unpredictable things will happen?
pixl9733 minutes ago
These things aren't really unpredictable. Work long enough on a robot with more dexterity and you will get just that. The timetable on when it happens is a bit more up in the air, economic downturns and wars can have huge impacts on it.
But in my mind robots and superintelligence are inevitable unless there is a massive setback in humanity before then. Nature already did it once. We're not inventing something totally new. Add in every new invention and bit of intelligence we build on and actualize pushes us that much closer to the goal. Information technology allows this to speed up even further with easy sharing of information and testing.
Superintelligence is not like faster than light travel. We have many frameworks that show us FTL is impossible. I don't believe there is a single widely accepted framework that shows there is some limit to intelligence and we're near it. Intelligence isn't even a singular thing. My calculator is a super intelligent adder compared to me. It would seem highly improbable that somehow nature random walked into making brains the most efficient general intelligence device in all dimensions and scales.
gs173 hours ago
Samsung is already building another fab (which was originally suspended due to low memory prices years ago). Hopefully when it goes live in a few years it's not all dedicated to HBM for AI.
heaney-5554 hours ago
Not even close.
nutjob23 hours ago
Not until the industry gets severe indigestion, which doesn't seem that far off.
blurbleblurble3 hours ago
The beginning of the bubble pop?
blovescoffeean hour ago
how would you even derive that conclusion from the headline/article?
esskay2 hours ago
its ram for the types of systems ai needs in datacenters so...no?
badgersnake3 hours ago
Now make some DDR5.
pixl972 hours ago
HBM is DDR5 in a different package.
badgersnake2 hours ago
If it doesn’t fit it’s no use to me.
psyphy23 hours ago
[flagged]
zahlman3 hours ago
Why wouldn't it get posted here?
esafak2 hours ago
Because you are not familiar with the site.
shevy-java4 hours ago
About 2 years ago I bought some SDRAM or something like that, DDR4 or DDR5, don't recall offhand. A few months ago I looked at the price today, and it was over 3x as high. That's just insane. Governments need to do something against this abuse system that AI amplified here.
digdigdag3 hours ago
> Governments need to do something against this abuse system that AI amplified here.
Which governments, and what exactly would you want these governments to do? The demand is global. There are no levers a single government can pull to meaningfully influence the global demand without fully committing to an protectionist economic policy, in which case the U.S. doesn't have the facilities to magically pop up world class fabs overnight, and South Korea and Taiwan don't have the market demand that the U.S. generates to justify their investments in making these chips and China lacks the IP to be able to build anything comparable to Nvidia's silicon at the moment.
No one has all the cards and no one controls all the levers.
carlm423 hours ago
Governments have sued DRAM companies for price fixing and forming cartels before: https://en.wikipedia.org/wiki/DRAM_industry_price_fixing. They can absolutely step in and if there are evidence of price fixing or anti-competitive behaviours, can impose fines. The choice of whether to do that though is political.
daedrdev3 hours ago
Several of these companies almost went under before the AI boom. There isn’t a cartel.
lovich2 hours ago
Is there any allegations of price fixing in this case? The demand just skyrocketed. Other than legislation like the defense act in the US I don’t know of any governments that have jurisdiction over these ram makers that mandate companies produce more of a product.
Especially not for consumer goods.
wmf2 hours ago
If they're charging one customer $X and another customer $5X for the same chips... there's got to be some law about this.
srdjanr37 minutes ago
I missed that this is happening, is it really? I thought it's 5X for everyone
wmf34 minutes ago
Supposedly OpenAI is paying 2025 prices.
CamperBob23 hours ago
No one has all the cards and no one controls all the levers.
I'm as fanatical a free-market fundamentalist as you'll ever meet, but if someone were to argue that government has a role in preventing bullshit like OpenAI's unilateral 40% attack on the entire DRAM market, backed by nothing but funny money, I would have a hard time coming up with defensible counterarguments.
bunderbunder3 hours ago
I don’t really know much about how this stuff works, but I have a feeling we wouldn’t even have to do anything punitive. We might just need to find and close whatever weird loophole allows the folks who are participating in this gold rush to feel confident enough that they’ve externalized their risks to be willing to engage in speculative data center buildout projects on such a grand scale in the first place.
rangestransform2 hours ago
Apple used to do the same thing with TSMC’s newer process nodes, and nobody really complained because people were getting superior products. I happen to like US antitrust enforcement that considers consumer benefit primarily.
pfdietz2 hours ago
Your attempts to insult what they are doing doesn't show they did anything wrong, but it does impeach the validity of your own argument.
CamperBob22 hours ago
Hit dog hollers, sounds like.
pfdietz2 hours ago
That's abuser logic.
CamperBob22 hours ago
The funny thing is, we're mostly in violent agreement, going by your other comments. This is a weird point to disagree on... so weird, in fact, that it's easy to believe you are associated with OpenAI. Hence my follow-up comment.
What Altman did, and the way he did it, was not OK. Unless you believe that humanity's adoption of AI-based intellectual work is best served by concentrating an enormous amount of power in one company at the expense of everyone else, turning the market into a literal zero-sum game. Do you?
newsclues3 hours ago
How much memory does it take for the consumer market not to be totally screwed?
Can that share of production be allocated to consumers, and the AI fights over the rest?
pfdietz3 hours ago
What abuse? Responding to supply and demand fluctuations isn't abuse, it's the proper operation of the market.
dumberquestions3 hours ago
Would if one industry gobbles up an important resource to the point that other consumers lose practical access to it, shouldn't there be limits?
jefftk3 hours ago
No. The other industries can bid for access to the resource, and under most circumstances, whether they are willing to bid higher tracks importance. Sometimes there are cases where something has significant positive externalities and so people are not willing to bid high enough, and then we can talk about some kind of government intervention (typically a subsidy that attempts to track the value of the externalities) but I don't see that applying here.
[deleted]3 hours agocollapsed
pfdietz3 hours ago
If others are willing to pay more, that a signal that more value is created if the resource goes to them rather than to the more price sensitive customers.
applfanboysbgon3 hours ago
This logic falls apart completely when gamblers allocate one trillion to bidding up the price, denying access to the resource to people who are responsibly spending their own money. The ideal world of the imaginary perfect free market never takes into consideration the messiness of the real world, like the fact that people will spend extreme sums of money irrationally. It can take years for this effect to correct, damaging the market severely in the meantime.
Alternatively, the money invested can be rational because it prices out competitors and establishes a monopoly, after which point the monopolist earns their absurd investment back with complete control of the market. This is also bad.
pfdietz3 hours ago
You can reach any conclusion you want if you assume others are behaving irrationally. But why should I assume they are wrong instead of you being wrong?
pixl972 hours ago
This has zero to do with the company that is making the product and all to do with the company buying them.
The problem you have here is hundreds of different companies are doing the "gambling" in a non collusionary manner so it's going to take decades in court to prove it. Are you saying governments should do an authoritarian take over of RAM allotment?
pfdietzan hour ago
And even if the companies are "gambling" (and what company doesn't do that) it doesn't mean the gambling is irrational.
joshheitzman3 hours ago
Where is the additional supply resulting from the sustained increase in demand?
ErneX3 hours ago
Some are building new fabs, example:
https://news.skhynix.com/en/fab-facility-investment-2026/
It's not an overnight thing obviously.
DonsDiscountGas2 hours ago
Check the article we're commenting under
yread2 hours ago
3x? I bought 128GB DDR5 SODIMMs for 300 eur. Now it's 4000 eur.
mrlonglongan hour ago
Damn you were lucky. I bought 96GB DDR5 last year for £600 UKP. They're now retailing for three times that.
Bastards.
jeroenhd3 hours ago
[dead]