Hacker News

maxall4
How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip spectrum.ieee.org

pama8 hours ago

Having worked with people doing bringup of specialized chips, I am awed at how the world has changed.

> When the first chips came back from the foundry in May, the team pointed its internal AI models at designing software to run benchmarks such as SemiAnalysis’s InferenceX. On DeepSeek’s multi-head latent attention kernel benchmark, performance climbed from 0.31 percent of the theoretical ceiling (set by the chip’s compute and memory bandwidth) to 88.94 percent in roughly 40 hours. Ho says this result is repeatable, so the time between when foundries deliver the first chips and when production ramps up can be reduced. “All our schedule assumptions are going to be based on the fact we have this capability now,” he says.

wmf7 hours ago

Back in the day you'd write the code before the chip came back but I guess today it's faster to wait.

threatripper5 hours ago

The longer you wait the faster you will go.

Mistletoe5 hours ago

Like space travel.

conmod27843 minutes ago

The successive generations of spaceships won't built themselves. Who will be responsible for setting up real world and software feedback loop?

marcelo-earth3 hours ago

> “All our schedule assumptions are going to be based on the fact we have this capability now”

is the world we live in, planning things while waiting for a more powerful LLM

cindyllm3 hours ago

[dead]

program_whiz8 hours ago

With a few handy tips and tricks from apple insiders. But sure, I guess the LLMs helped too.

m00xan hour ago

The lawsuit isn't for chip designers, but the consumer product lines. It's possible they also got IP for chips, but that was not brought up.

m4rtink3 hours ago

Yeah, how could those people even think of switching their owners!

stingraycharles3 hours ago

Didn’t they not just switch employers but actually handed over a lot of proprietary documents from Apple?

[deleted]7 hours agocollapsed

stogot7 hours ago

This is the part forgotten. Apple is claiming this is their IP embedded on chips that OPenAI stakes the future on. Will they settle?

wmf7 hours ago

The lawsuit appears to be about consumer devices, not NPUs or ASICs. If you think Jalapeno stole from anyone it would be Google.

[deleted]7 hours agocollapsed

mathisfun1237 hours ago

You people are so weird - gossiping on a site that literally reported the news as it broke smh. It's like sitting in the back of class gossiping about the popular kids.

Newsflash that lawsuit is about product designs not accelerator ASICs. And there wouldn't be anything to steal because Apple doesn't have any DC class accelerators.

camillomiller4 hours ago

[flagged]

muchdoubt8 hours ago

Seems pretty obvious now that OpenAI is just hyping their models in order to get companies (in this case, chip developers) to use their products in order to learn from their (exfiltrated) IP. Any corporation would be foolish to use any of their or Microsoft’s products, particularly those with valuable IP. There’s nothing in the article that says AI did anything creative but rather that it was used for software development within the overall project. Clear misleading title. Suggest to mark this as clickbait.

jimmySixDOF4 hours ago

IEEE Spectrum is such a good publication. Early in my career I worked at a place where the magazine would be passed around every month with a coversheet listing all us engineers we had to pass it around and sign we had read it. Been a while since I visited the website but love what they did with it.

Kwpolska2 hours ago

Every time their content appears here, it's a very shallow analysis written for a barely technical audience. And this article is no different, it's just "slop machine wrote verilog; all the hard bits were done by Broadcom, who have access to public AI models (we didn't talk to them and don't know if they used them, but ClosedAI wants us to think they did)"

karim799 hours ago

I grow Jalapeños. This conflation of AI and actual chili peppers irks me.

amelius9 hours ago

Guess how electrical engineers feel about the term "transformers".

frangonf8 hours ago

As a former EE, attention was all I needed to not get zapped.

cyberax8 hours ago

:groan:

georgemcbay4 hours ago

> Guess how electrical engineers feel about the term "transformers".

There is more to this story than meets the eye.

karim799 hours ago

This is an excellent comment. I'm still laughing.

Lalabadie9 hours ago

I do generative art (no relation to AI prompting). I feel your frustration.

fragmede8 hours ago

Cryptographers also got the same raw deal with cryptocurrency, and every one just said "crypto?"

monkpit7 hours ago

Or cyber…

karim796 hours ago

Like traditional generative art? Like worms WMD map generation or something? Cool!

TomGarden9 hours ago

Oh my!

DrewADesign7 hours ago

Same

asveikau9 hours ago

Just think of how the people of Xalapa, Mexico feel. They should send them a royalty check.

damowangcy6 hours ago

I thought I was in Reddit for a moment.

seanmcdirmid8 hours ago

Jalapeño also used to be a Java VM written in Java at IBM.

Duanemclemore6 hours ago

I'm a licensed architect. Welcome to our hell of the last 40 years.

glitchc9 hours ago

Feeling the burn?

smitty1e8 hours ago

To say nothing of the Red Hot Chili Peppers.

Razengan9 hours ago

> irks me

It's jalapeño grill would you say?

karim799 hours ago

Not sure what you're talking about. But I'll tell you, home grown Jalapeño peppers, fermented with 3% salt is the stuff of dreams.

wiml9 hours ago

"It's all up in yo' grill, would you say?"

Razengan9 hours ago

You know what really grinds my gears? Friction.

chrismarlow98 hours ago

slow claps

karim793 hours ago

Thank you.

honeycrispy9 hours ago

I'm annoyed that the meaning of the word "Agent" has been obliterated.

Like, why couldn't they invent a new word and not hijack an existing word?

imtringued41 minutes ago

https://en.wikipedia.org/wiki/Agent

Computing

* Agent architecture, a blueprint for software agents and control systems

* Agent-based model, a computational model for simulating the actions and interactions of individuals

* Agentic AI, autonomous artificial intelligence that can make decisions and act on those decisions on its own

* Forté Agent, an email and Usenet news client

* Intelligent agent, an autonomous, goal-directed entity which observes and acts upon an environment

* Software agent, a piece of software that acts for a user or other program

* User agent, software that is acting on behalf of a user

karim798 hours ago

Call it GPTChippomatic or something. Please leave my peppers alone.

Razengan8 hours ago

Did you not watch the Matrix documentary?

MadrasTh0rn8 hours ago

I guess I'll have to

delusional9 minutes ago

We were able to invent a chip that already existed so fast, you guys.

AI does not make anything new, it is not surprising that it can regurgitate what already exists much faster than humans can invent new things.

xpct8 hours ago

Aw, I was expecting more details but this just seems to be a rehash of what they unveiled a month ago.

amelius9 hours ago

At some point people will use an LLM to design an Apple M series competitor.

Lramseyer9 hours ago

Production grade CPU design is more than just the RTL (the source code.) To achieve the performance numbers that these companies get, you have to do a ton of optimization in your physical design to achieve the power/performance/area (PPA) metrics that make these products competitive. LLMs are not suitable for that kind of work.

There are people working on PPA optimization and trying to shake up how things are done, just not with LLMs.

menaerus2 hours ago

> LLMs are not suitable for that kind of work.

I wonder why not or you meant not suitable yet?

Systemerror7A69an hour ago

This is just speculation on my part, but LLMs work best when they get immediate, verifiable feedback on their task, and the kind of physical optimizations they mean might not give that to LLMs.

btown8 hours ago

Something that I think is fascinating, though, is that labs are no longer beholden to the limitations of commercial design software. Want to replace your simulator and optimizer with a fully custom verifiable stack of Lean proofs of optimality and correctness? Just throw your unlimited token budget at it.

xpct8 hours ago

I don't work in the business, but my understanding was that even with these companies' budgets, it's still too expensive to do any kind of verified performance optimality.

thfuran7 hours ago

And I think correctness for anything near the size of a CPU is off the table.

xpct7 hours ago

Isn't that weird? The full knowledge of how to make such chips may one day be accessible to anyone, yet only the entrenched companies will remain the makers.

If we imagine machines being able to do the full process end-to-end, and the quality of that process only dependent on capital spent on tokens, I don't see how new companies could ever enter the market.

bhouston9 hours ago

It is probably doable right not to push a risc-v design into that performance space.

bigyabai9 hours ago

They won't, because they'd need an ARM architecture license.

amelius9 hours ago

Why, the LLM can make up its own architecture.

The value lies in the design space exploration, which is what an LLM can easily do.

https://en.wikipedia.org/wiki/Design_space_exploration

nr3788 hours ago

Qualcomm have an architecture license and the Snapdragon X2 Elite Extreme X2E-96-100 isn't too far off the M5 Pro.

[1] https://browser.geekbench.com/processors/snapdragon-x2-elite...

[2] https://browser.geekbench.com/macs/macbook-pro-14-inch-2026-...

pazimzadeh5 hours ago

watt about in performance per watt?

bigyabai7 hours ago

Are they using LLMs to close that gap, or is this their Nuvia acquisition doing the heavy lifting?

wmf9 hours ago

Arm sells architecture licenses to anybody these days.

cmrdporcupine8 hours ago

Or they'll just build a competitor in RISC-V instead and that's fine.

Except the problem is not restricted to the actual ISA or its HDL implementation, etc.

It's even just getting space / time in a fab at that advanced of a process node.

pixl979 hours ago

I mean you can design anything without a license. Selling it is where the problems come up. Even then there are likely places in China that would still make it for you.

nullc7 hours ago

And be super-bankrupted by patent litigation from Apple. I don't think they're worried.

After all, they successfully threatened Adobe with spurious patent litigation unless they joined w/ apple in illegally fixing wages.

You don't think a criminal like apple would absolutely decimate any competition given the opportunity? They didn't hold back when it was a unambiguous crime, they surely wouldn't if it was merely bad for the world.

gozucito8 hours ago

It is surprising to me that recursive self-improvement seems more plausible now than it did in 2023. Am I the only one to be surprised?

I remember the paper proving that hallucinations could never be fully solved back in 2024: https://arxiv.org/abs/2409.05746

I also remember the hang-wringing about running out of new datasets to train on. Now it appears humans are always generating more data. It's just not as cheap to acquire as legacy data? Meta has to give a deep discount on their API prices to entice people.

I thought back then that humans had a few more breakthroughs in them as meaningful as the seminal Attention is all you need paper. Enough to 100x the capabilities of LLMs back then (10x the smarts and 10x the speed simultaneously).

RSI with a 20 month turnaround for a chip to be made is not exactly breakneck speed though. Physical manufacturing and logistical constraints are going to be and remain a hard obstacle to that process for the foreseeable future.

red75prime4 hours ago

> I remember the paper proving that hallucinations could never be fully solved back in 2024

The papers that use the halting problem or the Gödel's incompleteness theorem to prove something about LLMs are dime a dozen. The problem is they prove their results for any computable system. You need to also believe that the human brain contains "magic" to think that humans are exempt.

I believe I've said the same at the time this paper was published. There is no need for hindsight to notice the problem.

The required amount of compute and training data and whether the existing training methods were up to the task had the real potential to be show stoppers though.

chrisjj14 minutes ago

> Am I the only one to be surprised?

Did you think RSI cured "hallucination"?

globnomulous4 hours ago

> Jalapeño can reduce end-to-end latency (the time between prompt to last token) by up to 3.6 times

I'm never sure what on earth this kind of impressionistic math is supposed to tell me. Is the comparison between 4.6 and 1.0? 3.6 and 1.0? Clearly the comparison isn't supposed to be 1.0 and -2.6, even though that's what the words literally mean. I can't be the only person who finds this infuriating and distracting. These numbers shouldn't be impressionistic. They should be precise. That this is an article on spectrum.ieee.org makes the imprecision all the stranger. I'd expect their readershipt to care, for instance, about what's even being measured. Is this the geometric mean of something? The arithmetic mean? And what latency has improved?

caidanan hour ago

That odor you are detecting is just good old fashioned bullshit, my friend. It’s just that nowadays everything and everyone is covered in it, and we are not supposed to notice. The emperor has no clothes… and is covered in shit.

perching_aixan hour ago

It's... written right there? Like what?

Suppose you send in your marvelous prompt and hit Enter.

Machine churns for 18 seconds, types out a "reply", then yields back control.

18 / 3.6 = 5

So now the machine will only churn for 5 seconds before yielding back control.

This is confusing how exactly?

Why would an "up to" figure be a mean, or a geometric mean? It's clearly a max, that's why it's called "up to"...

Am I missing something?

ramshanker8 hours ago

So when can we start getting cheap chips? RAM anyone please!

faitswulff7 hours ago

Everyone's still bottlenecked on foundries, not designs.

jeffybefffy5197 hours ago

Cant AI build foundries?

altcognito7 hours ago

Something we can all agree with is we need more foundries and green power.

senectus17 hours ago

yup, but it'll take about 3-5 years.

ThrowawayTestr3 hours ago

AI can barely fold a shirt

geraneum8 hours ago

Whatever happened with the Apple lawsuit?

[deleted]8 hours agocollapsed

google2341238 hours ago

Congrats to the former TPU team

mathisfun1237 hours ago

I was surprised to see they were using XLS but then I remembered Chris went there a couple of years ago.

cute_boi9 hours ago

openai should figure out how to make lithography machine, so ASML don't have monopoly on it.

TomGarden8 hours ago

The Chinese have been working on EUV for a while

bigyabai9 hours ago

"Reverse engineer this DARPA project, make no mistakes"

hn-front (c) 2024 voximity
source