←back to thread

YC is wrong about LLMs for chip design

(www.zach.be)
321 points laserduck | 5 comments | 16 Nov 24 14:07 UTC | HN request time: 0s | source
Show context
aubanel ◴[16 Nov 24 19:06 UTC] No.42158417[source]▶
I know nothing about chip design. But saying "Applying AI to field X won't work, because X is complex, and LLMs currently have subhuman performance at this" always sounds dubious.

VCs are not investing in the current LLM-based systems to improve X, they're investing in a future where LLM based systems will be 100x more performant.

Writing is complex, LLMs once had subhuman performance, and yet. Digital art. Music (see suno.AI) There is a pattern here.

replies(7): >>42158545 #>>42158550 #>>42158576 #>>42159935 #>>42160061 #>>42165587 #>>42169569 #
zachbee ◴[16 Nov 24 19:24 UTC] No.42158545[source]▶
I didn't get into this in the article, but one of the major challenges with achieving superhuman performance on Verilog is the lack of high-quality training data. Most professional-quality Verilog is closed source, so LLMs are generally much worse at writing Verilog than, say, Python. And even still, LLMs are pretty bad at Python!
replies(3): >>42158764 #>>42159143 #>>42160983 #
1. jjk166 ◴[17 Nov 24 00:48 UTC] No.42160983[source]▶
I would imagine it is a reasonably straightforward thing to create a simulator that generates arbitrary chip designs and the corresponding verilog that can be used as training data. It would be much like how AlphaFold was trained. The chip designs don't need to be good, or even useful, they just need to be valid so the LLM can learn the underlying relationships.
replies(2): >>42163071 #>>42175981 #
2. astrange ◴[17 Nov 24 09:35 UTC] No.42163071[source]▶
I know just enough about chips to be suspicious of "valid". The right solution for a chip at the HDL layer depends on your fab, the process you're targeting, what % of physical space on the chip you want it to take up, and how much you're willing to put into power optimization.
replies(1): >>42185913 #
3. adrian_b ◴[18 Nov 24 19:29 UTC] No.42175981[source]▶
I have never heard of any company, no matter how big and experienced, where it is possible to decide that an ASIC design is valid by any other means except by paying for a set of masks to be made and for some prototypes to be manufactured, then tested in the lab.

This validation costs millions, which is why it is hard to enter this field, even as a fabless designer.

Many design errors are not caught even during hardware testing, but only after mass production, like the ugly MONITOR/MWAIT bug of Intel Lunar Lake.

Randomly-generated HDL code, even if it does not have syntax errors, and even if some testbench for it does not identify deviations from its specification, is not more likely to be valid when implemented in hardware, than the proverbial output of a typewriting monkey.

replies(1): >>42185871 #
4. jjk166 ◴[19 Nov 24 17:20 UTC] No.42185871[source]▶
Validating an arbitrary design is hard. It's equivalent to the halting problem. Working backwards using specific rules that guarantee validity is much easier. Again, the point is not to produce useful designs. The generated model doesn't need to be perfect, indeed it can't be, it just needs to be able to avoid the same issues that humans are looking for.
5. jjk166 ◴[19 Nov 24 17:23 UTC] No.42185913[source]▶
The goal is not to produce the right, or even a good solution. The point is to create a large library of highly variable solutions so the trained model can pick up on underlying patterns. You want it to spit out lots of crap.