YC is wrong about LLMs for chip design

You ask 100,000 humans each to make a photo realistic rendering of a alpaca playing basketball on the moon in 90 seconds, an LLM is going to outperform every single one of them.

Diffusion models aren't actually LLMs, they're a different architecture. Which makes it even weirder we invented them at the same time.

Also, they might not be able to do it. eg most models can't generate "horse riding an astronaut" or "upside-down car".

That's not a meaningful benchmark for valuing art or creating art

To be fair, most humans can't draw any better than stick figures.

This is true, but humans are much better at including specified elements in an image with specified spatial relationships. A description like a "A porpoise seated at a desk writing a letter" will reliably produce (terrible) drawings consisting of parts corresponding to the porpoise, parts corresponding to the desk, and parts corresponding to the letter, with the arrangement of the parts roughly corresponding to the description.

What meaningful benchmark would you use? Art by it's nature is subjectively experienced - what one person considers great, meaningful, soul-moving art, another may consider terrible, meaningless, and empty. Both opinions are equally valid.

But if you're using AI to create art, you're typically not trying to move someone's soul. You're trying to create a work that depicts something in a particular style with a particular fidelity with a certain amount of resource consumption. That is the only metric by which it makes any sense to evaluate the machine designed to do that specific task.

Humans being better at one specific aspect of a task is not equivalent to humans being overall better at the task.

I just entered your prompt into an AI image generator and in under a second it gave me an image[0] of what looks to me like an anthropomorphic dolphin sitting at a desk writing a letter in a little study. I then had to google what the difference between a porpoise and a dolphin was because I genuinely thought porpoises looked much more like manatees. While I could nitpick the AI's work for making the porpoise's snout a little too long, had I drawn it the porpoise would have been a vaguely marine looking blob with no anatomy detailed enough to recognize let alone criticize. I am quite confident that if you asked for a large number of images based on that prompt from humans, it would easily rank among the best, and it's unlikely you'd get any which were markedly better. The fact it can generate this image nearly instantaneously though is astounding. If your goal was to get one masterpiece hanging in the Louvre, this particular tool would not suffice, but if your goal was to illustrate children's books, this tool could do in hours what would have taken a team of humans months. That is superhuman performance.

[0] https://api.deepai.org/job-view-file/e0b80ca6-d934-42e4-9a7e...

(Sorry if the link doesn't remain good for long)

An AI image generator will sometimes do a good job on this sort of prompt, but it fails in different ways to the ways that humans fail.

Whether humans or AI are better at the task overall is probably too vague a question to answer, depending a lot on how you weight different desirables.