AI capex is so big that it's affecting economic statistics

I'm waiting for the shoe to drop when someone comes out with an FPGA optimized for reconfigurable computing and lowers the cost of llm compute by 90% or better.

Where is that improvement coming from? Hardware is already here to compute gemm as fast as possible.

We're already seeing it with DeepSeek's and other optimizations - like that law with highways - the wider the highway the even more usage of it. Dropping by 90% would open even more use cases.

For white-collar jobs replacement - we can always evolve up the knowledge/skills/value chain. It is the blue-collar jobs where bloodbath with all the robotics is coming.

Raw gemm computation was never the real bottleneck, especially on the newer GPUs. Feeding the matmuls i.e memory bandwidth is where it’s at, especially in the newer GPUs.

This is where I do wish we had more people working on the theoretical CS side of things in this space.

Once you recognize that all ML techniques, including LLMs, are fundamentally compression techniques you should be able to come up with some estimates of the minimum feasible size of an LLM based on: information that can be encoded in a given parameter size, relationship between loss of information and model performance, and information contained in the original data set.

I simultaneously believe LLMs are bigger than the need to be, but suspect they need to be larger than most people think given that you are trying to store a fantastically large amount of information. Even given lossy compression (which ironically is what makes LLMs "generalize"), we're still talking about an enormous corpus of data we're trying to represent.

Getting theoretical results along these lines that can be operationalized meaningfully is... really hard.

We do already have ASICs, see Google's TPU to get some cost estimates.

HBM is also very expensive.

I wouldn't wait. fpgas weren't design to serve this model architecture. yes they are very power efficient but the layout/p+r overhead, the memory requirement (very few on-the-market fpgas have hbm), slower clock speed, and just an unpleasant developer experience makes it a hard sell.

My chip friends constantly complain about Qualcomm owning a bunch of FPGA related patents that stifle any meaningful FPGA progress.

They'll expire eventually.

> For white-collar jobs replacement - we can always evolve up the knowledge/skills/value chain.

I'm not so sure about this one. I partially agree with the statement, but less-abled collegues might have troubles with this :( Ultimately there will be less stuff to do for a plain human being.