←back to thread

AI engineers claim new algorithm reduces AI power consumption by 95%

(www.tomshardware.com)
352 points ferriswil | 1 comments | 19 Oct 24 18:03 UTC | HN request time: 0.205s | source
Show context
kayo_20211030 ◴[19 Oct 24 19:34 UTC] No.41890110[source]▶
Extraordinary claims require extraordinary evidence. Maybe it's possible, but consider that some really smart people, in many different groups, have been working diligently in this space for quite a while; so claims of 95% savings on energy costs _with equivalent performance_ is in the extraordinary category. Of course, we'll see when the tide goes out.
replies(6): >>41890280 #>>41890322 #>>41890352 #>>41890379 #>>41890428 #>>41890702 #
1. throwawaymaths ◴[19 Oct 24 20:00 UTC] No.41890322[source]▶
I don't think this claim is extraordinary. Nothing proposed is mathematically impossible or even unlikely, just a pain in the ass to test (lots of retraining, fine tuning etc, and those operations are expensive when you dont have already massively parallel hardware available, otherwise you're ASIC/FPGAing for something with a huge investment risk)

If I could have a SWAG at it I would say a low resolution model like llama-2 would probably be just fine (llama-2 quantizes without too much headache) but a higher resolution model like llama-3 probably not so much, not without massive retraining anyways.