←back to thread

Apple Research unearthed forgotten AI technique and using it to generate images

(9to5mac.com)
129 points celias | 3 comments | 23 Jun 25 18:19 UTC | HN request time: 0.747s | source
Show context
rfv6723 ◴[27 Jun 25 03:07 UTC] No.44393382[source]▶
Apple AI team keeps going against the bitter lesson and focusing on small on-device models.

Let's see how this would turn out in longterm.

replies(5): >>44393454 #>>44393509 #>>44393622 #>>44394586 #>>44394727 #
1. janalsncm ◴[27 Jun 25 07:38 UTC] No.44394586[source]▶
The bitter-er lesson is that distillation from bigger models works pretty damn well. It’s great news for the GPU poor, not great for the guys training the models we distill from.
replies(1): >>44401947 #
2. rfv6723 ◴[28 Jun 25 02:26 UTC] No.44401947[source]▶
Distillation is great for researchers and hobbyists.

But nearly all frontier models have anti-distillation ToS, so distillation is out of question for western commercial companies like Apple.

replies(1): >>44402386 #
3. janalsncm ◴[28 Jun 25 04:55 UTC] No.44402386[source]▶
Even if Apple needs to train an LLM from scratch, they can distill it and deploy on edge devices. From that point, inference is free to them.