←back to thread

Kimi Linear: An Expressive, Efficient Attention Architecture

(github.com)
210 points blackcat201 | 2 comments | 31 Oct 25 00:07 UTC | HN request time: 0.001s | source
Show context
logicartisan ◴[31 Oct 25 11:58 UTC] No.45771056[source]▶
Amazing how fast AI keeps improving, every new model feels like a big step forward
replies(1): >>45771240 #
hirako2000 ◴[31 Oct 25 12:21 UTC] No.45771240[source]▶
It solely is improving on efficiency. While it is extremely valuable given the disproportionate (to value) costs of these things, your statement almost sounds like it has improved an even more challenging aspect, pushing performance.
replies(4): >>45771821 #>>45771931 #>>45774235 #>>45774316 #
1. giancarlostoro ◴[31 Oct 25 13:46 UTC] No.45771931[source]▶
I've uh said this a few times. But AI is a bunch of people overpaying CS students to implement old algorithms and then realizing that they need Software Engineers to optimize the existing known systems. Most of AI (if not ALL of it) as we know it today has been coded for decades, we just never had the hardware for it.

A lot of the optimizations are not some ground breaking new way to program, they're known techniques to any Software Engineer or Systems Engineer.

replies(1): >>45771990 #
2. embedding-shape ◴[31 Oct 25 13:53 UTC] No.45771990[source]▶
> A lot of the optimizations are not some ground breaking new way to program

Hindsight is a bitch huh? Everything looks simple now once people proved it kind of works, but I think you over-simplify "how easy it is".

Lots of stuff in ML, particularly recent ~5 years or so, haven't been "implementing old algorithms" although of course everything is based on the research that happened in the past, we're standing on the shoulders of giants and all that.