←back to thread

Zamba2-7B

(www.zyphra.com)
282 points dataminer | 1 comments | 14 Oct 24 22:45 UTC | HN request time: 0.206s | source
Show context
iamronaldo ◴[14 Oct 24 23:05 UTC] No.41843139[source]▶
Not transformer based?
replies(3): >>41843175 #>>41843177 #>>41843268 #
1. epistasis ◴[14 Oct 24 23:10 UTC] No.41843177[source]▶
Tri Gao and Albert Gu say "Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality"

https://arxiv.org/abs/2405.21060

Mamba-2 is used in Zamab2.