←back to thread

Zamba2-7B

(www.zyphra.com)
282 points dataminer | 1 comments | 14 Oct 24 22:45 UTC | HN request time: 0.205s | source
Show context
PoignardAzur ◴[15 Oct 24 09:14 UTC] No.41846612[source]▶
For the amount of theoretical work behind those Mamba2 blocks (I can barely understand their paper on the subject), those are some extremely modest performance gains.

Attention remains king.

replies(2): >>41848095 #>>41849186 #
1. erichocean ◴[15 Oct 24 12:57 UTC] No.41848095[source]▶
Mamba is also much more efficient, watt-wise, to run.