←back to thread

Zamba2-7B

(www.zyphra.com)
282 points dataminer | 3 comments | 14 Oct 24 22:45 UTC | HN request time: 0.421s | source
1. PoignardAzur ◴[15 Oct 24 09:14 UTC] No.41846612[source]▶
For the amount of theoretical work behind those Mamba2 blocks (I can barely understand their paper on the subject), those are some extremely modest performance gains.

Attention remains king.

replies(2): >>41848095 #>>41849186 #
2. erichocean ◴[15 Oct 24 12:57 UTC] No.41848095[source]▶
Mamba is also much more efficient, watt-wise, to run.
3. visarga ◴[15 Oct 24 14:47 UTC] No.41849186[source]▶
> I can barely understand their paper on the subject

Yannic Kilcher has a new video touching on Mamba in an intuitive way.

https://www.youtube.com/watch?v=jE9jAZC42NE