←back to thread

Zamba2-7B

(www.zyphra.com)
282 points dataminer | 1 comments | 14 Oct 24 22:45 UTC | HN request time: 0.294s | source
Show context
arnaudsm ◴[15 Oct 24 01:03 UTC] No.41843888[source]▶
I'm tired of LLM releases that cherry pick benchmarks. How does it compare to SOTA qwen2.5/phi3.5 ?

Anyone knows an up to date independent leaderboard? Lmsys and livebench used to be great but skipped most major models recently.

replies(2): >>41844092 #>>41846615 #
1. metalwhale ◴[15 Oct 24 01:40 UTC] No.41844092[source]▶
I think it cannot surpass SOTA in some LM evaluation sets, but please understand that achieving better results requires a very good training dataset, which not everyone can afford.

On the other hand, the main points of Zamba/Mamba are low latency, generation speed, and efficient memory usage. If this is true, LLMs could be much easier for everyone to use. All we need to do is wait for someone with a good training dataset to train a SOTA Mamba.