←back to thread

Zamba2-7B

(www.zyphra.com)
282 points dataminer | 1 comments | 14 Oct 24 22:45 UTC | HN request time: 0s | source
Show context
jwitthuhn ◴[15 Oct 24 01:20 UTC] No.41843985[source]▶
For anyone else looking for the weights which as far as I can tell are not linked in the article:

Base model: https://huggingface.co/Zyphra/Zamba2-7B

Instruct tuned: https://huggingface.co/Zyphra/Zamba2-7B-Instruct

replies(1): >>41844057 #
keyle ◴[15 Oct 24 01:33 UTC] No.41844057[source]▶
I couldn't find any gguf files yet. Looking forward to trying it out when they're available.
replies(3): >>41844163 #>>41844635 #>>41847638 #
alchemist1e9 ◴[15 Oct 24 01:55 UTC] No.41844163[source]▶
What can be used to run it? I had imagined Mamba based models need a different interference code/software than the other models.
replies(3): >>41844520 #>>41844782 #>>41847696 #
1. gbickford ◴[15 Oct 24 03:53 UTC] No.41844782[source]▶
If you look in the `config.json`[1] it shows `Zamba2ForCausalLM`. You can use a version of the transformers library to do inference that supports that.

The model card states that you have to use their fork of transformers.[2]

1. https://huggingface.co/Zyphra/Zamba2-7B-Instruct/blob/main/c...

2. https://huggingface.co/Zyphra/Zamba2-7B-Instruct#prerequisit...