←back to thread

Smollm3: Smol, multilingual, long-context reasoner LLM

(huggingface.co)
350 points kashifr | 1 comments | 08 Jul 25 16:13 UTC | HN request time: 0.236s | source
Show context
tiahura ◴[08 Jul 25 17:01 UTC] No.44501872[source]▶
Can anyone estimate how much of the 3B is necessitated by multi-language support?
replies(3): >>44502099 #>>44509476 #>>44509763 #
1. rockinghigh ◴[08 Jul 25 17:23 UTC] No.44502099[source]▶
The vocabulary size is fairly small (128,256) for a multilingual model. I would guess it doesn't require many additional parameters to support these 5 languages as many tokens can be shared.