←back to thread

1-Bit AI Infrastructure

(arxiv.org)
157 points galeos | 1 comments | 15 Nov 24 14:28 UTC | HN request time: 0s | source
Show context
ttyprintk ◴[15 Nov 24 14:32 UTC] No.42147278[source]▶
Later a4.8 quantization by some of the same team:

https://news.ycombinator.com/item?id=42092724

https://arxiv.org/abs/2411.04965

replies(1): >>42191454 #
skavi ◴[20 Nov 24 07:12 UTC] No.42191454[source]▶
and the repo for this project: https://github.com/microsoft/BitNet
replies(1): >>42192368 #
sinuhe69 ◴[20 Nov 24 10:02 UTC] No.42192368[source]▶
The demo they showed was full of repeated sentences. The 3B model looks quite dense, TBH. Did they just want to show the speed?
replies(1): >>42193481 #
1. newswasboring ◴[20 Nov 24 12:50 UTC] No.42193481[source]▶
3B models, especially in quantized state, almost always behave like this.