←back to thread

Microsoft BitNet: inference framework for 1-bit LLMs

(github.com)
167 points galeos | 2 comments | 18 Oct 24 09:10 UTC | HN request time: 0.399s | source
1. lostmsu ◴[18 Oct 24 14:45 UTC] No.41879915[source]▶
No GPU inference support?
replies(1): >>41879986 #
2. diggan ◴[18 Oct 24 14:52 UTC] No.41879986[source]▶
> that support fast and lossless inference of 1.58-bit models on CPU (with NPU and GPU support coming next).