←back to thread

Microsoft BitNet: inference framework for 1-bit LLMs

(github.com)
166 points galeos | 1 comments | 18 Oct 24 09:10 UTC | HN request time: 0.203s | source
Show context
lostmsu ◴[18 Oct 24 14:45 UTC] No.41879915[source]▶
No GPU inference support?
replies(1): >>41879986 #
1. diggan ◴[18 Oct 24 14:52 UTC] No.41879986[source]▶
> that support fast and lossless inference of 1.58-bit models on CPU (with NPU and GPU support coming next).