←back to thread

Microsoft BitNet: inference framework for 1-bit LLMs

(github.com)
167 points galeos | 1 comments | 18 Oct 24 09:10 UTC | HN request time: 0.223s | source
1. Scene_Cast2 ◴[18 Oct 24 16:26 UTC] No.41880929[source]▶
Neat. Would anyone know where the SDPA kernel equivalent is? I poked around the repo, but only saw some form of quantization code with vectorized intrinsics.