←back to thread

Microsoft BitNet: inference framework for 1-bit LLMs

(github.com)
167 points galeos | 1 comments | 18 Oct 24 09:10 UTC | HN request time: 0.203s | source
Show context
alkh ◴[18 Oct 24 15:56 UTC] No.41880605[source]▶
Sorry for a stupid question but to clarify, even though it is a 1-bit model, it is supposed to be working with any types of embeddings, even taken from larger LLMs(in their example, they use HF1BitLLM/Llama3-8B-1.58-100B-tokens). I.e. it doesn't have an embedding layer built-in and relies on embedding provided separately?
replies(1): >>41881253 #
1. danielmarkbruce ◴[18 Oct 24 16:57 UTC] No.41881253[source]▶
No. You can't put any type of embedding in.