(github.com)

167 points galeos | 1 comments | 18 Oct 24 09:10 UTC | HN request time: 0.203s | source

Show context

wwwtyro ◴[18 Oct 24 15:03 UTC] No.41880073[source]▶

Can anyone help me understand how this works without special bitnet precision-specific hardware? Is special hardware unnecessary? Maybe it just doesn't reach the full bitnet potential without it? Or maybe it does, with some fancy tricks? Thanks!

replies(3): >>41880204 #>>41880283 #>>41881707 #

1. eightysixfour ◴[18 Oct 24 15:24 UTC] No.41880283[source]▶

>>41880073 #

While fancy hardware would make it faster, what you are comparing it to is a bunch of floating point and large number multiplication. I believe in this case they just use a look up table:

If one value is 0, it is 0.

If the signs are different, it is -1.

If the signs are the same, it is 1.

I’m sure those can be done with relatively few instructions using far less power hungry hardware.

↑

Microsoft BitNet: inference framework for 1-bit LLMs