BitNet b1.58 2B4T Technical Report

I built it at home this morning and tried it, perhaps my expectations were high but I wasn't terribly impressed. I asked it for a list of ten types of data I might show on a home info display panel. It gave me three. I clarified that I wanted ten, it gave me six. Every request after that just returned the same six things.

I know it's not chatGPT4 but I've tried other very small models that run on CPU only and had better results

> I've tried other very small models that run on CPU only and had better results

Maybe you can you share some comparative examples?

sure, here's my conversation with BitNet b1.58 2B4T

https://pastebin.com/ZZ1tADvp

here's the same prompt given to smollm2:135m

https://pastebin.com/SZCL5WkC

The quality of the second results are not fantastic. The data isn't public, and it repeats itself mentioning income a few times. I don't think I would use either of these models for accurate data but I was surprised at the truncated results from bitnet

Smollm2:360M returned better quality results, no repetition, but it did suggest things which didn't fit the brief exactly (public data given location only)

https://pastebin.com/PRFqnqVF

Edit:

I tried the same query on the live demo site and got much better results. Maybe something went wrong on my end?