(twitter.com)

255 points tbruckner | 2 comments | 07 Sep 23 14:36 UTC | HN request time: 0.421s | source

1. tiffanyh ◴[07 Sep 23 16:11 UTC] No.37421142[source]▶

  system_info: n_threads = 4 / 24

Am I seeing correctly in the video that this ran on only 4 threads?

replies(1): >>37422416 #

2. wmf ◴[07 Sep 23 17:25 UTC] No.37422416[source]▶

It's using the GPU so I guess not that many CPU threads are needed to feed the GPU.

Running a 180B parameter LLM on a single Apple M2 Ultra