Does this mean that we can also run the 60B model on a 16GB ram computer now?
I have the M2 air and can't wait until further optimisation with the Neural Engine / multicore gpu + shared ram etc.
I find it absolutely mind boggling that GPT-3.5(4?) level quality may be within reach locally on my $1500 laptop / $800 m2 mini.
replies(1):