(github.com)

1311 points msoad | 1 comments | 31 Mar 23 20:37 UTC | HN request time: 0.206s | source

Show context

Dwedit ◴[31 Mar 23 22:47 UTC] No.35394801[source]▶

> 6GB of RAM

> Someone mentioning "32-bit systems"

Um no, you're not mapping 6GB on RAM on a 32-bit system. The address space simply doesn't exist.

1. jiggawatts ◴[31 Mar 23 22:53 UTC] No.35394865[source]▶

Windows Server could use up to 64 GB for a 32-bit operating system. Individual processes couldn't map more than 4 GB, but the total could be larger: https://en.wikipedia.org/wiki/Physical_Address_Extension

Llama.cpp 30B runs with only 6GB of RAM now