Show HN: oLLM – LLM Inference for large-context tasks on consumer GPUs

20 minutes is a huge turnoff, unless you have it run over night.... Just to get the hint that you should exercise self care in the morning when presenting a legal paper and have the ai check it for flaws.

We are talking about 100k context here. 20k would be much faster, but you won't need KVCache offloading for it