←back to thread

Show HN: Cactus – Ollama for Smartphones

(github.com)
217 points HenryNdubuaku | 1 comments | 10 Jul 25 19:20 UTC | HN request time: 0.207s | source

Hey HN, Henry and Roman here - we've been building a cross-platform framework for deploying LLMs, VLMs, Embedding Models and TTS models locally on smartphones.

Ollama enables deploying LLMs models locally on laptops and edge severs, Cactus enables deploying on phones. Deploying directly on phones facilitates building AI apps and agents capable of phone use without breaking privacy, supports real-time inference with no latency, we have seen personalised RAG pipelines for users and more.

Apple and Google actively went into local AI models recently with the launch of Apple Foundation Frameworks and Google AI Edge respectively. However, both are platform-specific and only support specific models from the company. To this end, Cactus:

- Is available in Flutter, React-Native & Kotlin Multi-platform for cross-platform developers, since most apps are built with these today.

- Supports any GGUF model you can find on Huggingface; Qwen, Gemma, Llama, DeepSeek, Phi, Mistral, SmolLM, SmolVLM, InternVLM, Jan Nano etc.

- Accommodates from FP32 to as low as 2-bit quantized models, for better efficiency and less device strain.

- Have MCP tool-calls to make them performant, truly helpful (set reminder, gallery search, reply messages) and more.

- Fallback to big cloud models for complex, constrained or large-context tasks, ensuring robustness and high availability.

It's completely open source. Would love to have more people try it out and tell us how to make it great!

Repo: https://github.com/cactus-compute/cactus

1. pxc ◴[11 Jul 25 18:50 UTC] No.44535678[source]▶
This is a cool app! I'm happy to play with it. Some feedback:

1. The lack of a dark mode is an accessibility issue for me. I have a genetic condition that causes severe light sensitivity and special difficulty with contrast. The only way for me to achieve sufficient contrast without uncomfortable and blinding brightness is dark mode, so at present I can only use your app by disabling dark mode and inverting colors across my phone. This is of course not ideal because it ruins photos in other apps, and I end up with some unavoidable very bright white "hotspots" on my phone that I don't normally have when I can just use dark mode. Relatedly, the contrast for some of the text in the app is low to the point of being practically unreadable for me (getting enough contrast with it similarly requires cranking up the brightness). :(

2. I tried downloading a few other models, namely Jan Nano and SmolLM3, using the GGUF link download functionality, but every time I select them, the app just immediately crashes.

I understand that the chat app on the Play Store is basically just a demo for the framework, and if I were really using it I would be in charge of my own theming and downloading the required models and so on, but these still seem worth fixing to me.