Ollama enables deploying LLMs models locally on laptops and edge severs, Cactus enables deploying on phones. Deploying directly on phones facilitates building AI apps and agents capable of phone use without breaking privacy, supports real-time inference with no latency, we have seen personalised RAG pipelines for users and more.
Apple and Google actively went into local AI models recently with the launch of Apple Foundation Frameworks and Google AI Edge respectively. However, both are platform-specific and only support specific models from the company. To this end, Cactus:
- Is available in Flutter, React-Native & Kotlin Multi-platform for cross-platform developers, since most apps are built with these today.
- Supports any GGUF model you can find on Huggingface; Qwen, Gemma, Llama, DeepSeek, Phi, Mistral, SmolLM, SmolVLM, InternVLM, Jan Nano etc.
- Accommodates from FP32 to as low as 2-bit quantized models, for better efficiency and less device strain.
- Have MCP tool-calls to make them performant, truly helpful (set reminder, gallery search, reply messages) and more.
- Fallback to big cloud models for complex, constrained or large-context tasks, ensuring robustness and high availability.
It's completely open source. Would love to have more people try it out and tell us how to make it great!
The performance is quite good, even on CPU.
However I'm now trying it on a pixel, and it's not using GPU if I enable it.
I do like this idea as I've been running models in termux until now.
Is the plan to make this app something similar to lmstudio for phones?
Some Android models won't support GPU hardware; we'll be addressing that as we move to our own kernels.
The app itself is just a demonstration of Cactus performance. The underlying framework gives you the tools to build any local mobile AI experience you'd like.