Which small model is good for fine tuning to various enterprise data sets? Our business units are wanting to run small models in browser and on mobile devices, without dealing with RAG and cloud resources.
replies(5):
Also tried SmolVLM 256M and 500M, they load faster and you can embed them in assets, they work if you know what you're doing
Just keep in mind that smaller models don't perform as well due to their limited parameters
Also on Android, since you can't ship files larger than 2GB due to Java compression issues, you need to download models separately, then you can't load the model from the download folder, you have to copy it into the app's own folder, this means a Gemma 3N 2B model that's 3.14 GB would need at least 7 GB of free space on the user's phone