Is AI back end complexity a pain point? Seeking feedback on an open source idea

Hi all,

I am a founder working on AI applications, and I have noticed that building the backend for AI apps feels much more complex and fragmented than for traditional SaaS. Things like usage-based billing, managing credits, LLM streaming (with session resuming), user behavior analytics, and integrations with multiple model providers all add a lot of overhead before you can even focus on the product itself.

I am thinking of starting an open source project called AiBase (https://github.com/liurenju/AiBase) to handle these backend pain points out of the box, so teams can focus on building their core AI features instead of wrestling with infrastructure.

For those building or planning to build AI products, do you feel these are major pain points? Would you use an open source Backend as a Service for this, or do you prefer rolling your own solution? What would you want to see in such a project for it to actually be useful? Would love to hear your experiences and honest opinions, including “this is not a real problem,” “I would never use BaaS for AI,” or any similar feedback.

> Things like usage-based billing, managing credits, LLM streaming (with session resuming), user behavior analytics, and integrations with multiple model providers all add a lot of overhead before you can even focus on the product itself.

That doesn't make sense to me. The product needs should dictate whether or not you need those things. You shouldn't be boiler plating them in without a product-driven reason. So if you found the need and are responding to that need... you are focusing on the product.

Thanks for your comments. We distill these features as our priority because our current product SellToSky needs them and we spent significant amount of time building it. We'll not re-invent the wheel but rather glue the available open source project + any missing parts. The whole purpose is to abstract the backend needs as a service -> reducing the repetitive works from the fellow AI developers/founders.

That's why I am coming here to see whether any other developers or founders experienced the same pain?