Django 6.0 beta 1 released

Looks like Django 6 is getting an offical task system.

There’s no real world brokers or workers supported (at least built in), but still centralising around a standard interface might make things nicer than the celery tentacle monsters Django apps eventually tend to mutate into.

I don´t understand how you could use this new tasks system in production if there is no real workers when it's released? Are there any 3rd party yet?

I guess the idea is first to provide a generic interface to connect various backends to and let the community develop those. Users of Django should then be able to swap one out for another. Maybe one will emerge as a quasi-standard or maybe it will be like database backends where different backends serve different purposes.

At leas that's my guess.

My understanding is that if you just need to return a response to the client as quickly as possible, but are ok with then processing your task directly after that, then it's still usable today.

But if you want to schedule a task further in the future, then a new backend will be needed for that.

I think the bigger use case is being able to (backoff) retry failing API calls to 3rd party services. AFAIUI the new tasks package doesnt offer this in v1 which is a deal breaker for my project, at least.

For me I think it works well as is because my use case is sending several different emails after POST'ing to a view, which, there is no need to make the user wait for in my case, as they don't care about the status of the mail delivery.

But I realize there are many other usecases too that will need proper workers.

Right, I'd never touch celery, but RQ is simple and has never let me down.

I'm curious, why is that? I've heard that sentence before but usually from people who want to write their own task system and end up partially implementing what celery implements just worse.

Celery is large and complex now and edge cases always show up at scale but that is usually not a reflection of the platform quality. The custom implementations I've seen are no where near what celery is capable of and can cater to so haven't seen the edge cases yet but that doesn't mean they implemented bug-free code and celery hasn't.

After asking about it, the issue always went towards a hand-wavey "performance". What is your experience on that front?

My experience is that Celery is fragile. It loses the connection with RabbitMQ sometimes and it needs a restart to recover. I never had those problems with Rails and Sidekiq + Redis.

I'll choose the small, simple, reliable tool over the larger framework every time, if I can get away with having the smaller feature set. I'd go out of my way to make the simpler tool work if I could, to keep the overall cognitive load and complexity of the project down.

I never considered writing my own, I just want a tool to plug in and work so I can focus on the application. I also never thought performance of this part of the stack would be a bottleneck for my use cases.

In contrast, I am happy to use Django over Flask because you can't get away from needing lots of web framework features for a nontrivial web app. The bigger framework is usually justified, especially if it's high quality like Django. But spawning tasks (for my use cases) is just an aspect of the project that can have a simple interface and it doesn't justify adopting a big framework. Time I've invested in Django (going back to 2009) is still paying off for me today, but there'd be much lower ROI on the time I'd have to put into Celery. Unless my projects really needed its complexity, but I think far fewer projects actually require that in practice than their architects tend to think.