←back to thread

Django: what’s new in 6.0

(adamj.eu)
378 points rbanffy | 4 comments | 09 Dec 25 20:33 UTC | HN request time: 0.003s | source
Show context
teagee ◴[09 Dec 25 21:37 UTC] No.46211038[source]▶
Any code or blog written by Adam is worth spending some time on.

It will be interesting to see how the tasks framework develops and expands. I am sad to see the great Django-Q2 lumped in with the awful Celery though.

replies(9): >>46211810 #>>46212076 #>>46212142 #>>46212219 #>>46212861 #>>46215681 #>>46215766 #>>46216187 #>>46217127 #
hintoftime ◴[09 Dec 25 23:19 UTC] No.46212076[source]▶
Why is celery awful?
replies(4): >>46212860 #>>46213125 #>>46213237 #>>46213833 #
akoumjian ◴[10 Dec 25 01:50 UTC] No.46213125[source]▶
Celery is great and awful at the same time. In particular, because it is many Python folks' first introduction to distributed task processing and all the things that can go wrong with it. Not to mention, debugging can be a nightmare. Some examples:

- your function arguments aren't serializable - your side effects (e.g. database writes) aren't idempotent - discovering what backpressure is and that you need it - losing queued tasks during deployment / non-compatible code changes

There's also some stuff particular to celery's runtime model that makes it incredibly prone to memory leaks and other fun stuff.

Honestly, it's a great education.

replies(2): >>46213509 #>>46213842 #
ffsm8 ◴[10 Dec 25 03:47 UTC] No.46213842[source]▶
> your side effects (e.g. database writes) aren't idempotent

What does idempotent mean in this context, or did you mean atomic/rollback on error?

I'm confused because how could a database write be idempotent in Django? Maybe if it introduced a version on each entity and used that for crdt on writes? But that'd be a significant performance impact, as it couldn't just be a single write anymore, instead they'd have to do it via multiple round trips

replies(3): >>46215820 #>>46216405 #>>46216758 #
1. jon-wood ◴[10 Dec 25 11:05 UTC] No.46216405[source]▶
In the context of background jobs idempotent means that if your job gets run for a second time (and it will get run for a second time at some point, they all do at-least-once delivery) there aren't any unfortunate side effects to that. Often that's just a case of checking if the relevant database updates have already been done, maybe not firing a push notification in cases of a repeated job.
replies(1): >>46216796 #
2. 7bit ◴[10 Dec 25 12:06 UTC] No.46216796[source]▶
If you need idempotent db writes, then use something like Temporal. You can't really blame Celery for not having that because that is not what Celery aims to be.
replies(2): >>46217005 #>>46217201 #
3. hintoftime ◴[10 Dec 25 12:34 UTC] No.46217005[source]▶
Temporal requires a lot more setup than setting up a Redis instance though. That's the only problem with it. And I find the Python API a bit more difficult to grasp. But otherwise a solid piece of technology.
4. SkyArrow ◴[10 Dec 25 13:00 UTC] No.46217201[source]▶
With Temporal, your activity logic still needs to ensure idempotency e.g. by checking if an event id / idempotency key exists in a table. It's still at-least-once delivery. Temporal does make it easy to mint an idempotency key by concatenating workflow run id and activity id, if you don't have a one provided client-side.