←back to thread

320 points willm | 1 comments | | HN request time: 0s | source
Show context
atomicnumber3 ◴[] No.45106455[source]
The author gets close to what I think the root problem is, but doesn't call it out.

The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc).

Meanwhile, go showed us what good green threads can look like. Then java did it too. Meanwhile, js had better async support the whole time. But all it did was show us that async code just plain sucks compared to green thread code that can just block, instead of having to do the async dances.

So, why engage with it when you already had good solutions?

replies(24): >>45106558 #>>45106616 #>>45106659 #>>45106663 #>>45106684 #>>45106758 #>>45107048 #>>45107148 #>>45107247 #>>45107394 #>>45107701 #>>45107865 #>>45108486 #>>45108978 #>>45109142 #>>45109610 #>>45109958 #>>45110033 #>>45110052 #>>45110805 #>>45111877 #>>45111901 #>>45113010 #>>45113188 #
pbalau ◴[] No.45107394[source]
You make a very good case for why python's async isn't more prevalent, but I think this is not painting the full image.

Taking a general case, let's say a forum, in order to render a thread one needs to search for all posts from that thread, then get all the extra data needed for rendering and finally send the rendered output to the client.

In the "regular" way of doing this, one will compose a query, that will filter things out, join all the required data bla bla, send it to the database, wait for the answer from the database and all the data to be transferred over, loop over the results and do some rendering and send the thing over to the client.

It doesn't matter how async your app code is, in this way of doing things, the bottle neck is the database, as there is a fixed limit on how many things a db server can do at once and if doing one of these things takes a long time, you still end up waiting too much.

In order for async to work, one needs to split the work load into very small chunks that can be done in parallel and very fast, therefore, sending a big query and waiting for all the result data is out of the window.

An async approach would split the db query into a search query, that returns a list of object ids, say posts, then create N number of async tasks that given a post id will return a rendered result. These tasks will do their own query to retrieve the post data, then assemble another list of async tasks to get all the other data required and render each chunk and so on. Throw in a bunch of db replicas and you get the benefits of async.

This approach is not generally used, because, let's face it, we like making the systems we use do complicated things, eg complicated sql requests.

replies(2): >>45111723 #>>45113872 #
1. LtWorf ◴[] No.45111723[source]
Why do you think that all of that extra compute work would be better?