←back to thread

The first year of free-threaded Python

(labs.quansight.org)
256 points rbanffy | 1 comments | 16 May 25 09:42 UTC | HN request time: 0s | source
Show context
AlexanderDhoore ◴[16 May 25 11:03 UTC] No.44003888[source]▶
Am I the only one who sort of fears the day when Python loses the GIL? I don't think Python developers know what they’re asking for. I don't really trust complex multithreaded code in any language. Python, with its dynamic nature, I trust least of all.
replies(19): >>44003924 #>>44003936 #>>44003940 #>>44003943 #>>44003945 #>>44003958 #>>44003971 #>>44004203 #>>44004251 #>>44004431 #>>44004501 #>>44005012 #>>44005100 #>>44005259 #>>44005773 #>>44006165 #>>44007388 #>>44011009 #>>44011917 #
miohtama ◴[16 May 25 11:13 UTC] No.44003943[source]▶
GIL or no-GIL concerns only people who want to run multicore workloads. If you are not already spending time threading or multiprocessing your code there is practically no change. Most race condition issues which you need to think are there regardless of GIL.
replies(3): >>44004241 #>>44005583 #>>44011886 #
immibis ◴[16 May 25 11:49 UTC] No.44004241[source]▶
With the GIL, multithreaded Python gives concurrent I/O without worrying about data structure concurrency (unless you do I/O in the middle of it) - it's a lot like async in this way - data structure manipulation is atomic between "await" expressions (except in the "await" is implicit and you might have written one without realizing in which case you have a bug). Meanwhile you still get to use threads to handle several concurrent I/O operations. I bet a lot of Python code is written this way and will start randomly crashing if the data manipulation becomes non-atomic.
replies(3): >>44004284 #>>44005054 #>>44005728 #
1. OskarS ◴[16 May 25 14:08 UTC] No.44005728[source]▶
That doesn't match with my understanding of free-threaded Python. The GIL is being replaced with fine-grained locking on the objects themselves, so sharing data-structures between threads is still going to work just fine. If you're talking about concurrency issues like this causing out-of-bounds errors:

    if len(my_list) > 5:
        print(my_list[5])
(i.e. because a different thread can pop from the list in-between the check and the print), that could just as easily happen today. The GIL makes sure that only one python interpreter runs at once, but it's entirely possible that the GIL is released and switches to a different thread after the check but before the print, so there's no extra thread-safety issue in free-threaded mode.

The problems (as I understand it, happy to be corrected), are mostly two-fold: performance and ecosystem. Using fine-grained locking is potentially much less efficient than using the GIL in the single-threaded case (you have to take and release many more locks, and reference count updates have to be atomic), and many, many C extensions are written under the assumption that the GIL exists.