Most active commenters
  • igouy(3)
  • zahlman(3)

←back to thread

259 points rbanffy | 21 comments | | HN request time: 1.101s | source | bottom
Show context
AlexanderDhoore ◴[] No.44003888[source]
Am I the only one who sort of fears the day when Python loses the GIL? I don't think Python developers know what they’re asking for. I don't really trust complex multithreaded code in any language. Python, with its dynamic nature, I trust least of all.
replies(19): >>44003924 #>>44003936 #>>44003940 #>>44003943 #>>44003945 #>>44003958 #>>44003971 #>>44004203 #>>44004251 #>>44004431 #>>44004501 #>>44005012 #>>44005100 #>>44005259 #>>44005773 #>>44006165 #>>44007388 #>>44011009 #>>44011917 #
jillesvangurp ◴[] No.44004251[source]
You are not the only one who is afraid of changes and a bit change resistant. I think the issue here is that the reasons for this fear are not very rational. And also the interest of the wider community is to deal with technical debt. And the GIL is pure technical debt. Defensible 30 years ago, a bit awkward 20 years ago, and downright annoying and embarrassing now that world + dog does all their AI data processing with python at scale for the last 10. It had to go in the interest of future proofing the platform.

What changes for you? Nothing unless you start using threads. You probably weren't using threads anyway because there is little to no point in python to using them. Most python code bases completely ignore the threading module and instead use non blocking IO, async, or similar things. The GIL thing only kicks in if you actually use threads.

If you don't use threads, removing the GIL changes nothing. There's no code that will break. All those C libraries that aren't thread safe are still single threaded, etc. Only if you now start using threads do you need to pay attention.

There's some threaded python code of course that people may have written in python somewhat naively in the hope that it would make things faster that is constantly hitting the GIL and is effectively single threaded. That code now might run a little faster. And probably with more bugs because naive threaded code tends to have those.

But a simple solution to address your fears: simply don't use threads. You'll be fine.

Or learn how to use threads. Because now you finally can and it isn't that hard if you have the right abstractions. I'm sure those will follow in future releases. Structured concurrency is probably high on the agenda of some people in the community.

replies(4): >>44004471 #>>44004545 #>>44005797 #>>44005830 #
HDThoreaun ◴[] No.44004545[source]
> But a simple solution to address your fears: simply don't use threads. You'll be fine.

Im not worried about new code. Im worried about stuff written 15 years ago by a monkey who had no idea how threads work and just read something on stack overflow that said to use threading. This code will likely break when run post-GIL. I suspect there is actually quite a bit of it.

replies(5): >>44004632 #>>44004665 #>>44004939 #>>44008198 #>>44010469 #
1. bayindirh ◴[] No.44004665[source]
Software rots, software tools evolve. When Intel released performance primitives libraries which required recompilation to analyze multi-threaded libraries, we were amazed. Now, these tools are built into processors as performance counters and we have way more advanced tools to analyze how systems behave.

Older code will break, but they break all the time. A language changes how something behaves in a new revision, suddenly 20 year old bedrock tools are getting massively patched to accommodate both new and old behavior.

Is it painful, ugly, unpleasant? Yes, yes and yes. However change is inevitable, because some of the behavior was rooted in inability to do some things with current technology, and as hurdles are cleared, we change how things work.

My father's friend told me that length of a variable's name used to affect compile/link times. Now we can test whether we have memory leaks in Rust. That thing was impossible 15 years ago due to performance of the processors.

replies(4): >>44005661 #>>44005802 #>>44007054 #>>44010622 #
2. delusional ◴[] No.44005661[source]
> Software rots

No it does not. I hate that analogy so much because it leads to such bad behavior. Software is a digital artifact that can does not degrade. With the right attitude, you'd be able to execute the same binary on new machines for as long as you desired. That is not true of organic matter that actually rots.

The only reason we need to change software is that we trade that off against something else. Instructions are reworked, because chasing the universal Turing machine takes a few sacrifices. If all software has to run on the same hardware, those two artifacts have to have a dialogue about what they need from each other.

If we didnt want the universal machine to do anything new. If we had a valuable product. We could just keep making the machine that executes that product. It never rots.

replies(6): >>44005751 #>>44005771 #>>44005775 #>>44006313 #>>44006656 #>>44010640 #
3. kstrauser ◴[] No.44005751[source]
That’s not what the phrase implies. If you have a C program from 1982, you can still compile it on a 1982 operating system and toolchain and it’ll work just as before.

But if you tried to compile it on today’s libc, making today’s syscalls… good luck with that.

Software “rots” in the sense that it has to be updated to run on today’s systems. They’re a moving target. You can still run HyperCard on an emulator, but good luck running it unmodded on a Mac you buy today.

replies(1): >>44010649 #
4. dahcryn ◴[] No.44005771[source]
yes it does.

If software is implicitly built on wrong understanding, or undefined behaviour, I consider it rotting when it starts to fall apart as those undefined behaviours get defined. We do not need to sacrifice a stable future because of a few 15 year old programs. Let the people who care about the value that those programs bring, manage the update cycle and fix it.

5. eblume ◴[] No.44005775[source]
Software is written with a context, and the context degrades. It must be renewed. It rots, sorry.
replies(1): >>44006626 #
6. cestith ◴[] No.44005802[source]
My only concern is this kind of change in semantics for existing syntax is more worthy of a major revision than a point release.
replies(2): >>44009055 #>>44012014 #
7. indymike ◴[] No.44006313[source]
>> Software rots > No it does not.

I'm thankful that it does, or I would have been out of work long ago. It's not that the files change (literal rot), it is that hardware, OSes, libraries, and everything else changes. I'm also thankful that we have not stopped innovating on all of the things the software I write depends on. You know, another thing changes - what we are using the software for. The accounting software I wrote in the late 80s... would produce financial reports that were what was expected then, but would not meet modern GAAP requirements.

8. igouy ◴[] No.44006626{3}[source]
You said it's the context that rots.
replies(1): >>44006736 #
9. rocqua ◴[] No.44006656[source]
Fair point, but there is an interesting question posed.

Software doesn't rot, it remains constant. But the context around it changes, which means it loses usefulness slowly as time passes.

What is the name for this? You could say 'software becomes anachronistic'. But is there a good verb for that? It certainly seems like something that a lot more than just software experiences. Plenty of real world things that have been perfectly preserved are now much less useful because the context changed. Consider an Oxen-yoke, typewriters, horse-drawn carriages, envelopes, phone switchboards, etc.

It really feels like this concept should have a verb.

replies(1): >>44008334 #
10. bayindirh ◴[] No.44006736{4}[source]
It's a matter of perspective, I guess...

When you look from the program's perspective, the context changes and becomes unrecognizable, IOW, it rots.

When you look from the context's perspective, the program changes by not evolving and keeping up with the context, IOW, it rots.

Maybe we anthropomorphize both and say "they grow apart". :)

replies(1): >>44008270 #
11. spookie ◴[] No.44007054[source]
The other day I compiled a 1989 C program and it did the job.

I wish more things were like that. Tired of building things on shaky grounds.

replies(2): >>44009051 #>>44012370 #
12. igouy ◴[] No.44008270{5}[source]
We say the context has breaking changes.

We say the context is not backwards compatible.

13. igouy ◴[] No.44008334{3}[source]
obsolescence
14. rbanffy ◴[] No.44009051[source]
If you go into mainframes, you'll compile code that was written 50 years ago without issue. In fact, you'll run code that was compiled 50 years ago and all that'll happen is that it'll finish much sooner than it did on the old 360 it originally ran on.
15. rbanffy ◴[] No.44009055[source]
It's opt-in at the moment. It won't be the default behavior for a couple releases.

Maybe we'll get Python 4 with no GIL.

/me ducks

16. zahlman ◴[] No.44010622[source]
> A language changes how something behaves in a new revision, suddenly 20 year old bedrock tools are getting massively patched to accommodate both new and old behavior.

In my estimation, the only "20 year old bedrock tools" in Python are in the standard library - which currently holds itself free to deprecate entire modules in any minor version, and remove them two minor versions later - note that this is a pseudo-calver created by a coincidentally annual release cadence. (A bunch of stuff that old was taken out recently, but it can't really be considered "bedrock" - see https://peps.python.org/pep-0594/).

Unless you include NumPy's predecessors when dating it (https://en.wikipedia.org/wiki/NumPy#History). And the latest versions of NumPy don't even support Python 3.9 which is still not EOL.

Requests turns 15 next February (https://pypi.org/project/requests/#history).

Pip isn't 20 years old yet (https://pypi.org/project/pip/#history) even counting the version 0.1 "pyinstall" prototype (not shown).

Setuptools (which generally supports only the Python versions supported by CPython, hasn't supported Python 2.x since version 45 and is currently on version 80) only appears to go back to 2006, although I can't find release dates for versions before what's on PyPI (their own changelog goes back to 0.3a1, but without dates).

17. zahlman ◴[] No.44010640[source]
>execute the same binary

Only if you statically compile or don't upgrade your dependencies. Or don't allow your dependencies to innovate.

18. zahlman ◴[] No.44010649{3}[source]
> You can still run HyperCard on an emulator, but good luck running it unmodded on a Mac you buy today.

I grew up with HyperCard, so I had a moment of sadness here.

replies(1): >>44011007 #
19. kstrauser ◴[] No.44011007{4}[source]
We all have our own personal HyperCard.
20. necovek ◴[] No.44012014[source]
Python already has a history of "misrepresenting" the ycope of the change (like changing behaviour of one of core data types and calling it just a major version change — that's really a new language IMHO).

Still, that's only a marketing move, technically the choice was still the right one, just like this one is.

21. Too ◴[] No.44012370[source]
Hello world without -O2 -Werror? I've done several compiler toolchain updates on larger code bases and every new version of Clang, GCC or glibc will trip up new compiler warnings. Worse, occasionally taking advantage of some UB laying around leading to runtime bugs.

I'm not complaining, the new stricter warnings are usually for the better, what I'm saying is that the bedrock of that world isn't as stable as it's sometimes portraited.