r/Python • u/Veqq • 2d ago

Resource Nathan Goldbaum Interviewed about the Move to Free Threading and its Implementation

https://alexalejandre.com/interviews/interview-with-nathan-goldbaum/

After a lot of effort, the free threading build works but there is still much community work, making packages compatible etc.

44 Upvotes

92% Upvoted

24 comments sorted by

13

u/poppy_92 2d ago

Meanwhile Guido commented that "Personally I wish we’d stuck with the GIL. Everything was simpler."'

This was in relation to how ujson is defining its FT policy - https://github.com/ultrajson/ultrajson#multi-threadingfree-threading-support

11

u/snugar_i 1d ago

And Python 2 was simpler than Python 3. And Python 1 was probably simpler than that. No-GIL Python is basically Python 4 in everything but the name. And everyone's line where things start being "too complex" is different.

-8

u/Grouchy-Friend4235 1d ago

Ft was a mistake and will damage Python beyond recognition.

4

u/HommeMusical 1d ago

It's opt-in. The worst that will happen is that it doesn't get used.

2

u/Grouchy-Friend4235 1d ago

Right now it is. At some point nogil will be the default, and then most everybody will have to deal with its impact, not least on single threaded performance.

The GIL was added to Python because it made the interpreter free from concurrency issues, albeit at the tradeoff of being efficient for IO bound workloads only.

For CPU bound workloads we have multiprocessing, and since recently subinterpreters which all have their own GIL.

For performance critical code there are several tools such as Numba and Cython, both of which can disable the GIL for free threading.

There is really no need for nogil.

10

u/riksi 1d ago

I understand that many people want Python to remain a toy language, but it needs to evolve.

There are many people against async (I am one of them, I prefer gevent/green-threads as example), but it's still a better evolution than no async-at-all.

There are those against type hinting (I've met em!), I'm for full typescripting as example.

-1

u/Grouchy-Friend4235 1d ago

I suggest if you don't like Python's maturity, don't use it 😀There is plenty of other options like C++ or Rust, or whatever floats your boat.

Incidently we agree on async, greenlet, yet not on typing. I came to Python from a Java background and I never looked back, Python has been a liberation.

5

u/sudomatrix 1d ago

I was like you on typing, but now that some of my code is either written or updated by idiot savant AIs I welcome strict typing to minimize mistakes.

1

u/Grouchy-Friend4235 14h ago

yes I get that too, still I don't like it. I prefer docstrings with type info, it makes for cleaner code imho and IDEs like Pycharm can work with it just as well.

-7

u/Grouchy-Friend4235 2d ago

People with a conviction to change everything at a fundamental level just because they can and want to are the demise of Python.

9

u/HommeMusical 1d ago

The idea of simply brushing off almost a decade of careful, intelligent discussion of this feature as 'because they can!'

It's pretty clear you have no understanding of why free threading was needed, so here's the explanation.

By default, Python uses only one core because of the GIL. Python's threads are not native threads, so they are still stuck on one core. You need to use either multiprocessing or subprocess to use more than one core, but then you have to send all your data from one process to another, which is usually expensive and also somewhat messy.

Free threading is a solution to this problem. Everyone wanted some solution to this.

tl; dr: just because you're ignorant about this subject doesn't mean that people created free threading just for the lols.

4

u/wyldstallionesquire 1d ago

Python threads are real threads. They will just still follow the GIL, and only let Python interpret one set of bytecode at a time. So the result is that in most cases you get single thread performance, but it’s confusing to say they’re not real threads.

2

u/HommeMusical 1d ago

Yes, I wrote too fast, I was grumpy.

Free threading is cool. I like it.

-2

u/Grouchy-Friend4235 1d ago

It isn't, and if you'd care to research the subject properly you would know that shared state threaded programming is an anti pattern that has been discouraged for decades. Python has plenty of other options if you need to utilize multiple cores concurrently, and none of them require nogil/freethreading. In fact you can even share memory between them if you relly need to, however the practice is discouraged in general, not specific to Python.

0

u/wyldstallionesquire 1d ago

This is wildly not true?

0

u/Grouchy-Friend4235 1d ago

Well yes, everything I wrote is true as a matter of fact.

3

u/wyldstallionesquire 1d ago

Threads are used pretty much all the time in a bunch of languages.

1

u/Grouchy-Friend4235 1d ago edited 1d ago

Threads yes, shared state between threads no. The SOTA is shared nothing parallism and lock-free algorithms. Nogil is built on the premise of enabling shared memory using fine grained locks, i.e. the opposite.

1

u/snugar_i 17h ago

Shared mutable state is not good, shared read-only data, on the other hand, is very nice. You don't have to copy the data to each process or jump through memory-sharing hoops (that don't work with Python's refcounting anyway).

And no-GIL means "no global interpreter lock", it literally removes a lock, and yet you say that it is wrong, but at the same time locks are wrong?

→ More replies (0)