Jerod is joined by the co-hosts of core.py , Pablo Galindo & Łukasz Langa, a podcast about Python internals by people who work on Python internals. Python 3.13 is right around the corner, which means the Global Interpeter Lock (GIL) is now experimentally optional! This is a huge deal as Python is finally free-threaded. There's more to discuss, of course, so we get into all the gory details.
What up, Python nerds? I'm Jared, and you are listening to The Change Log, where each and every week we sit down with the hackers, the leaders, and the innovators of the software world to pick their brain, to learn from their mistakes, to get inspired by their accomplishments, and to have a lot of fun along the way.
On this episode, I'm joined by the co-hosts of the Core.py podcast, Pablo Galindo and Lukas Langa, whose name I will pronounce Lukas from here on out because it's just a lot easier for me. On Core.py, they talk about Python internals because they work on Python internals. And today we're talking about Python 3.13, which is right around the corner.
When we recorded this conversation last week, it was slated to be released on October 1st, but now they are targeting October 7th. So if you're listening to this in the future, 3.13 is fully baked. But if you are listening right after we hit publish, wait a week or grab the release candidate, which is 99% baked. Why are we all so excited about Python 3.13?
Well, the global interpreter lock, aka the GIL, is now experimentally optional. This is a huge deal, as Python is finally free-threaded and able to run with true parallelism. There's more, of course, and we get into all the details. I think you'll enjoy it, even if, like me, you aren't a regular Pythonista. But first, a mention of our partners at Fly.io.
Over 3 million apps have launched on Fly, including ours. And you can too, in less than five minutes, learn how at Fly.io. Okay, free threaded Python on the changelog. Let's do this.
Hey friends, I'm here with Dave Rosenthal, CTO of Sentry. So Dave, I know lots of developers know about Sentry, know about the platform, because hey, we use Sentry and we love Sentry. And I know tracing is one of the next big frontiers for Sentry. Why add tracing to the platform? Why tracing and why now?
When we first launched the ability to collect tracing data, we were really emphasizing the performance aspect of that, the kind of application performance monitoring aspect, you know, because you have these things that are spans that measure how long something takes.
And so the natural thing is to try to graph their durations and think about their durations and, you know, warn somebody if the durations are getting too long. But what we've realized is that the performance stuff ends up being just a bunch of gauges to look at. And it's not super actionable, right?
Sentry is all about this notion of debug ability and actually making it easier to fix the problem, not just sort of giving you more gauges. A lot of what we're trying to do now is focus a little bit less on the sort of just the performance monitoring side of things and turn tracing into a tool that actually aids the debug ability of problems.
I love it. Okay, so they mean it when they say code breaks. Fix it faster with Sentry. More than 100,000 growing teams use Sentry to find problems fast, and you can too. Learn more at Sentry.io. That's S-E-N-T-R-Y.io. And use our code CHANGELOG. Get $100 off the team plan. That's almost four months free for you to try out Sentry. Once again, Sentry.io.
Today I'm joined by Pablo Galindo and Lucas Longa from Core.py, a podcast all about Python internals because these two work on Python internals. Welcome to the show guys.
Thank you very much for having us. Happy to be here.
Now your podcast, is it usually just the two of you talking to each other or do you have a third party ever?
Every now and again, we would invite a guest, but mostly it's just the two of us.
Yeah, when people are fed up of listening to us, and then we reach someone to...
Now, we've been asked to start a Python podcast a few times, as well as an Elixir podcast and a Rust podcast. And we don't usually feel like we have anything to add to the conversation, except when it came to Python, I remember telling our friend Brett Cannon, I said, you guys should have a podcast where it's like the people who work on Python talking about Python, not...
randos like I would be or people on the, you know, it's a huge community, but there's no voice that is like coming from the core team. And so I think it's pretty cool. When did you guys start this and what was the big idea?
So we started actually one year ago. We were in the Right now we are in the co-developer sprint at Meta.
So this is basically a meeting that we do every single year when we try to bring all the co-developers together and, you know, the sparks and conversation in person because, you know, open source is basically based on asynchronous communication, but from time to time it's very good to, you know, not having to discuss and wait like an entire day just to tell the other person, I don't like your proposal.
So after that, these meetings have a lot of discussions, and they are super rich regarding that. It's fantastic. It's one of my favorite moments of the year. And so I reach to Gukesh, and I say, it would be actually very cool, because what happens here, people don't really know the discussions. They just know the output of them.
like there is many of these projects that you know take years or like they only show after a time and there are some of these projects that just die and they never happen and I was telling Gukes like it would be super cool if we have this space when we can talk about like you know how the sausages make right and all these crazy ideas that we're having and how they kind of transform into it because like there is this I mean to some extent it's improving but there is this perception from people that core developers are these alien developers that you know like they go into this room and then
Python happens. But it's just like, we are just regular people, right? Exactly. So our idea was first kind of demystify the code development thing. So it's not just, you know, we're just regular people, right? Just doing the thing. And the other is that trying to inspire people to join this, right?
So ideally we can talk about like not only the price that we're doing, but also like how high it is or how we struggle. And And we try to always have some inspirational notes on the episode so people can say, hey, maybe this is something that I want to do. And then, and this was a good idea.
We also try to highlight contributions that people have done because like at the end of the day, you know, people only know the big things, you know, okay. new parser or the JIT or the garbage collector or whatever it is. But there is a lot of very cool contributions for core developers or regular people that are contributing to CPython.
So we spend at the end of the episode some time trying to go through the last, we call this section, what's going on in CPython. And we try to highlight the different things that people are doing so people know about them. And hopefully if a name comes again and again, people say, I know that person.
That's cool. So Lucas, Pablo speaks very fast. How do you keep up with him?
Experience. It's just a matter of just doing this for long enough. We've known each other for a few years before that. So I was prepared for the Spanish 2x, you know, speaking speed. Yeah, built in 2x. Right.
Like we were joking, you know, about this like many times because people who are listening to our podcast, which for the most part is Python enthusiasts, but really the Python core team itself, you know, so they're saying, yes, like this is the one podcast that they cannot listen to us at 2x because Paolo is speaking quickly. I am not speaking so fast.
So maybe, you know, with this development of AI and everything, we should have some adaptive playback speed where it recognizes who is talking. And for me, it would be 2x. For Pablo, it would be like, whoa, whoa, whoa, no.
I don't think this 1x is enough.
Yeah, some sort of normalization where they just both go to the same speed. That would be a cool feature.
We, as an anecdote, a small anecdote, we at PyCon, this is this big conference that happens every year. Well, there's several other ones, but the big one is in the US. So I normally just normally give like a talk every year. So especially in PyCon US, there is these people that try to transcribe live. So like people with hearing problems can read. So it's really, really cool.
And this year, before giving the talk, I apologized to the person. It's like, I'm really sorry about what is going to happen, but I'm not being able to speak slower. And she was like, yeah, no problem. But like, oof. She really struggled with that.
But yeah, she kept up. She was really good. Yeah, the closed captioning at PyCon is a magical thing. It turns out that it does help people with hearing disabilities to participate in the conference. But it's also helpful even for other participants, right? So just as subtitles are being turned on by most people watching now like TV shows and other programs at home,
the closed captions happen to be helpful for regular attendees as well. So this is something that has become a staple of PyCon US. I'm very happy that this is there. Definitely some people present more of a challenge to the persons actually doing the closed captioning.
I'm always amazed not just at how fast they go, but how much they can express the different TLAs and Python-specific terms that we're using. You would think, okay, this is very industry-specific, and if you talk to your dad or to some of your friends that are not in tech... some of the words, verbs, and so on that we're using might be mystifying. You don't even know how to spell this.
And yet the people doing the closed captioning are excellent at this. You would be surprised how obscure project names and so on, they are just perfectly able to spell just correctly. And it always touches my heart when I see my own name spelled correctly, which also does happen. They added me to their alphabet. So I'm like, ah, this is awesome.
That is awesome. The personal touch means so much. It does. And it's worth going that extra mile in those circumstances. Well, we're here today to talk about Python 3.13. Hopefully, I'll be able to keep up with you, Pablo. Because not only do you speak fast, but I'm kind of a visitor to your guys' world. So maybe keep them in check, Lucas.
We're going to specifically talk about the biggest... And I guess, I mean, this is monumental, in fact... Our community member who thought we should do an episode on this, a shout out to Christian Klaus, says this is kind of massive for Pythonistas. This is the biggest feature coming out in 3.13, which should probably be out there if you're listening to this.
We're shipping this October 2nd and slated to release October 1st officially. At least the RC is out there, but software and ship dates, we think it's probably out there. if not coming very, very, very soon, the ability to disable or remove the global interpreter lock or the GIL, as it's so lovingly referred to. Let's start at base principles. What is the GIL?
And then after that, why would you want to remove it?
Right. So the GIL is basically a ginormous lock all around Python. So when, like Python is really, you know, is the language, but it really doesn't exist, right? Like it's an abstraction. And really, normally there is a second language behind. In CPython it's C, which is, CPython is the default implementation of Python that normally people use. There are others.
But then when this program that is the Python interpreter is executing Python code, it can have multiple threads at the same time. And all these little objects that Python exposes could be mutated at the same time, which is a problem. And Python in particular, Cpython as well, and most of the implementations is reference counted.
This means that to decide if an object is still alive or the interpreter can actually get rid of it, we have this little number inside every object that tells how many different structures or fields are using it. When this number reaches zero, we know that we can remove it.
But as any person who has done some multi-threading in computer science knows, having a little number being mutated by multiple threads at the same time is not bueno. So the solution is basically to add a ginormous lock all around the interpreter. Because we don't know what the threads are doing, we need to just basically lock the entire thing.
So this means that only one thread at a time can be accessing Python code and executing Python code. which means that you can still use threads because those threads can switch from one to the other, which is what we call concurrency. You don't need to wait until one thread finishes to start the other.
So they interleave, but there's no parallelism really because only one thread at a time can be executing Python code, which is a big problem in these days when we have... you know, machines with 120-something cores if you have a repair or servers, right? Or even your phone, right? Like, your phone has plenty of cores.
And right now, you're running Python on iOS, which is now, I think, by the way, this is something... Python on iOS is a thing as well.
Let's earmark that for a future part of this conversation. So what you're saying is... When running Python, you can switch quickly between cores, but you can't actually run the same code on multiple cores at the exact same time. So you're saying it's concurrent, but it's not parallel. Yes, correct.
Because of that lock, one at a time, and it can switch at the speed of light, but no true parallelism until now.
Right, so there were a few ways in which we worked around this obviously huge design limitation. So there are plenty of cases in which we are confident we can drop the global interpreter lock for what is happening on a particular thread.
So if your particular thread is doing some big NumPy computation that is only on the sides of numbers that are not shared as Python objects, you can drop the global interpret lock in the time that this computation happens so you can let other threads execute Python in that time.
If your thread is waiting on a blocking network connection, it also does not need to hold the lock because that particular moment in time, there is not going to be any Python code executing. So you can give that gill to somebody else. So there are special cases in which we were already sort of parallel more than not. But it is very implicit.
It depends on us carefully finding all those special cases and enabling them. And in the end, it does not scale for regular user applications. So if you just write Python code that loops over things and just executes some string concatenations, all these regular things that you expect from a Python program, most of the time you're not going to be seeing any parallelism there.
And what is worse, one important thing here is that technically a GIL is more than just a lock. It's a condition variable, which is a very specific kind of synchronization primitive. It's a Boolean that tells you if you have it or not, and an actual lock. And the thing is that you cannot select which, like when one thread says, oh, I don't need the GIL, like someone else, you know, pick it up.
You cannot select which one is going to do it. Like this is up to the OS. You are using Linux, so the Linux kernel decides who is the next one. And this can be Not great. Because the Linux guild may not know what you're doing. So you can have one thread, for instance, that is super CPU hungry. It's just crunching numbers. So that thread is all the time wanting this guild, right?
And then you have many threads that are doing this IO operation. Maybe they are waiting on a socket or something like that. But because of how condition variables work and the scheduler in Linux works, which you can pick a bit, but not from Python normally, that CPU thread will pick up that condition variable and the guild many more times. So it's almost like a race, right?
When the guild is released, It's like you have a gun in a race and you fire the gun to signal that the race starts and then all the threats try to grab it at the same time. And the first one that has it, that's the one that wins. But there is nothing preventing that threat to keep it getting it the next time and the next time.
So even if you are used to it and you can still say, well, maybe I can work with that limitation, it's very difficult to reason about it and to try to adapt it to your needs because it's really up to the OS who gets the kill or not.
This is one of the primary reasons why Python's reputation is being serviceable in terms of speed, but for certain workloads, certain things, it's too slow, right? Because of the GIL. This is one of the reasons why people will say, eh, I might pick something else because of the GIL. Fair or not fair?
Yes, well, like, it depends. It depends, yes. It's a factor. Tell me why. Right. So first of all, the speed of Python is mostly affected by the fact that it does a ton of indirection. The thing that costs the most in computer science, say for bad algorithms, is indirection.
So every time where some language can just point at memory, read a value, and it's done, it's going to be faster than a language that, well, actually everything is dynamic. So you point at a value, but then that value has another value. So you have to go somewhere else in memory to read another value, then decide what to do. and read yet another value.
So every time in Python that you're accessing an attribute, you're executing a ton of code compared to accessing a field in a struct in C. So that's going to be the main factor of it being slow. But the way where Gale is hampering us is scalability, which means simply for things that you could parallelize, you cannot do that effectively.
So this is where the reputation of not really scalable comes in. But ironically, the GIL also makes Python faster than it would be without it. The reason why is because reference counting is a major reason for the global interpreter lock's existence. But it's not the only reason you need a lock in your interpreter.
There's going to be internal data structures like dictionaries where if you insert another key that wasn't there before, maybe there's going to be some rehashing that happens, right? So suddenly the simple operation is going to take a little more time than usual. So there's going to be a time, a split second where the internal structure of the dictionary is inconsistent, right?
For example, you would be able to see that it has more elements in terms of count than you actually are able to access or you're going to be trying to access an element that is there. It claims it's there, but you cannot get it yet because the dictionary is being mutated, right?
So for all those things, and there's plenty of cases like that, the global rapid log is very convenient because we are already holding it because we care about the reference counts to be always correct. So there's no other lock we need to care about. And this is actually making matters cheaper because we only have this one. And once you hold it, you can just run and it's great.
And plenty of authors of C extensions like NumPy and so on can also depend on the fact that as soon as you have the golden number lock, you have a lock that you can yourself depend on for your own synchronization needs. So now, without that, you're going to have to have fine-grained locking. So for every of those use cases that require synchronization, you're going to have to have a smaller lock.
And it turns out that if you do this, then you gain scalability, but you pay the price of single-threaded performance, since catching, you know, being able to acquire multiple locks is necessarily slower than just doing it once, right?
It also has other consequences apart from this particular aspect, which is correctness. Right now, if you have ever done multi-thread in a language which requires locks, like C or C++ or something like that, when you don't have those locks, then you go into this thing that people call undefined behavior. Unicorns can appear in your program and things can happen.
In Python, because it's serial, like, sure, the results can be garbage, but it's not going to crash because, like, you know, it's just one at a time. And this especially happens as well on these C extensions or these native extensions, which can be anything. Like, if you have, you could think about it, like, well, you know, if I add one of these logs per extension,
Sure, I may have these problems that Gukesh is mentioning, but it doesn't sound too bad. But one of the problems, if you have many locks, is that now those locks can interact between themselves.
So if one extension just happens to call another for any reason, like there is callbacks scheduled or there is actually direct interaction or something like that, those locks now can do things like deadlocking or livelocking or a reentrancy if you don't have something prepared. And remember that these extensions are made by different people. So now you have bugs that only happen across them.
And that is a nightmare. It's a real nightmare, right? And Regil is basically protecting this because there is one single one. So all of them share it. So it's very easy to reason about it compared to not having it.
Hey friends, you know we're big fans of fly.io and I'm here with Kurt Mackey, co-founder and CEO of Fly. Kurt, we've had some conversations and I've heard you say that public clouds suck. What is your personal lens into public clouds sucking and how does Fly not suck?
All right, so public clouds suck. I actually think most ways of hosting stuff on the internet sucks. And I have a lot of theories about why this is, but it almost doesn't matter. The reality is if I've built a new app for generating sandwich recipes, because my family's just into specific types of sandwiches that use Braunschweiger as a component, for example.
And then I want to like put that somewhere. You go to AWS and it's harder than just going and getting like a dedicated server from Hetzner. It's like it's actually like more complicated to figure out how to deploy my dumb sandwich app on top of AWS because it's not built for me as a developer to be productive with. It's built for other people.
It's built for platform teams to kind of build the infrastructure of their dreams and hopefully create a new UX that's useful for the developers that they work with. And again, I feel like every time I talk about this, it's like I'm just too impatient. I don't particularly want to go figure so many things out purely to put my Sandwich app in front of people.
And I don't particularly want to have to go talk to a platform team once my Sandwich app becomes a huge startup and IPOs and I have to do a deploy. I kind of feel like all that stuff should just work for me without me having to go ask permission or talk to anyone else. And so this has informed a lot of how we've built Fly. Like, we're still a public cloud.
We still have a lot of very similar low-level primitives as the bigger guys. But in general, they're designed to be used directly by developers. They're not built for a platform team to kind of cobble together. They're designed to be... useful quickly for developers.
One of the ways we've thought about this is if you can turn a very difficult problem into a two hour problem, people will build much more interesting types of apps. And so this is why we've done things like made it easy to run an app multi-region. Most companies don't run multi-region apps on public clouds because it's functionally impossible to do without a huge amount of upfront effort.
It's why we've made things like the the virtual machine primitives behind just a simple API. Most people don't do like code sandboxing or their own virtualization because it's just not really easy. It's not there's just no path to that on top of the clouds. So in general, like I feel like and it's not really fair of me to say public cloud suck because they were built for a different time.
If you build One of these things starting in 2007, the world's very different than it is right now.
And so a lot of what I'm saying, I think, is that public clouds are kind of old and there's a new version of public clouds that we should all be building on top of that are definitely gonna make me as a developer much happier than I was like five or six years ago when I was kind of stuck in this quagmire.
So AWS was built for a different era, a different cloud era. And Fly, a public cloud, yes, but a public cloud built for developers who ship. That's the difference. And we, here at ChangeLog, are developers who ship. So you should trust us. Try out Fly, fly.io. Over 3 million apps, that includes us, have launched on Fly.io.
They leverage the global anti-cast load balancing, the zero config private networking, hardware isolation, instant wire guard VPN connections with push button deployments, scaling to thousands of instances. This is the cloud you want. Check it out, fly.io. Again, fly.io. And I'm also here with Kyle Carberry, co-founder and CTO over at Coder.com. And they pair well with fly.io.
Coder is an open source cloud development environment, a CDE. You can host this in your cloud or on premise. So, Kyle, walk me through the process. A CDE lets developers put their development environment in the cloud, walk me through the process. They get an invite from their platform team to join their coder instance. They got to sign in, set up their keys, set up their code editor.
How's it work?
Step one for them, we try to make it remarkably easy for the dev. We never gate any features ever for the developer. They'll click that link that their platform team sends out. They'll sign in with OIDC or Google, and they'll really just press one button to create a development environment. Now that might provision like a Kubernetes pod or an AWS VM.
You know, we'll show the user what's provisioned, but they don't really have to care. From that point, you'll see a couple of buttons appear to open the editors that you're used to, like VS Code Desktop or, you know, VS Code through the web. Or you can install our CLI. Through our CLI, you really just log into Coder and we take care of everything for you.
When you SSH into a workspace, you don't have to worry about keys. It really just kind of like beautifully, magically works in the background for you and connects you to your workspace. We actually connect peer to peer as well. You know, if the coder server goes down for a second because of an upgrade, you don't have to worry about disconnects. And we always get you the lowest latency possible.
One of our core values is we'll never be slower than SSH, period, full stop. And so we connect you period to period directly to the workspace. So it feels just as native as it possibly could.
Very cool. Thank you, Kyle. Well, friends, it might be time to consider a cloud development environment, a CDE. And open source is awesome. And Coder is fully open source. You can go to Coder.com right now, install Coder open source, start a premium trial, or get a demo. For me, my first step, I installed it on my Proxmox box and played with it. It was so cool. I loved it. Again, Coder.com.
That's C-O-D-E-R.com.
So if you have slower single-threaded performance, you have loss of features in terms of it being more complexity, potentially, because your loss of a single lock, now you need fine-grained locks and all kinds of other things you can do, which is problematic. And a huge lift by many developers over how long you guys have been working on this?
This has been many years in the making.
Many years in the making. Yes. Okay, add all those together. Worth it? Well, we'll see.
We'll see. We cannot answer that. We cannot answer that. We don't know. So we're in the stage of... So actually, this is an important thing, by the way, which I don't think we have mentioned. This, in 3.13, indeed, there is a build of Python that will not have the Glow interpreted log, but you don't get it by default. You need to manually compile Python.
I mean, there is people that are already distributing Python with this, so probably you don't need to manually do it. Like, you can fetch it from your distribution probably.
Somebody will build it.
Yes, exactly. But it's not the normal one. Like, if you install Python 3.13, it has the guild.
It's not like a flag that you give when you run Python. It's a separate executable.
Yes, you need to, it's actually called Python 313T. T. Or 43. Whatever. It's just the way it is. Sure. It's free-threaded. So you need to run that. And with that build, you can actually select a runtime only with that one. You can select a runtime if you want the guild or not.
And this is very useful in case you want to try the performance difference or because you want to, like imagine, for instance, you are doing one of these libraries and you want to test if your library works with and without but you don't want to run your CI with two builds or whatever. So you can do it.
There are consequences of this because, for instance, in Python over the years, we have added a lot of new optimizations. Like, for instance, one of the things that I think Lucas was missing on the reason Python is slow is that apart from this whole new direction, and it's actually part of the reason, but is that it's dynamic, right? Like, so this is very important.
Like, if you are adding two variables, x and y, the interpreter is going to say, what is x? Oh, x is an integer. Okay, let me fix the code for adding integers. Okay, what is y? Oh, it's an integer. Okay, let me fix the code. Okay. And then the next iteration of the loop is going to say the same thing. What is x?
So, like, it's very annoying compared to just, like, you know, here's a bunch of bytes, and I'm going to add this to this bunch of bytes because I know what they are, right? So in this particular case, precisely to avoid this dynamism, we have added a lot of optimizations that try to find this pattern. So they say, okay, you're adding these two variables together.
But, you know, the last 3,000 times I have seen this addition, they were integer. So, you know, it's kind of a good chance that they are integers the next time. So I'm going to create this like super performance code that is going to just reach into the internals of the integer object in Python. It's going to grab the real integer and it's going to add it at CSP.
It's going to get the result without asking all these stupid questions and then it's going to box it again into a Python object. And it's much faster. You can still have a float suddenly and it will de-optimize and whatnot, but that is a good problem. All those optimizations don't work without the GIL because they're not thread-safe.
So if you use the single-threaded version or the free-threaded version, which is a bit of a mouthful, so you use Python 3.13 T, but you don't have the GIL activated, you also don't have the optimizations activated because we don't know if they are safe or not. So what does it mean? It means that the free-threaded version is slower in single-threaded.
So you only run one thread and you're using the Python 3.13 T version, it's slower because they don't have those optimizations activated.
slower for now or slower for good right for now great question so let's maybe take a step back and explain the kind of strategy that we took with this particular project which is a huge undertaking but at the same time we have a release cadence that releases pythons every year predictably in october you're gonna get a new version of python
So there's no way you can shove a big transition like this within 12 months. So what we're doing instead, essentially what you're getting is a state of the project as we are in right now. So there are things that we still need to do that are still not done.
most importantly this split for the main interpreter still having the global interpreter lock and so on and an experimental one that you have to build yourself and then you can run without a governor block that is deliberate because with all those costs that we described um we really need to reach out to the community saying like look this is now not an idea you can actually grab this python
and check whether this helps you is it better is it worthwhile the scalability for this extra complexity within the implementation and maybe this cognitive complexity that is going to be sort of passed on to the user is that worth it so the community will have to now step in and try out this new python and see okay this is slower than the non-sorted version but that's hopefully temporary but
the question to answer is, does the scalability that this enables help? So we really need them to give us two thumbs up, like, okay, keep actually working on this. Since this specialization that Pablo described and the JIT compilation that follows from that and so on and so on are obviously features that we really want to have, including in the free-threaded version.
And there are ways to go around it, but they are very complex. It is in fact like on the bleeding edge of computer science as we know it. There are plenty of known strategies for improving performance of interpreted languages. that are single-threaded. For free-threaded interpreted languages, this is really where the science sort of is still unclear, right?
Because JavaScript, there has been so much investment in that, like making that fast with V8, right? Like in Chrome and so on and so on. Well, that is single-threaded for the intents and purposes of our understanding of the interpreter. With a free-threaded Python, how to make that fast is more of a challenge.
Literally, members of our team are going to be the people writing papers about this later. So this is going to take time, no longer than one release, longer than two releases. But at the same time, we want to release what we already have so that it's no longer just a pipe dream. It's no longer just a plan or idea. Users can actually experiment with this and tell us, okay, this is actually awesome.
This enables a use case we couldn't even try before. Because that's our chicken and egg problem. Before we actually enable this, we don't see... use cases that are now impossible. So we cannot tell whether there's an untapped area of Python usage that would be now enabled with free threading. So this is what we're doing with 3.13. This is essentially this current state of the project.
Now you can experiment with it. You can see whether you're web framework is going to be faster now, whether now you're going to be able to process data in parallel and so on and so on. This is essentially, you know, one step of a longer project.
Right. And it's out there in Release Canada. It's probably out there as we speak, at least in the official 3.13 experimental, as you guys pointed out. Are there early indicators? Are there people who have been just pining for this and they've already tried it in the RC and are saying, wow, we're seeing this or...
or no actually that's a very good point so one of the things that actually surprised me a lot because like we didn't knew one of the unknowns here was how the community was going to adopt this the reason being that it's just not that python needs to be multi-threaded that now if you compile uh c extensions which are really really popular in python right that's one of the reasons people still people so many people use python the language because you know all the machine learning doesn't happen in pure python it happens on
compiled code and CUDA and GPUs, but like Python is the driver or the glue. So all that code needs to be changed or adapted at least. Because like, you know, now you don't have that lock protecting you and like who knows what happens now, right?
So you need to, which is really challenging because right now you need to go through the entire code base, like the huge thing and sanitize the code base through this particular problem. And there is not a manual for that if you think about it. So you need to reason about the entire thing.
And we were a bit scared about how people will approach this or what will happen in the sense that if you don't do this or you don't do this correctly, it's going to crash all the time or it's going to crash just once every blue moon or whatever, right? So we don't know. So one of the things that we have seen, and it's really impressive, is that the community has jumped this incredible.
People are building two versions of every of these extensions, one for the normal build, one for the free-threaded build, which is already huge, right? This double your CI time. That was your storage for the extensions. So it's a lot of work, actually.
And there's a lot of people that are so excited to jump to these extensions and they try to fix it and reason and try these little locks all around. So all those people have been trying it. And one of the things I have been super, super surprised here is that
once you add like those extensions and you have like a final application like i don't know a machine learning whatever it is right or like an llm or whatever you're thinking that uses a lot of these things right it's just not one like like five six seven really complex one plus the interpreters plus maybe a web server something like that and then you run it through the free thread version it just works i mean again who knows like maybe maybe every saturday just
you know like you need your entire SRE team looking at it but like but right now like it's not mega crashing because what we thought is that this was going to immediately crash like when you add enough complexity to this because it's just a statistics thing right like the chances that there is some bad interaction when you pull out the threads and the amount of code that is here is just is inevitable right and it's still you know from time to time we find
very funky interpreter when like you know you call a method on something sometimes like another method is called because like there was something like a memory reordering problem which is a very technical like deep thing right but like like and debugging that particular thing is just hard but like that is very rare and obviously we fix it immediately
and we are really surprised about first how many people have jumped at this and they're super excited and they want all of those extensions and working and second how quote-unquote stable this is like meaning that how little bags we have seen in the wild like we expected many many many more right and we are seeing very little so so right now it's super super promising yeah like what what does help is that we do have the people behind it like some girls who actually did the prototype of 39 and then 312
now who is working on making this for 3.13 and forwards, is being helped by other engineers who are like really into this and are able to find threat safety issues and data races and so on. We were able to set up threat sanitizer, which is very complex to set up in an interpreter environment like Python. Turns out it does help us with finding those issues.
Some of them are pretty funny, like figuring out that for this entire time, the thread handle in Python was not thread safe itself. Or the threading lock was not thread safe itself. So once you actually did the operations of locking, yes, those were thread safe. But the Python object that you used was not because it didn't have to be. And now all of those things have to be actually fixed.
So they are probably formally correct now. So yeah, there's been a lot of work, and a lot of work is still to be done. But our hearts are warmed by how little terrible and unsolvable issues we've seen so far. Some of them are complex to debug and complex to diagnose, but we've been successful in improving the situation so far as we go.
So for your average programmer out there with their average Python app, whether it be a web app or a desktop app or whatever, should they be hopping on this experimental and giving it a try? Or is it a thing where it's like, if you don't know that your code is thread safe, it's probably not? Or is it the other way around? What's the water feel like? Is it warm? Are there alligators in there?
It depends, right? If they are paid to make sure that the thing is stable, the recommendation is no. I mean, try it out, sure. You can get very excited, by the way. One of the things that we have seen, me, myself, right? For instance, I'm at the company I work, Bloomberg. is that we have tried to see what will happen.
Even if it crashes every three months, whatever it is, but just to see a sense of what is the improvement here. Because if then down the line you're going to justify any effort on your side to adapt your thing to the free-threaded version, it's really good that you know what you're going to gain.
So that for sure, trying, we really want people to try and tell us what they see and what doesn't work. Is this a production-ready workload? Absolutely not. Not because we know it's going to crash or anything, but because we don't know the real status of the thing, right? We take stability very, very seriously in the core team, so we are not comfortable right now.
That's why the thing is called experimental. Because we are not comfortable right now recommending people, especially when you not only have the interpreter, but all these other things. And still we need to figure out good APIs for those extensions to hook into the interpreter and things like that. And that is not there. right now.
So the chances that we can look at these things and say, yeah, this can run in production, no problem, is almost zero right now. But as we release Python 3.14 and 15, we will move this experimental into supported, which means that you can run it in production, but it still will be two versions.
And eventually, when we gain enough confidence that this is the future we all wanted, it will be only one Python, which will be Python without the free-threaded version. But until we call it stable and, you know, supported as the keyword supported, I wouldn't recommend people to actually use it in production environments.
Yeah, production environments. However, I would be a little more optimistic about this, saying that if you are an end user, probably the kind of the amusement park is still being built for you. So like this is this is not like something that is ready for end consumption. unless you are really just trying to just see how pure Python works without all the libraries that you depend on.
However, if you are a maintainer of a library that a lot of people depend on, you might be interested in checking it right now. So we already see plenty of interest from the data science libraries and web frameworks and so on and so on. And that's exactly the right moment for them to,
to identify whether there's any blocking issues or whether the gains are not enough for us to justify the change, or maybe the opposite, or maybe they're amazed at how better things are right now. And like, funnily enough, even things that were not ever designed with free threading in mind, like asyncio.
You run the test suite of asyncio and it's just faster on free threading because it just can magically utilize the fact that it is now truly parallel. So there is promise there. Definitely we expect that once the library authors actually go through some of those hoops that are required for their things to be supported by the free threaded version,
we're going to see uptake on more libraries and then more users and so on. So now's the time for the library maintainers to check it out. But for end users, it is a little soon.
That makes a lot of sense. Somebody should start a list of libraries which have taken that initiative.
It's there.
Cool.
Very cool. Well, it's definitely exciting times. I'm even excited. I don't use Python on the regular, but I think that's early indicators of something that's potentially a huge success. So congrats to you guys and the whole team for the effort, at least, to
make it no longer theoretical and make it an actual piece of software, bleeding edge piece of software that's going to produce even some papers. Should we move on to the JIT? Lucas, you mentioned the JIT is another feature of 3.13. Is there more to say about the GIL? I'm assuming these things will probably interact as topics.
Right. So the JIT is still one of those other topics that I remember being this big pipe dream of CPython for as long as I can remember. We grew a few attempts to have a JIT in forks of Python or entirely new Python interpreters. Most of those efforts failed. There's one holdout that is actually doing very well. That's PyPy.
And that is actually JITed, which proves that this language can successfully use this performance strategy for good results.
As a clarification also for the people that don't know, PyPy is not a JIT for CPython. It's an entirely different interpreter. So it's like the same language, but it's just a different interpreter written actually in Python, which is JITed then. But it's a separate thing.
Yes, it is an entirely separate technological stack. It is attempting to be bug-to-bug compatible with CPython. People adopting it have seen, your mileage may vary sort of results where it might be really a drop-in replacement in some cases. It might be more tricky to actually adopt it otherwise.
Plus the approach of Jiting they're using has some characteristics with memory usage being larger and so on. So it depends on your use case. But obviously the most popular version of Python is CPython is the version that we are working on. So this is an idea that just comes back and comes back and comes back. Like when will... Python have a just-in-time compiler.
So to a large approximation, if you squint really hard, this specialization that Pablo already mentioned is sort of slowly becoming a form of what digits are doing, which is to remove all this indirection.
But the last step that they did not take so far, those optimizations, is to actually replace running this C code with this C code with an actual compilation to machine code as we go dynamically, which is what the JIT is doing. And Brandt Bucher, one of our core team members, found out this paper about how you can automatically generate a JIT
for the Lua interpreter and how it is slower than a handwritten JIT, but not by that much as you would expect, and it is much more maintainable. And using this approach with this additional machinery that is needed for the just-time compiler that tries to compile during our build time, like the simple opcodes that are later glued together to this JIT,
optimize machine code and so on and to discover whether that trace that we're running is still correct or maybe we need to throw it away because we thought this thing that we're running is like adding two integers, right? But now we're seeing, oh, there's a float that we did not expect. So we need to throw out this optimized code and actually go back to the interpreter again
So all this infrastructure is already in 3.13. And so far, we can proudly say that with all this added, Python is not slower. 0% faster. Yes, it is 0% faster because, again, with the release cadence that we're having, 12 months of work. is a lot, right, obviously, but it is not enough to like just say we started from zero JIT and ended up with a mature JIT like V8, we are done.
It is just simply impossible, especially that our team is much smaller than the V8 teams, much smaller than the Java hotspot team and so on and so on. So yeah, so far there is another experimental feature in Python 3.13, which is the JIT. You have to compile with that and you have to enable it when you're running Python.
So it does not have most of the optimizations that you would expect from the just-in-time compiler. Because that entire technology of just-in-time compilation is essentially a framework on top of which you can now make optimizations that make things faster. For example, if within the trace that you're building, connecting all those machine code blocks that are now faster and removing
indirection, you have multiple checks whether something is none or something is an integer, you can move that check of that same piece of data to the front of the trace and just do that check once and you don't have to repeat it in different parts of the code and so on. and to make sure that the traces are as long as possible. Those are different levels of optimization.
You want to JIT relatively quickly so that people can see the uplift very quickly, but you don't want to JIT every code that is being executed because that would make the memory usage, crazy. So there's things that you need to dial in there and implement. So it is very much an in-progress project.
It looks very promising, but so far Python 3.13 is a snapshot of the current stage of that effort as well.
Pablo, anything to add?
Well, one of the things that is very exciting with the JIT right now is that the approach itself is quite cool.
Like, for instance, if you're making a JIT for a dynamic language, one of the biggest challenges is not only, you know, adding the infrastructure itself for a particular CPU, but like architecture is that you need to do it for all of them and for every single OS because, you know, at the end you're writing machine code and machine code depends on like, you know, your CPU or
architecture, your platform, you know, so you need one. So the JIT is not the JIT or as we like to know, call it legit. But, you know, you need one for Python, sorry, one for Windows, one for macOS, one for Linux, but also one for AMD64, one for AR64, one for ARMv7. So you can imagine how hard this actually is because You basically are implementing a compiler, right?
But this approach that the brand is taking not only has the advantage that is basically leveraging an existing compiler behind, in this particular case, LLVM, but also is doing it at the front. So it doesn't happen at compile time. All that assembly code has been generated before, and the only thing that we need to do at runtime is stitch those things together. So we have these templates.
So Clang is basically, or LLVM is not a runtime dependency. It's just a build time dependency. So you build all this assembly code, and it uses the native compiler for your platform. So if you're in some forsaken platform, like, I don't know, AAX or something like that, it will work if you are able to run Clang there. So that's great, but also,
which is really cool, and most JITs need to also implement themselves, is that we are able to leverage an optimizing compiler for this particular workload. So not only that assembly code works for every architecture because someone else actually implemented the backend, but also we can leverage all the optimizations that everybody that's using LLVM is using.
It's the same optimizations that Rust is using. Rust is using LLVM these days still. And they are using, I mean, if you program the IR, the intermediate representation correctly, and then you are able to leverage that well, which is not, you know, it's easier said than done.
But the idea is that now you can use this set common of very, like these many years of compiler research just to make your code faster. And you can just leverage that immediately as opposed to having to reimplement all those things and have like SSA ourselves and like, you know, dreaming and all that stuff. Like now, you know, you just run Clang and you get some
super cool assembly out we need to just stitch it together and feel like the symbols and whatnot but like it's a much like you can get results much much faster than we need to implement a full-blown git for uh for for python so very excited it's like a jit factory yeah right you guys use factories in python git template factory yeah
What's up friends? I'm here in the breaks with Dennis Pilarinos, founder and CEO of Unblocked. Check them out at getunblocked.com. It's for all the hows, whys, and WTFs. So Dennis, you know we speak to developers. Who is Unblocked best for? Who needs to use it?
Yeah, unblocked helps large teams with old code bases understand why something has been done in the past. It helps them understand what happens if they make changes to it.
Basically, all the questions that you would typically ask a co worker, you no longer have to interrupt them, you don't have to wait for their response, you don't have to, if you're geographically distributed, you don't have to wait for that response, you don't have to wait for You know, you don't have to dig through documentation. You don't have to try to find the answer in Confluence and Jira.
What we basically do is give you the answer by you just asking a question. The way that we got to the problem was a consequence of our kind of lived experience. We were actually going to call the company Bother, which is like, you don't bother me, I don't bother you, right?
Instead of like being tapped on the shoulder or interruptive Slack messages, we could just use Bother and get the answers that we wanted.
We didn't go with that name because it's a little bit of a negative connotation, but helping developers get unblocked by answering questions or surfacing data and discussions within the context of their IDE relative to the code that they're looking at is something that thousands of developers love so far.
I think our listeners are really familiar with AI tooling, very familiar with code generation, LLMs. How is unblocked different from what else is out there?
A lot of code generation tools help you write the code to solve a problem. We sit upstream of that. Our goal is to help provide the context that you need. If you think about where you spend your time when you're trying to solve a new problem, understanding how that system works, why it was built that way, what are the ramifications of changing something?
That's the problem that Unblock tries to solve for you. We take the data and discussions of all of these, the source code and all those systems to provide that answer for you so that you can get that context and then you can go and write that code. We had this great example of a company who hires, you know, very competent developers.
It took them five days, that developer, five days to write 12 lines of code. And his feedback to us was it's not that it takes you five days to write 12 lines of code. It took them five days to get the information that they needed to write those 12 lines of code. And that takes probably about 30 minutes to write those 12 lines of code and rip off that PR.
Okay, the next step to get unblocked for you and your team is to go to getunblocked.com. Yourself, your team can now find the answer they need to get their jobs done and not have to bother anyone else on the team, take a meeting, or waste any time whatsoever. Again, getunblocked.com. That's G-E-T-U-N-B-L-O-C-K-E-D.com. And get unblocked.
If we can move this conversation up a level for a moment, do you guys like this one-year release cadence? It seems like it has its drawbacks.
Well, this guy implemented that idea. So what do you think he's going to say?
Who did? You did this?
Yes. So, like, we can make this, like, good cop, bad cop, or, like, you know, actually, like, devil's advocate, and, you know, like, this guy who's trying to buy this, this is good. So, I'm also the release manager of some ancient Python versions. In fact, when Python 3.13 comes out, the first version I released, which is Python 3.8, is going end of life.
So, you know, we're going to say goodbye to that. It was a very, very... you know, nice release, I really enjoyed Python 3.8, but it's time to move on. So if your use case, if your production is still running on 3.8, well, now it's high time to move since it's not going to receive any security updates anymore.
Right, so at the time I joined as the release manager, the release process was a little bit hand-wavy in terms of when are we releasing Python. So the theory said we're releasing it every 18 month-ish. But it could be anywhere between 18 and two years, and you couldn't really tell. And definitely what you could not expect was which time of the year is this gonna happen in.
And the consequence of this was that Linux distributions in particular, but also other operating systems and users, were lagging behind adoption of Python 3 versions a lot. People are like, well, Python 2, no, because Python 2.7, right? But Python 2.7 was out and then essentially frozen for... a long, long time.
So people forgot how this thing looked like before in times of Python 2.3, 2.4, 2.5, 2.6, and then 2.7. That unstable release cadence was problematic. And that cost, for example, for Fedora, which later is source for what is happening in Red Hat, was lagging behind with the releases that we had.
And it turns out that the infrastructure that the big Linux distributions have, like Ubuntu and like Red Hat, is extremely useful for us to early find bugs in a new version that we are developing in the betas, in the release candidates, and so on.
So it would be worthwhile if we could actually ship the newest version of Python with those Linux distributions as they are also developed and go through their respective betas and so on. That's also great for the end user, right? Because then by your autumn release of Fedora, you are released with the newest version of Python that was just released. That is a great experience for the end user.
So I... wrote a Python enhancement proposal document arguing those things that I'm just telling you, that it would be amazing to have a fixed release cadence that we can trust and share with the Linux distributions such that they can essentially rely on us having certain milestones in the release, like reaching beta one, like reaching release candidate one.
so that they can use the latest version as soon as possible. And we switched to the release cadence in Python 3.9, and we've been using it ever since. So yeah, Your Honor, that's my position on this issue.
Well, as the other release manager of Antim version now. So I release 3.10 and 3.11. I mean, one of the problems that this thing, I mean, it has worked really well, I think. Like people in general, and just to be clear, like, you know, so Wookas doesn't kill me after we finish the podcast. I think it's a positive change, right? So in general, very good. I think it has been some predictability.
People actually now have like, you know, dates and they can, you know, put their lives around.
one of the the interesting you know side effects of this is that now people have this this impression that python is releasing too often so every single year is a new version and then the old version is like you know um out of uh support or like first back fix support and then security support so they have this feeling that you know they're going to be uh if they don't move fast enough uh they are going to be able to support super soon which which was always true is that
you get the numbers changing slower. So the impression was that it's happening slower. But right now, this is moving this impression that Python is changing too quickly, right? Or whatever it is.
The other downside that this has is that, as you can see, for instance, for Python 3.13, it means that because we need to release every single year, we're releasing these features, maybe like big ones, like the Nogil or the JIT in particular, not being super exciting. So for instance, JIT in particular is 0% faster. Well, it's kind of cool, but why would you use it?
If we had a two-year release, maybe it would be a bit better. So you get these weird situations. So planning big projects right now is a bit more challenging. For instance, one of the things that happened in Python 3.9, if I recall, we changed the entire parser of the language. Different parser, which I'd like to describe as changing the wheels of a car that is moving. Not the best feeling.
We could have some extra rest if that was a two-year release cadence and we didn't have to finish everything and all the validation in one year. I mean, this doesn't mean that it was rushed. It's just that we have to put more hours just because we wanted to make to the release. But on the other hand, like this, you know, it's humans making it.
So even before when the situation was, like Lukasz was describing, the day of beta freeze, which is the moment when no new features can come in and only backfixes from that point on in the release, everybody was rushing their feature on the last day, you know, like, All the new features. Humans are humans, so it doesn't really matter.
Certainly the heat has been raised a bit because right now it's released more often and the numbers are changing faster.
That was more the angle I was coming from. When the train leaves the station once a year, you better get your stuff on it, whether it's ready or not. And I appreciate, I didn't know the history of where it came from. So I think this is definitely progress and improvement and predictability. You know, each time of year it's going to go out, et cetera, et cetera.
And it makes a lot of sense, especially a large project like this with tons of people using it. you want to be predictable, but also not too frequent. I would probably tend towards more frequent because now you have the train leaving the station more times. And if you don't, if you miss this one, that's all right. We got another train leaving six weeks from now.
And so you can hop on when it's ready versus having to hop on once a year. And that's, you know, we've seen success in projects like Chrome, et cetera, with like rolling every six weeks, whatever's ready, we're going to ship a new version. I don't know if that works with languages and runtimes. You guys know way better than I do, which is why I asked the question.
I guess for compiled languages that might in fact work better, for us the difference is that we're interpreted. So there's a lot of code that is lying next to your interpreter and then you're trying to run it with whatever is available to you. With a lot of the libraries that we're supporting, they're a particular issue. So now I'm going to be the devil's advocate because Pablo was too apologetic.
Wow, wow. Sounds right. Yeah, let me just say like, hey, this is all a compromise, you know, and there's obviously going to be things that are better and things that are worse. The price to be paid for shorter release cadences is that the matrix of tests of CI that a library maintainer has to now kind of suffer with, has to run, is larger, right?
Because the number of versions that is out there that people are using, there's just more versions than there were before, right? You had 2.7 and 3.6 maybe, and then 3.7 appeared some 18 or 19 months later, so... then the growth of this matrix was sort of spread out in time. Now that matrix is bigger.
I would claim that, okay, it's good that we're testing with every particular version to identify where an incompatibility arose or something. However, I always like to say that it's still the same sausage. We're still the same sort of team that is developing the thing. We still have the same velocity, only we slice it thinner now.
So every release is easier to migrate to compared to the last release because necessarily there were fewer changes.
Yeah, psychologically speaking, that's a very good point, actually. I'm glad that you mentioned it.
Because one of the things that happens when you release every two years or more, especially for a language that needs to evolve and needs to change things, and maybe doesn't have... I mean, we can add our backwards compatibility, don't get me wrong, but other languages like C++ clearly care even more. So for a language that changed this much,
changing and adapting your code and all the libraries for a two-year worth of changes is brutal. Which means that, and we saw that before, right? Which means that until new libraries are compatible with the new version, there's a non-trivial amount of time.
So as a user or as a company employee that you want to leverage these new features, you probably want to wait a lot because now people talk about scheduling the work and finding even how much it is. And library maintainers say, well, I will fix it later. But when the changes are smaller,
that is like, psychologically speaking, obviously, in reality is more or less the same, modulus and nonlinear interaction. But in general, when the changes are smaller, then it's more enticing because at least you know, okay, maybe this, this, this, and this thing. And sure, maybe you need to wait for another library to do it, but like,
Normally, like right now, what we have seen compared with 3.8 when we started doing this, right, is that people jump earlier. Like before, for instance, one of the biggest libraries is NumPy, which is numerical computing in C and Fortran and whatnot. So that library used to take a long time to be compatible with new versions, like months or sometimes even like half a year or something.
And right now we are having it almost day one. So this is fantastic because people that want to use the new features in the language, they are not held back by the fact that their own particular dependencies don't have the support for this.
Most of these things, I don't think you can approach it from a data-driven kind of, like, perspective because it's very difficult to say what is better, what is worse. But, like, certainly, like, you know, from a psychological point of view, I think it's, again, I think you have a very good point. Like, not too soon, like, not too often, sorry, but, like, also not too, like, lagging behind.
I think it's, earlier is better, maybe not too early, like, every three weeks it will be impossible for a compiled language, like, We will ship a JIT compiler that unoptimizes your code, right? It will be like 6% or something. But one year, yeah, good.
I want to add one thing. So NumPy is not doing as well as Pablo is claiming. They're doing way better. There are already wheels for Python 3.13. Before the release of Python 3.13, those wheels are going to work on the final version of Python 3.13 released October 1st. We're releasing this like mid-September. They're already there. Moreover, they're also already there for the specific version
that is this experimental interpreter that does not have the GIL.
I think everyone, a non-trivial amount of listeners probably are saying, what is that wheel? What is this thing?
It's this binary package that lives on the Python package index on PyPI that people download NumPy from when they say pip install NumPy. So if you say pip install NumPy from Python 3.13, on day one, you're going to be using wheels that are already there before the release of Python 3.0.
Is this another word for a build? It's like a build? Basically, it's a binary package. So it's a package that doesn't contain Python code only.
Why don't we make up another term for it?
Well, this is the key, right? Because the Python package index used to, well, I think it still is, it's called a cheese shop because it's all about these Monty Python references and jokes, right? So it's a sketch of Monty Python that goes to this cheese shop. And a wheel is a wheel of cheese.
Ha ha!
taking that one a bit too far maybe or like pip do you know pip I mean this is a slightly different joke but like do you know pip pip is a package installer for python right so when you install python you do pip install do you know what pip means python install packages pip installs packages Yes.
Okay.
So a recursive acronym.
Recursive acronym.
Yeah. Recursive acronyms I'm well aware of, but I'm not a Monty Python guy, so I missed completely this reference.
Right. So, well, you know, I was missing this reference for years and someone pointed it out and I was like, well, I suppose it makes sense.
Like I have to watch it. Is it from Holy Grail or which movie is this?
No, I think it's one of the Flying Circus, maybe?
Yes, one of the episodes of the Flying Circus has a skit about the cheese shop, and that's where that comes from.
A little obscure reference to build an entire Python ecosystem around, but okay.
I think it's not helping for Python programmers to look like they're in a co-win or something.
But you just need to have some file extension for those things and some name for those things. We used to have a more primitive, early format for this sort of thing, and those were called eggs. Eggs were because, you know, snakes lay eggs, right?
So that made this other sort of reference, because Python and the name comes from Monty Python, but a lot of people just see the name Python and they're like, yes, snakes are cool, so snakes. We accept both sort of ways of interpreting this because snakes are indeed cool, yes, sure.
But yeah, now that we knew that eggs are not enough for what we need for proper binary packages, we were like, where to go now with names? And naming is hard, let me tell you this much. Naming is very hard.
Someone jumping in the middle of the podcast listening to us talk about eggs and snakes. Wasn't this about tech?
Again, this background is spectacular because you do run out of references with snakes. How many things can you actually reference? But with Monty Python, there's just countless references you can make, just people won't get them.
But yeah, they're very obscure these days. It's an increasingly old and hard to get.
For instance, one of the funny ones, one of these attempts at Git compilers from back in the day, I think from Google, was called Unladen Swallow. From the Monty Python Life of Ranger? No, no, no, sorry.
Holy Grail.
Holy Grail, yeah. The speed of the Unladen Swallow.
Good stuff. Okay, good meta conversation on release cadence. I think I agree with all your points. You have to find that sweet spot and as frequent as you can be reliable without being too frequent. And that probably varies per project. And I'm happy that you guys are happy with once a year. Certainly better than what y'all were doing before. That's very cool.
Let's go back to Python 3.13, because Pablo, at the beginning you mentioned it's officially iOS compatible or something. What does this mean? You can just fire up Python on your iPhone?
Well, not quite yet, I would say. So see, Python has a very old build system. It's AutoTools, which is this computer make. People that have been doing Linux for a long time probably have run into this. And one of the challenges of this is that And then it's another different for Windows, but both macOS and most Unix platforms use that.
So for a long time, that was okay, but there was a lot of platforms that we couldn't support, partially because of the build system and partially because the source required changes. Like many of the things that we're, the efforts that we're trying to do right now to make Python available in the browser.
I don't know if you have heard about this, but right now there is a good effort to ensure that Python can run in the browser and it's a good alternative to JavaScript. This is a project, for instance, called PyScript, which allows people to run like Python instead of JavaScript, which is quite cool. So those require changes in these build systems and even in the source of itself.
And iOS in particular has quite a lot of challenges. Actually, probably Lucas can chime in to specify those since this is very close to his heart. But one person in particular that has been working for iOS support, which is Russell Kim-McGee. Yeah, he works right now at Anaconda, which is one of these Python companies.
So he has been working quite hard to ensure that both the build system and the source itself works on iOS, which has a lot of challenges, right? Because iOS has not only an alien way of doing things, but many APIs are not available there, or other apps are not available, but you need these specific things that you need to upload to
bless you with their power just to be able to use the JIT compiler. How does the JIT compiler write in iOS? Who knows?
Probably cannot. Maybe let's start from the very beginning there. So the goal that we have with this iOS support, for the time being at least, is not to have an application on the App Store called Python. That's not what we're doing.
What we are doing right now is to have the Python distribution, the source distribution, ready entirely for being embedded within another application that somebody wants to release on the App Store for iOS, right?
This was theoretically possible in the past with already a ton of hacks that Russell Keith-Magee historically made for a project called Beware that is just running Python on mobile on both Android and iOS. But the Python source code, our source distribution was not making this easy.
There are plenty of changes that you have to make because the phone platform that Apple came up with is very limited. It is limited by design such that it's more secure and it doesn't waste your battery too much and it does a bunch of other things that make it easier for applications to be just swiped up to kill them and they don't run any runaway processes and whatnot.
So there are APIs there that are not allowed. There are others that are just not implemented, so they're not available. So you cannot run threads and so on. So this sort of thing causes embedding Python to be complex. Then there's the new and increasingly strict rules around when you're packaging some sort of binary blob and want to put it on the official app store for Apple,
They have this automated and sometimes in person, but mostly automated review process, whether what you're publishing is safe. So there's plenty of things that interpreted language like Python would do that would be flagged by those automated systems as unsafe, that this code is doing something fishy.
So there are changes needed for those things to just pass review so that your application that embeds Python can successfully actually appear on the App Store. Like including until very recently, you couldn't really have just dynamic libraries lying around in your bundle that you're publishing on the app store.
Everything had to be statically compiled, which puts, depending on libraries like NumPy, into jeopardy because it is very hard for external C extensions to be built in a way that they are both all part of a single binary. Now this extension is lifted, however, you have to sign everything that you're publishing to the App Store separately.
So it has to be notarized by Apple, it has to be signed with your application developer certificate and so on. So you know when you're sending out this code to Apple that, okay, this is the code I built, nobody meddled with it. And then Apple notarizes it that, okay, this one is safe to be run by users' phones and so on. So all this infrastructure, it seems like
something that shouldn't really be our concern. But all of this was making it hard for Python to just be embedded. And now with Russell's hard work, this is a platform that we support. So if you want to have a text editor that has automation in Python, if you do want to have an editor for Python source code, there's a few of those, like Pythonista, Pyto, and so on.
Now it's going to be much easier for the authors of those to upgrade to a newer version of Python. This is why all those apps lagged behind and they were still on Python 3.8 or 3.10 because it was a ton of work to upgrade and redo all those hacks in the newer Python versions. So now, unnecessary, we support it.
That definitely wasn't what I was thinking, but it makes a ton of sense. I think it's going to make some people very excited.
What were you thinking about? Yeah, what were you thinking about?
Write some Python code and deploy an app based on it. Like, build an app in Python.
Right, so this is kind of step two for the Beware project that Russell Keith-Magee is part of and which is now being essentially sponsored by Anaconda where, yeah, you're going to be able to build Python applications from scratch and end up with an app on the iPhone that you can put on the app store and it's fine, right?
But the first step of this is that all this additional tooling that BWare is building on top of Python needs to rely on Python being embeddable.
Right, also conceptually, speaking is not different from the first use case because like you're not going to have one single Python in the app store that everybody can use and you know you need to like have everyone will have a different Python version so it's a trivial embedding so you're not embedding this on a text editor you're embedding it on a binary that is just Python Yes.
But you also want to see, like, you know, not only your Python code, you want to see your dependencies as well, right? So now, this is the same. Like, if a text editor, you want a text editor to have Python inside to do some extension, you may want some extensions that Python can do to interact with something else, right? So maybe you want also that on the dependencies.
So it's technically, you know, if you screen hard enough, it's really the same case. It's just that for the developer that just wants to put some files and maybe to say, I require this and that, that's a lot of work. You're in the business of compiling Python yourself. What are you talking about? So it's much better if we can offer a specific workflow that says, okay,
If your case is only like this and that, and you only have these dependencies, then you just click this, put this here, and the build process will take that for you so you don't need to. But conceptually speaking, it's just a dev tooling, not really a huge undertaking.
But really, it's like, you know, it's again, incremental steps. So the important thing here is as well that like, this is now part of our CI. We are building Python on iOS every time you make a change.
So it's going to be much less probable for us to introduce changes without knowing that they break support for running this on the phone, which should be great news for anybody that actually wants to see Python on mobile devices, because now we are going to care to keep this running.
Well, now you have me thinking about the future. Let's close on this wish list. Imagine that the three of us are sitting down next September and talking about 314. What do you want us to be talking about? What do you hope we're talking about going into 314, each of you?
Oh, okay. So my dreams are simple. I want everything. I like that. So honestly, I sponsored the, you know, making the optional PEP. So this is very close to my heart. What I would like to see is for this to pay off, for the library developers to step in, try it out and say like, hey, actually this is helping us.
What is related to this, to see that the performance, you know, efforts that we are having separately from the global lock removal actually converge and, you know, kind of do work with this free-threaded version. And also, finally, again, free-threading.
We don't expect people to actually start new threads randomly and end up in this crazy world of free-threading where you can just easily just write bad applications by writing on top of your variables and accessing data you shouldn't. We don't want to reintroduce models that already we know failed in other languages. We want tooling to be more high level.
So I want this more high level tooling to start popping up slowly. We deliberately didn't suggest any so far. We want to see what the community actually wants to use. There's kind of things that happened in other more younger programming languages. You know, you have
go routines right like you have channels in different programming languages and so on so I'm very excited to see where this goes so this free threading evolution is what I would like to be able to tell you next year that you know has done great leaps and it's now way more usable and it's literally the future so yeah
Well, in my case, I think, apart from what Kukash just said, I think the key for me is that this pays off, which is not trivial, right? It may pay off for some users, for sure, that are really hungry about this. But what I really want to know is that people, you know, for a lot of time, removing the guild has been almost like a joke about, you know, this is what is missing.
This will be the last thing and everything will be so green and fantastic when we remove the guild. Well, no, no, no, prove it. So I want to see everybody saying, now that there is no guild, I finally can do this and this thing, which is not going to happen fully, because people still will reach to other languages for other reasons and performance. But I just want to see a big impact there.
And ideally, we talk again for 3.14, we are saying, okay, this makes a difference. So it was cool, sure, and some people really like it, but wow, now it's a different play field. So it's different.
The other thing that I would like to see, and this is a bit more personal, so this is a project that I started, I think, in Python 3.10, which is that a lot of people were complaining that the user experience of the language was not really great in terms of messages and things like that. So syntax errors, and you mistype something, or there is some important problem.
So that was really bad, to the point that there were some situations that was laughable.
And we have worked quite a lot with the new parsers and with many other improvements to improve this to the point that right now I'm super happy to see that many, even papers that are researching error messages are citing Python as one of the, obviously not in the same category as Rust because we can all agree that Rust has fantastic error messages.
Different kind because those are compiled time error messages that we're talking about runtime ones, if you don't consider the parser bad. But we are right now quite high in that sense, to the point that I think there was some research about how LLMs can reason about what's wrong about your code from the errors.
And you run it in 3.11, it was like a 30% better performance of those LLMs just because it's telling you a bit more, right? So you can fix the code a bit better, which is quite cool.
So we continue doing this thing and we are trying to add even bigger changes to the interpreter that help in this regard, which is challenging because those changes sometimes require some non-trivial performance games there. So obviously we don't want to affect performance, but to do that, we need to put a lot of engineering there.
So in 3.14, we are probably going to try some new ones that are quite big. So hopefully when we talk again, those have actually been implemented and they have been successful, which I would love to see.
Yeah, that's awesome. I was reading about your improved error messages, and I think that's the kind of thing that has to be a sustained effort over time. It's like one of those paper cuts kind of things where it's like we're just going to continue to polish this thing. And so a great effort. I'm glad that you struck that up.
Well, the podcast Core.py, if you are a super Python nerd and want to nerd out with these two about internals. on a more frequent basis, then we'll cover it here on The Change Log, which sounds like it's going to be about once a year. I'd love to have you guys back next September and be talking about 314.
Hopefully, all of our wishes will have come to fruition and we'll have people out there free birding it. I mean, free threading it. Q Leonard Skinner here. And enjoying the new Python with no gill and the performance improvements that are possible there. Lots of potential. I'm certainly looking forward to it. Guys, thanks so much for coming on the show this week. Thanks for having us.
Yeah, thanks for having us. It's been a pleasure.
Okay, so now that Python is officially experimentally free birded, I mean free threaded, are you going to defenestrate the gill? Let us know in Zulip. Yes, the changelog community's conversations are now happening in Zulip. Join today by signing up. at changelog.com slash community. If you've already signed up, but you want your Zulip invite, just sign in and click on send me a Zulip invite.
You'll see it. Of course, it's totally free and a totally cool place to hang out and discuss. Let's do one more big thanks to our sponsors of this episode. Sentry. We love Sentry. You might too. Use code changelog, save 100 bucks. Fly.io, the home of changelog.com. Ship something in five minutes or less. Learn how at fly.io. And of course, to Coder.com and Unblocked. Please support our sponsors.
They support us. And thank you, of course, to the GOAT. Breakmaster Cylinder is the greatest beat freak of all times. Finally, thank you for listening to our shows. We truly appreciate you spending time with us each week. That is all for now. But on Friday, we are talking about developer happiness and unhappiness and productivity or lack thereof with our friend Abinoda from DX.
day.