Pablo Galindo
👤 PersonAppearances Over Time
Podcast Appearances
And what is worse, one important thing here is that technically a GIL is more than just a lock. It's a condition variable, which is a very specific kind of synchronization primitive. It's a Boolean that tells you if you have it or not, and an actual lock. And the thing is that you cannot select which, like when one thread says, oh, I don't need the GIL, like someone else, you know, pick it up.
And what is worse, one important thing here is that technically a GIL is more than just a lock. It's a condition variable, which is a very specific kind of synchronization primitive. It's a Boolean that tells you if you have it or not, and an actual lock. And the thing is that you cannot select which, like when one thread says, oh, I don't need the GIL, like someone else, you know, pick it up.
You cannot select which one is going to do it. Like this is up to the OS. You are using Linux, so the Linux kernel decides who is the next one. And this can be Not great. Because the Linux guild may not know what you're doing. So you can have one thread, for instance, that is super CPU hungry. It's just crunching numbers. So that thread is all the time wanting this guild, right?
You cannot select which one is going to do it. Like this is up to the OS. You are using Linux, so the Linux kernel decides who is the next one. And this can be Not great. Because the Linux guild may not know what you're doing. So you can have one thread, for instance, that is super CPU hungry. It's just crunching numbers. So that thread is all the time wanting this guild, right?
And then you have many threads that are doing this IO operation. Maybe they are waiting on a socket or something like that. But because of how condition variables work and the scheduler in Linux works, which you can pick a bit, but not from Python normally, that CPU thread will pick up that condition variable and the guild many more times. So it's almost like a race, right?
And then you have many threads that are doing this IO operation. Maybe they are waiting on a socket or something like that. But because of how condition variables work and the scheduler in Linux works, which you can pick a bit, but not from Python normally, that CPU thread will pick up that condition variable and the guild many more times. So it's almost like a race, right?
When the guild is released, It's like you have a gun in a race and you fire the gun to signal that the race starts and then all the threats try to grab it at the same time. And the first one that has it, that's the one that wins. But there is nothing preventing that threat to keep it getting it the next time and the next time.
When the guild is released, It's like you have a gun in a race and you fire the gun to signal that the race starts and then all the threats try to grab it at the same time. And the first one that has it, that's the one that wins. But there is nothing preventing that threat to keep it getting it the next time and the next time.
So even if you are used to it and you can still say, well, maybe I can work with that limitation, it's very difficult to reason about it and to try to adapt it to your needs because it's really up to the OS who gets the kill or not.
So even if you are used to it and you can still say, well, maybe I can work with that limitation, it's very difficult to reason about it and to try to adapt it to your needs because it's really up to the OS who gets the kill or not.
It also has other consequences apart from this particular aspect, which is correctness. Right now, if you have ever done multi-thread in a language which requires locks, like C or C++ or something like that, when you don't have those locks, then you go into this thing that people call undefined behavior. Unicorns can appear in your program and things can happen.
It also has other consequences apart from this particular aspect, which is correctness. Right now, if you have ever done multi-thread in a language which requires locks, like C or C++ or something like that, when you don't have those locks, then you go into this thing that people call undefined behavior. Unicorns can appear in your program and things can happen.
In Python, because it's serial, like, sure, the results can be garbage, but it's not going to crash because, like, you know, it's just one at a time. And this especially happens as well on these C extensions or these native extensions, which can be anything. Like, if you have, you could think about it, like, well, you know, if I add one of these logs per extension,
In Python, because it's serial, like, sure, the results can be garbage, but it's not going to crash because, like, you know, it's just one at a time. And this especially happens as well on these C extensions or these native extensions, which can be anything. Like, if you have, you could think about it, like, well, you know, if I add one of these logs per extension,
Sure, I may have these problems that Gukesh is mentioning, but it doesn't sound too bad. But one of the problems, if you have many locks, is that now those locks can interact between themselves.
Sure, I may have these problems that Gukesh is mentioning, but it doesn't sound too bad. But one of the problems, if you have many locks, is that now those locks can interact between themselves.
So if one extension just happens to call another for any reason, like there is callbacks scheduled or there is actually direct interaction or something like that, those locks now can do things like deadlocking or livelocking or a reentrancy if you don't have something prepared. And remember that these extensions are made by different people. So now you have bugs that only happen across them.
So if one extension just happens to call another for any reason, like there is callbacks scheduled or there is actually direct interaction or something like that, those locks now can do things like deadlocking or livelocking or a reentrancy if you don't have something prepared. And remember that these extensions are made by different people. So now you have bugs that only happen across them.
And that is a nightmare. It's a real nightmare, right? And Regil is basically protecting this because there is one single one. So all of them share it. So it's very easy to reason about it compared to not having it.
And that is a nightmare. It's a real nightmare, right? And Regil is basically protecting this because there is one single one. So all of them share it. So it's very easy to reason about it compared to not having it.