Re: Managing the GIL across competing threads

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Adam Preble
I edited the post so many times I forgot the subject line!  I am a bad person.

Maybe the most condensed question here is: How do you normally manage multiple resources competing with the GIL in a way that could cause deadlocks with it?  For the details, see the long post.

On Thu, Mar 15, 2012 at 11:59 PM, Adam Preble <[hidden email]> wrote:
I discovered recently that during callbacks to Python objects, sometimes the interpreter would try to do stuff at the same time, despite the fact I made a call to ensure the GIL.  I read the solution for that kind of thing I'm doing is calling PyEval_InitThreads().  This worked at first, but like more race conditions and stuff, all it takes is walking away from the computer for it all to fall apart.

What I'm seeing is a rather elaborate deadlock situation revolved around securing the GIL.  I think I need to basically put my interpreter in its own subsystem, design-wise, and ferret calls to invoke things in the interpreter to it, in order to ultimately get around this.  However, what I'm asking of the distribution is how they've gotten around this.

To give something a little more concrete--this is pretty gross:
1. Main thread starts interpreter and is running a script
2. The script defines an implementation of callback interface A
3. The script starts some objects that represent threads in the C++ runtime.  These are threads 1 and 2.
4. The script starts to create an object that is wrapped from C++
5. The object requires a resource from thread 1, where I use promises and futures enqueue and fulfill the request when thread #1 isn't busy.
6. Meanwhile, the interpreter thread is waiting for the resource since it cannot put it off any further
7. At this point, thread 2 tries to invoke the callback to interface A, and it needs the interpreter thread.
8. thread #1 needs thread #2 to complete this critical step before the get() call will complete for the master interpreter thread
9. Thread #2 needs thread #1 to finish so it can get the GIL.  It's basically locked up in PyGILState_Ensure().

Heck of a deadlock.  I am pondering having something else control the interpreter in its own thread and have everybody enqueue stuff up to run in it, like with the promises and futures I'm using elsewhere already.  The reason is that thread #2 doesn't really need to steal the interpreter at that very exact moment.  And furthermore, I'm trying to use Stackless, and it's my understanding there I can link it up so that the interpreter gets ahold of the Python runtime at controlled intervals--if desired--to invoke other stuff.


_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig


_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Niall Douglas
The key to avoiding deadlocks is, in every case, the appropriate
design.

In what you posted you appear to be doing too much at once, so you're
holding onto too many locks at once. Either replace those with a
single, master lock or try to never hold more than one lock at any
one time. That includes the GIL. In well designed code, one almost
NEVER holds more than two locks at any time.

Consider breaking object instantiation up into well defined stages.
Consider throwing away your current implementation entirely and
starting from scratch. Debugging a bad design will take longer and
cost more than throwing it away and starting again with the benefit
of hindsight.

Niall


On 16 Mar 2012 at 0:04, Adam Preble wrote:

> I edited the post so many times I forgot the subject line!  I am a bad
> person.
>
> Maybe the most condensed question here is: How do you normally manage
> multiple resources competing with the GIL in a way that could cause
> deadlocks with it?  For the details, see the long post.
>
> On Thu, Mar 15, 2012 at 11:59 PM, Adam Preble <[hidden email]> wrote:
>
> > I discovered recently that during callbacks to Python objects, sometimes
> > the interpreter would try to do stuff at the same time, despite the fact I
> > made a call to ensure the GIL.  I read the solution for that kind of thing
> > I'm doing is calling PyEval_InitThreads().  This worked at first, but like
> > more race conditions and stuff, all it takes is walking away from the
> > computer for it all to fall apart.
> >
> > What I'm seeing is a rather elaborate deadlock situation revolved around
> > securing the GIL.  I think I need to basically put my interpreter in its
> > own subsystem, design-wise, and ferret calls to invoke things in the
> > interpreter to it, in order to ultimately get around this.  However, what
> > I'm asking of the distribution is how they've gotten around this.
> >
> > To give something a little more concrete--this is pretty gross:
> > 1. Main thread starts interpreter and is running a script
> > 2. The script defines an implementation of callback interface A
> > 3. The script starts some objects that represent threads in the C++
> > runtime.  These are threads 1 and 2.
> > 4. The script starts to create an object that is wrapped from C++
> > 5. The object requires a resource from thread 1, where I use promises and
> > futures enqueue and fulfill the request when thread #1 isn't busy.
> > 6. Meanwhile, the interpreter thread is waiting for the resource since it
> > cannot put it off any further
> > 7. At this point, thread 2 tries to invoke the callback to interface A,
> > and it needs the interpreter thread.
> > 8. thread #1 needs thread #2 to complete this critical step before the
> > get() call will complete for the master interpreter thread
> > 9. Thread #2 needs thread #1 to finish so it can get the GIL.  It's
> > basically locked up in PyGILState_Ensure().
> >
> > Heck of a deadlock.  I am pondering having something else control the
> > interpreter in its own thread and have everybody enqueue stuff up to run in
> > it, like with the promises and futures I'm using elsewhere already.  The
> > reason is that thread #2 doesn't really need to steal the interpreter at
> > that very exact moment.  And furthermore, I'm trying to use Stackless, and
> > it's my understanding there I can link it up so that the interpreter gets
> > ahold of the Python runtime at controlled intervals--if desired--to invoke
> > other stuff.
> >
> >
> > _______________________________________________
> > Cplusplus-sig mailing list
> > [hidden email]
> > http://mail.python.org/mailman/listinfo/cplusplus-sig
> >
>


--
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/



_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Adam Preble
Well the current design does have a problem, but I think it's more to do with the Python side than the sum of the whole thing.  Most of my threads are basically subsystems in their own rights, and whatever that subsystem manage is encapsulated inside it.  I get into this little spats when I need to request a resource from one or more of them--not necessarily from one subsystem to another, just in my main script I have to create an object that involves a piece from one and a piece from another.  The subsystems will normally get to them when they are not in their own critical sections.  I just got bit when they both went through what is essentially an unprotected interpreter.

So I assume I would satisfy your suggestion of a master lock if I basically put the interpreter in its own subsystem.  Everybody then gets their turn at it.

Something I'm curious about has to do with the busy what in my promises and future implementation.  When I end up waiting on something, is there a way right now with Python to give up the GIL until the wait is done?  If I were to do a Release before the wait and an Ensure right after it, will there be inconsistent issues?  I ask this despite probably trying it tonight since these are the kind of issues that tend to bite me after the fact and not up front.

On Fri, Mar 16, 2012 at 10:38 AM, Niall Douglas <[hidden email]> wrote:
The key to avoiding deadlocks is, in every case, the appropriate
design.

In what you posted you appear to be doing too much at once, so you're
holding onto too many locks at once. Either replace those with a
single, master lock or try to never hold more than one lock at any
one time. That includes the GIL. In well designed code, one almost
NEVER holds more than two locks at any time.

Consider breaking object instantiation up into well defined stages.
Consider throwing away your current implementation entirely and
starting from scratch. Debugging a bad design will take longer and
cost more than throwing it away and starting again with the benefit
of hindsight.

Niall



_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Niall Douglas
On 16 Mar 2012 at 12:57, Adam Preble wrote:

> Well the current design does have a problem, but I think it's more to do
> with the Python side than the sum of the whole thing.

If by "Python side" you mean Boost.Python, then I agree: BPL has no
support for GIL management at all, and it really ought to. This was
one of the things that was discussed in the BPL v3 discussions on
this list a few months ago.

If by "Python side" you just mean Python, well it's basically one big
fat lock. There's nothing wrong with that design, indeed it's
extremely common.

> So I assume I would satisfy your suggestion of a master lock if I basically
> put the interpreter in its own subsystem.  Everybody then gets their turn
> at it.

You need to clarify what you mean by "own subsystem". Do you mean
"own process"?

> Something I'm curious about has to do with the busy what in my promises and
> future implementation.  When I end up waiting on something, is there a way
> right now with Python to give up the GIL until the wait is done?  If I were
> to do a Release before the wait and an Ensure right after it, will there be
> inconsistent issues?  I ask this despite probably trying it tonight since
> these are the kind of issues that tend to bite me after the fact and not up
> front.

You're going to have to be a lot clearer here. Do you mean you want
BPL to give up the GIL until the wait is done, or Python?

Regarding inconsistency over releases and locks, that's 101 threading
theory. Any basic threading programming textbook will tell you how to
handle such issues.

Niall

--
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/



_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Adam Preble


On Sat, Mar 17, 2012 at 1:44 PM, Niall Douglas <[hidden email]> wrote:
If by "Python side" you mean Boost.Python, then I agree: BPL has no
support for GIL management at all, and it really ought to. This was
one of the things that was discussed in the BPL v3 discussions on
this list a few months ago.


Hey do you know any terms or thread names where I could go digging through some of those discussions?  I'm just trying stuff superficially and finding some things that are at least interesting, but I'm not sure I got that stuff yet.
 
You need to clarify what you mean by "own subsystem". Do you mean
"own process"?


In the model I'm using, a subsystem would be a thread taking care of a particular resource.  In this case, I figured I would make the Python interpreter that resource, and install better guards around interacting with it.  For one thing, rather than anything else grabbing the GIL, they would enqueue stuff for it to execute.  That's about as far as I got with it in my head.  I managed to unjam the deadlock I was experiencing by eliminating the contention the two conflicting threads were having with each other.
 
You're going to have to be a lot clearer here. Do you mean you want
BPL to give up the GIL until the wait is done, or Python?


Something, somewhere.  I wasn't being to picky.  I wondered if there was a way to do it that I hadn't found with the regular Python headers. 
 
Regarding inconsistency over releases and locks, that's 101 threading
theory. Any basic threading programming textbook will tell you how to
handle such issues.

I'm hoping I'm asking about, say, 102 threading stuff instead. ;)  Specifically, I'm trying my darndest to keep explict lock controls outside of the main code because it really is hard to get right all the time.  Rather, I am trying to apply some kind of structure.  The subsystem model I did a terrible job of mentioning is an example, as well as using promises and futures.  If there are tricks, features, or something similar to keep in mind, that would steer how I approach this.


_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Jim Bosch-2
On 03/17/2012 11:20 PM, Adam Preble wrote:

>
>
> On Sat, Mar 17, 2012 at 1:44 PM, Niall Douglas
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     If by "Python side" you mean Boost.Python, then I agree: BPL has no
>     support for GIL management at all, and it really ought to. This was
>     one of the things that was discussed in the BPL v3 discussions on
>     this list a few months ago.
>
>
> Hey do you know any terms or thread names where I could go digging
> through some of those discussions?

Look for "[Boost.Python v3]" and "New Major-Release Boost.Python
Development" in the subject line.

We didn't go into a lot of depth on the threading, I'm afraid, as one of
the problems is that the guy starting the effort - me - doesn't actually
know much about threaded programming.  But I am hoping that I can design
things in such a way that someone like Niall could easily take it from
there.


Jim
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Stefan Seefeld-2
On 2012-03-18 11:54, Jim Bosch wrote:

> On 03/17/2012 11:20 PM, Adam Preble wrote:
>>
>>
>> On Sat, Mar 17, 2012 at 1:44 PM, Niall Douglas
>> <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>     If by "Python side" you mean Boost.Python, then I agree: BPL has no
>>     support for GIL management at all, and it really ought to. This was
>>     one of the things that was discussed in the BPL v3 discussions on
>>     this list a few months ago.
>>
>>
>> Hey do you know any terms or thread names where I could go digging
>> through some of those discussions?
>
> Look for "[Boost.Python v3]" and "New Major-Release Boost.Python
> Development" in the subject line.
>
> We didn't go into a lot of depth on the threading, I'm afraid, as one
> of the problems is that the guy starting the effort - me - doesn't
> actually know much about threaded programming.  But I am hoping that I
> can design things in such a way that someone like Niall could easily
> take it from there.

I recall seeing a discussion of locking policy being attached to
individual functions by means of the return-value and argument-passing
policy traits; in other words, something that's associated with
from-python and to-python conversion. I found that rather elegant. I'm
not sure whether anyone has any practical experience with that
technique, or whether it was just an idea worth exploring.

FWIW,
        Stefan

--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Niall Douglas
In reply to this post by Adam Preble
On 17 Mar 2012 at 22:20, Adam Preble wrote:

> > If by "Python side" you mean Boost.Python, then I agree: BPL has no
> > support for GIL management at all, and it really ought to. This was
> > one of the things that was discussed in the BPL v3 discussions on
> > this list a few months ago.
> >
> Hey do you know any terms or thread names where I could go digging through
> some of those discussions?  I'm just trying stuff superficially and finding
> some things that are at least interesting, but I'm not sure I got that
> stuff yet.

Try "[Boost.Python v3] Conversions and Registries", about October of
last year.

If you search right back to many years ago, I used to maintain a
patch which implemented GIL management for Boost.Python. It's long
bitrotted though.

> > You need to clarify what you mean by "own subsystem". Do you mean
> > "own process"?
> >
> In the model I'm using, a subsystem would be a thread taking care of a
> particular resource.  In this case, I figured I would make the Python
> interpreter that resource, and install better guards around interacting
> with it.  For one thing, rather than anything else grabbing the GIL, they
> would enqueue stuff for it to execute.  That's about as far as I got with
> it in my head.  I managed to unjam the deadlock I was experiencing by
> eliminating the contention the two conflicting threads were having with
> each other.

The only way to get two Python interpreters to run at once is to put
each into its own process. As it happens, IPC is usually fast enough
relative to the slowness of Python that often this actually works
very well.

> > You're going to have to be a lot clearer here. Do you mean you want
> > BPL to give up the GIL until the wait is done, or Python?
> >
> Something, somewhere.  I wasn't being to picky.  I wondered if there was a
> way to do it that I hadn't found with the regular Python headers.

Generally Python releases the GIL around anything it thinks might
wait. So long as you write your extension code to do the same, all
should be well.

> > Regarding inconsistency over releases and locks, that's 101 threading
> > theory. Any basic threading programming textbook will tell you how to
> > handle such issues.
>
> I'm hoping I'm asking about, say, 102 threading stuff instead. ;)
>  Specifically, I'm trying my darndest to keep explict lock controls outside
> of the main code because it really is hard to get right all the time.
>  Rather, I am trying to apply some kind of structure.  The subsystem model
> I did a terrible job of mentioning is an example, as well as using promises
> and futures.  If there are tricks, features, or something similar to keep
> in mind, that would steer how I approach this.

Y'see, I'd look at promises and futures primarily as a latency hiding
mechanism rather than for lock handling. I agree it would have been
super-sweet if Python had been much more thread aware in its design,
but we live with the hand we're dealt. Trying to bolt on futures and
promises to a system which wasn't designed for it is likely to be
self-defeating.

There is a school of thought that multithreading is only really
worthwhile doing in statically compiled languages. Anything
interpreted tends, generally speaking, to be too chunky to fine grain
holding locks to make threading useful for anything except i/o.
Certainly Python is extremely chunky, and I'd generally avoid using
threads in Python at all except as a way of portably doing
asynchronous i/o. Otherwise it's not worth it. If you're thinking it
is, it's time for a redesign.

Niall

--
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/



_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Niall Douglas
In reply to this post by Jim Bosch-2
On 18 Mar 2012 at 11:54, Jim Bosch wrote:

> We didn't go into a lot of depth on the threading, I'm afraid, as one of
> the problems is that the guy starting the effort - me - doesn't actually
> know much about threaded programming.  But I am hoping that I can design
> things in such a way that someone like Niall could easily take it from
> there.

I'm currently in early negotiations with a Silicon Valley crowd, part
of which involve doing some substantial improvements to Boost.Python
in this area. I generally don't believe anyone until I either see
money or a signed contract, so take this news with a pinch of salt,
but I hope that the list knows that I would love to work on this
topic if it were financially viable for me to do so.

Niall

--
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/



_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Managing the GIL across competing threads

Niall Douglas
In reply to this post by Stefan Seefeld-2
On 18 Mar 2012 at 12:05, Stefan Seefeld wrote:

> > We didn't go into a lot of depth on the threading, I'm afraid, as one
> > of the problems is that the guy starting the effort - me - doesn't
> > actually know much about threaded programming.  But I am hoping that I
> > can design things in such a way that someone like Niall could easily
> > take it from there.
>
> I recall seeing a discussion of locking policy being attached to
> individual functions by means of the return-value and argument-passing
> policy traits; in other words, something that's associated with
> from-python and to-python conversion. I found that rather elegant. I'm
> not sure whether anyone has any practical experience with that
> technique, or whether it was just an idea worth exploring.

Indeed, I had argued for a DLL and SO aware type registry to hold
default BPL<=>Python enter/exit policy (of which locking is an
example) which can be optionally overridden at the point of call. It
was from this and other reasons I had argued for a fused compile
time/runtime registry, and it was at that point that Dave came in to
argue against such a design for a series of good reasons.

I actually don't think myself and Dave disagreed, rather as usual he
sees the world one way and I see the world another, and we use
different vocabulary so we both think we're arguing when often we
agree. That said, I would be at pains to think twice or thrice about
an idea if Dave queries it, not least because it encourages me to
articulate myself much more clearly. He's also generally correct, or
has had an insight I have missed.

Actually, on the basis of that previous discussion I have raised
within ISO standards the point that we really ought to do something
about standardising shared libraries in ISO WG14. It's long overdue
and every attempt has failed so far. Yet as things stand, BPL is
simply broken in a multi-extension use context and there is no
standards-compliant way of fixing it. As the C language is the
primary interop for all other programming languages, and POSIX's
approach to shared libraries is broken, I fear we might just have to
go stomp on some eggshells to get this fixed.

That said, if done right we could seriously improve interop for all
languages. Imagine a standardised way of talking to Haskell from
Python via a modern C interop specification? Now that would be cool!

Niall

--
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/



_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Loading...