[C++-sig] Regarding thread compatibility of boost python

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[C++-sig] Regarding thread compatibility of boost python

Andrew Athan-3

Could someone please elaborate on what is and is not safe to do with
respect to threads and boost python?

My specific situation involves an embedded python interpreter within a
C++ app that uses multiple NPTL threads.

The Python module which is initially called creates a number of threads
within Python code.

Following initialization, calls tend to originate from C++ into Python,
and I can guarantee that these are serialized so that no more than one
thread is calling into Python at once.

However, Python may call back out to C++, and when doing so I need to
ensure thread safety of the C++ code.  However, such "backout" calls (to
coin a new term) never result in an intermediate callback into Python.  
I.e., C++->Python->C++ is possible, but not C++->Python->C++->Python.  
And, such "backout" calls are not long running.


Here are what I believe are the correct answers:

*  I assume that thread safety of the C++ code, when invoked from within
Python, is really a matter of the C++ code itself.  Python threads map
to NPTL threads, correct?  So, it's possible that my C++ code will be
invoked from a different thread than the one used to call into Python.  
Therefore, I should appropriately protect the C++ code using NPTL mutexes.

* Calls into Python should be wrapped with PyGILState_Ensure() and
PyGILState_Release()

* "backout" calls don't have to be protected, really, since they are not
long running, it's probably safe not to wrap them in
Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS



Despite the constraints I have described above, which seem (unless
someone corrects me) to relax the thread safety requirements, I am
looking into using the boost python patches in the Tn FOX library.  Can
anyone characterize these?  There is not much included in the sources by
way of documentation.  I have not yet looked into the actual Tn FOX
documentation package.

Thanks,
A.

_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Regarding thread compatibility of boost python

Niall Douglas
On 8 Mar 2006 at 1:29, Andrew Athan wrote:

> Despite the constraints I have described above, which seem (unless
> someone corrects me) to relax the thread safety requirements, I am
> looking into using the boost python patches in the Tn FOX library.  Can
> anyone characterize these?  There is not much included in the sources by
> way of documentation.  I have not yet looked into the actual Tn FOX
> documentation package.

I would pull BoostPatches.zip from the Python directory inside TnFOX
from SVN. I'd then diff the files to see what I did to patch things.

Some of the patching is merging in TnFOX features eg; exception
handling. But much of it is the threading stuff. Of course, TnFOX has
its own threading system so if you don't want to use TnFOX, you'll
need to replace all that with your own.

I should add that TnFOX can be built as a non-GUI library and thus
just as a reasonably small portability library. You may find it
easier to simply link against TnFOX generated via this method. I
should add that the python bindings currently don't respect the non-
GUI build just yet (it's coming). The TnFOX python bindings include a
full copy of Boost.Python.

Cheers,
Niall



_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig