How do I make correct wrappers to interfaces while using shared_ptrs?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

How do I make correct wrappers to interfaces while using shared_ptrs?

Adam Preble
I am trying to run this on Win32, using Stackless Python 2.6, and Boost 1.47.

I am having a problem with passing a shared_ptr of a C++ object implementing a C++ interface to a wrapper of a Python object implementing a C++ interface.  That's a mouthful, so I thought I'd simplify the code.  The thing is, I can't seem to get the wrapping correct.  If I can get this going, I can start adding complication to it and see what the problem really is.

Basically, I have an interface that works with itself for communicating stuff.  There is a C implementation of it, and a Python implementation of it.  I have to write a wrapper.  The Python implementation can communicate with instances of the C implementation, but not vice versa.  When the C implementation calls the Python one, I get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: No to_python (by-value) converter found for C++ type: class boost::shared_ptr<class Communicatable>

The magic call is in the python script.  

Python script
http://pastebin.com/DJ65tKEW

Header

Source (Boost.Python declarations here)

I have done similar wrappers like this before so I'm just befuddled about why this time it hates me.  The shared_ptrs are part of the larger problem I'm trying to reproduce, so I want to incorporate them into this test program.  In the larger program, while doing a get_override() call on a wrapper to an interface, I get a segfault inside the Python DLLs while it tries to convert the shared_ptr.  The last line I can see i shared_ptr_to_python.hpp:

        return converter::registered<shared_ptr<T> const&>::converters.to_python(&x);

That's what I am eventually hoping to reproduce by shooting back and forth all this stuff.  




_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: How do I make correct wrappers to interfaces while using shared_ptrs?

Jim Bosch-2
On 02/04/2012 11:11 AM, Adam Preble wrote:

> I am trying to run this on Win32, using Stackless Python 2.6, and Boost
> 1.47.
>
> I am having a problem with passing a shared_ptr of a C++ object
> implementing a C++ interface to a wrapper of a Python object
> implementing a C++ interface.  That's a mouthful, so I thought I'd
> simplify the code.  The thing is, I can't seem to get the wrapping
> correct.  If I can get this going, I can start adding complication to it
> and see what the problem really is.
>
> Basically, I have an interface that works with itself for communicating
> stuff.  There is a C implementation of it, and a Python implementation
> of it.  I have to write a wrapper.  The Python implementation can
> communicate with instances of the C implementation, but not vice versa.
>   When the C implementation calls the Python one, I get:
>
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> TypeError: No to_python (by-value) converter found for C++ type: class
> boost::shared_ptr<class Communicatable>
>
> The magic call is in the python script.
>
> Python script
> http://pastebin.com/DJ65tKEW
>
> Header
> http://pastebin.com/6BX8e6KA
>
> Source (Boost.Python declarations here)
> http://pastebin.com/s5YC1hyC
>
> I have done similar wrappers like this before so I'm just befuddled
> about why this time it hates me.  The shared_ptrs are part of the larger
> problem I'm trying to reproduce, so I want to incorporate them into this
> test program.  In the larger program, while doing a get_override() call
> on a wrapper to an interface, I get a segfault inside the Python DLLs
> while it tries to convert the shared_ptr.  The last line I can see i
> shared_ptr_to_python.hpp:
>
>          return converter::registered<shared_ptr<T>
> const&>::converters.to_python(&x);
>
> That's what I am eventually hoping to reproduce by shooting back and
> forth all this stuff.
>

If I understand your intend correctly, I think your class_ definition
needs to be:

class_<Communicable,shared_ptr<CommunicableWrapper>,noncopyable>

(right now you have class_<CommunicableWrapper,...>)



Jim
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: How do I make correct wrappers to interfaces while using shared_ptrs?

Adam Preble


On Sat, Feb 4, 2012 at 10:46 AM, Jim Bosch <[hidden email]> wrote:
If I understand your intend correctly, I think your class_ definition needs to be:

class_<Communicable,shared_ptr<CommunicableWrapper>,noncopyable>

(right now you have class_<CommunicableWrapper,...>)



Something like this?

class_<Communicatable, boost::shared_ptr<CommunicatableWrapper>, boost::noncopyable>("Communicatable")

I get burnt by the compiler when I try that:

d:\coding\boost_1_47_0\boost\python\object\pointer_holder.hpp(217): error C2664: 'CommunicatableWrapper::CommunicatableWrapper(const CommunicatableWrapper &)' : cannot convert parameter 1 from 'PyObject *' to 'const CommunicatableWrapper &'
          Reason: cannot convert from 'PyObject *' to 'const CommunicatableWrapper'
          No constructor could take the source type, or constructor overload resolution was ambiguous
 
It sounds like if I had a constructor that would take a PyObject *, that would shut it up, but I have no idea what to make of such a thing.

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: How do I make correct wrappers to interfaces while using shared_ptrs?

Jim Bosch-2
On 02/04/2012 08:23 PM, Adam Preble wrote:

>
>
> On Sat, Feb 4, 2012 at 10:46 AM, Jim Bosch <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     If I understand your intend correctly, I think your class_
>     definition needs to be:
>
>     class_<Communicable,shared___ptr<CommunicableWrapper>,__noncopyable>
>
>     (right now you have class_<CommunicableWrapper,...__>)
>
>
>
> Something like this?
>
> class_<Communicatable, boost::shared_ptr<CommunicatableWrapper>,
> boost::noncopyable>("Communicatable")
>
> I get burnt by the compiler when I try that:
>
> d:\coding\boost_1_47_0\boost\python\object\pointer_holder.hpp(217):
> error C2664: 'CommunicatableWrapper::CommunicatableWrapper(const
> CommunicatableWrapper &)' : cannot convert parameter 1 from 'PyObject *'
> to 'const CommunicatableWrapper &'
>            Reason: cannot convert from 'PyObject *' to 'const
> CommunicatableWrapper'
>            No constructor could take the source type, or constructor
> overload resolution was ambiguous
> It sounds like if I had a constructor that would take a PyObject *, that
> would shut it up, but I have no idea what to make of such a thing.
>

Oh, you're right.  I was confusing the manual way of doing Python-side
polymorphism (derived-class holders) with the more automatic (and
better) way you're doing it (with wrapper).  Your original code was fine
in that regard.

Anyhow, looking closer, here's what you need to do:

  - Put your code back to what it was before I told you to change it.

  - Add another line, after the class wrappers:

register_ptr_to_python< boost::shared_ptr<Communicatable> >();

That explicitly registers a shared_ptr converter for Communicatable.
I'm not sure why Boost.Python doesn't automatically do that when you
register converters for its derived classes (those are registered by
putting the shared_pr types in the template parameters to class_).  It
might just be an oversight, or it might be that doing the base class
registration automatically would cause problems in some contexts.


Jim

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: How do I make correct wrappers to interfaces while using shared_ptrs?

Adam Preble

On Sat, Feb 4, 2012 at 8:48 PM, Jim Bosch <[hidden email]> wrote:
Oh, you're right.  I was confusing the manual way of doing Python-side polymorphism (derived-class holders) with the more automatic (and better) way you're doing it (with wrapper).  Your original code was fine in that regard.

Anyhow, looking closer, here's what you need to do:

 - Put your code back to what it was before I told you to change it.

 - Add another line, after the class wrappers:

register_ptr_to_python< boost::shared_ptr<Communicatable> >();

That explicitly registers a shared_ptr converter for Communicatable. I'm not sure why Boost.Python doesn't automatically do that when you register converters for its derived classes (those are registered by putting the shared_pr types in the template parameters to class_).  It might just be an oversight, or it might be that doing the base class registration automatically would cause problems in some contexts.


The appears to do the trick.  I'm wondering--how do you figure out stuff like that?  That isn't the kind of thing that could have just occurred to me.  

This little test program works, but so far I haven't resolved the crash in my more complicated code that's coming a Python subclass of a C class trying to get a callback from Python instance of a C class . . . it's rough even explaining it.  I'll have to review my code and make sure I'm doing that in all the right places.  Either I get it working because of these little things here, or I'll be back with a modification of this source to show the problem with shared_ptr conversion that I was seeing.

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: How do I make correct wrappers to interfaces while using shared_ptrs?

Jim Bosch-2
On 02/05/2012 12:56 AM, Adam Preble wrote:

>
> On Sat, Feb 4, 2012 at 8:48 PM, Jim Bosch <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Oh, you're right.  I was confusing the manual way of doing
>     Python-side polymorphism (derived-class holders) with the more
>     automatic (and better) way you're doing it (with wrapper).  Your
>     original code was fine in that regard.
>
>     Anyhow, looking closer, here's what you need to do:
>
>       - Put your code back to what it was before I told you to change it.
>
>       - Add another line, after the class wrappers:
>
>     register_ptr_to_python< boost::shared_ptr<__Communicatable> >();
>
>     That explicitly registers a shared_ptr converter for Communicatable.
>     I'm not sure why Boost.Python doesn't automatically do that when you
>     register converters for its derived classes (those are registered by
>     putting the shared_pr types in the template parameters to class_).
>       It might just be an oversight, or it might be that doing the base
>     class registration automatically would cause problems in some contexts.
>
>
> The appears to do the trick.  I'm wondering--how do you figure out stuff
> like that?  That isn't the kind of thing that could have just occurred
> to me.
>

To be honest, it's probably just a result of having stared at the
Boost.Python internals a lot while writing extensions for it.  Ideally,
that's not something every user would have to do - it's a great library,
but the tutorial documentation does not go very deep, and the reference
docs are only good if you know what you're looking for.  So just keep
mailing the list :)

Another great way to learn more advance Boost.Python usage is to run
Py++.  Even if you don't use it to produce your final wrappers (I
generally don't), looking at the output can teach you a lot of new tricks.

The important bit of knowledge in this case is that putting a shared_ptr
type in the template args to class_ does more than just register a
to-python converter for shared_ptrs.  It also causes new instances
created in Python to be held in a shared_ptr.  That's necessary in your
case because you're using enable_shared_from_this, but usually it isn't
if you just want to be able to return things by shared_ptr.  So I
usually use register_ptr_to_python on all my classes instead, and that's
why I was already familiar with it.

> This little test program works, but so far I haven't resolved the crash
> in my more complicated code that's coming a Python subclass of a C class
> trying to get a callback from Python instance of a C class . . . it's
> rough even explaining it.  I'll have to review my code and make sure I'm
> doing that in all the right places.  Either I get it working because of
> these little things here, or I'll be back with a modification of this
> source to show the problem with shared_ptr conversion that I was seeing.
>

Good luck!

Jim

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: How do I make correct wrappers to interfaces while using shared_ptrs?

Adam Preble


On Feb 5, 2012 10:15 AM, "Jim Bosch" <[hidden email]> wrote:
>
> On 02/05/2012 12:56 AM, Adam Preble wrote:
>>
>>
>> On Sat, Feb 4, 2012 at 8:48 PM, Jim Bosch <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>    Oh, you're right.  I was confusing the manual way of doing
>>    Python-side polymorphism (derived-class holders) with the more
>>    automatic (and better) way you're doing it (with wrapper).  Your
>>    original code was fine in that regard.
>>
>>    Anyhow, looking closer, here's what you need to do:
>>
>>      - Put your code back to what it was before I told you to change it.
>>
>>      - Add another line, after the class wrappers:
>>
>>    register_ptr_to_python< boost::shared_ptr<__Communicatable> >();
>>
>>
>>    That explicitly registers a shared_ptr converter for Communicatable.
>>    I'm not sure why Boost.Python doesn't automatically do that when you
>>    register converters for its derived classes (those are registered by
>>    putting the shared_pr types in the template parameters to class_).
>>      It might just be an oversight, or it might be that doing the base
>>    class registration automatically would cause problems in some contexts.
>>
>>
>> The appears to do the trick.  I'm wondering--how do you figure out stuff
>> like that?  That isn't the kind of thing that could have just occurred
>> to me.
>>
>
> To be honest, it's probably just a result of having stared at the Boost.Python internals a lot while writing extensions for it.  Ideally, that's not something every user would have to do - it's a great library, but the tutorial documentation does not go very deep, and the reference docs are only good if you know what you're looking for.  So just keep mailing the list :)
>
> Another great way to learn more advance Boost.Python usage is to run Py++.  Even if you don't use it to produce your final wrappers (I generally don't), looking at the output can teach you a lot of new tricks.
>

I will have to look at it in the next few days, although I fear it may be an adventure on my real project.

> The important bit of knowledge in this case is that putting a shared_ptr type in the template args to class_ does more than just register a to-python converter for shared_ptrs.  It also causes new instances created in Python to be held in a shared_ptr.  That's necessary in your case because you're using enable_shared_from_this, but usually it isn't if you just want to be able to return things by shared_ptr.  So I usually use register_ptr_to_python on all my classes instead, and that's why I was already familiar with it.

Jim,

Would using register_ptr_to_python explicitly and consistently instead of the online shared_ptr declaration potentially eliminate some side-effects?  I cannot yet isolate why my real app crashes in python when it tries to do an eval into one of my wrappers.


_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: How do I make correct wrappers to interfaces while using shared_ptrs?

Jim Bosch-2
On 02/05/2012 07:49 PM, Adam Preble wrote:

> Would using register_ptr_to_python explicitly and consistently instead of
> the online shared_ptr declaration potentially eliminate some side-effects?
> I cannot yet isolate why my real app crashes in python when it tries to do
> an eval into one of my wrappers.

Overall, I doubt it.  Usually putting shared_ptr in the class_ template
params would allow you to do more fancy shared_ptr tricks, not less.
And as I said before, it might be necessary if you use
enable_shared_from_this.

What your previous example showed, though, was that it might be useful
to use register_ptr_to_python on every type you ever make a shared_ptr
to in C++ *in addition* to putting shared_ptr in the class_ templates -
you'll get some warnings in Python about redundant converters, and those
will tell you which of those extra declarations it's safe to remove.
Any that remain might help fix your problem.

Jim
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig