weak_ptr to this in C++ classes extended from python

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

weak_ptr to this in C++ classes extended from python

Holger Brandsmeier
Dear list,

how is it possible to have a class in C++ that can be extended from
python and that stores a weak_ptr to itself?

The reason why I want to do this:
 - using a weak_ptr to itself does not introduce a memory leak.
 - from a C++-member of that class I can call other functions that
expect a shared_ptr<>, in that case I convert the weak_ptr to a
shared_ptr and all the memory management works nicely.

The way I would suspect this to work is:
struct I {
  void setThisRCP(boost::shared_ptr<I> ptr) {
    thisRCP = ptr;
  }

  boost::shared_ptr<I> getThisRCP() const {
    return thisRCP.lock();
  }

  boost::weak_ptr<I> thisRCP;
};

Then I export this class to C++ and in python I would do:
class IDer(I):
  def __init__(self):
    I.__init__(self)
    self.setThisRCP(self)

And I would suspect that I can use that class now in C++ and in
particular calling getThisRCP() from C++ returns a valid shared
pointer. However, immediately after the call of
`self.setThisRCP(self)` the weak_ptr is already invalid and getThisRCP
returns an shared pointer to Null (None in python).

Attached to this message is a very simple example that demonstrates
how this failes for the above implementation, the ouptu for the
snipped:

print '\ntestcase for thisRCP as from python extended class:'
i = IDer()
assert i.getThisRCP() is not None

fails the assertion. However if I do the equivalent to IDer purely in
C++ (which is not really an option) then everything works (see
attachment) with the putput
testcase for thisRCP as set from C++:
42
destructor for J called ...

I suspect that the error comes from the fact that boost python create
as temporary shared_ptr that is passes to setThisRCP() which is not
the shared pointer that actually holds the instance of IDer in python
(is it actually hold as a shared_ptr?).

Do you have any proposal of how I can implement the above pattern with
boost python? Or is this completly impossible at the moment?

I also reported this issue before "storing weak_ptr to this in classes
extended from python" but at that time I wasn't using shared_ptr and
now I really suffer from the memory leak that was introduced by my
workaround which was to store the thisRCP as a "shared_ptr" and I
really need to fix that now. So here is a small little example that
does not depend on any other parts of my library.

-Holger

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig

thisRCPPy.cpp (2K) Download Attachment
test_rcp.py (524 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Jim Bosch-2
On 04/22/2012 03:42 PM, Holger Brandsmeier wrote:

> Dear list,
>
> how is it possible to have a class in C++ that can be extended from
> python and that stores a weak_ptr to itself?
>
> The reason why I want to do this:
>   - using a weak_ptr to itself does not introduce a memory leak.
>   - from a C++-member of that class I can call other functions that
> expect a shared_ptr<>, in that case I convert the weak_ptr to a
> shared_ptr and all the memory management works nicely.
>
> The way I would suspect this to work is:
> struct I {
>    void setThisRCP(boost::shared_ptr<I>  ptr) {
>      thisRCP = ptr;
>    }
>
>    boost::shared_ptr<I>  getThisRCP() const {
>      return thisRCP.lock();
>    }
>
>    boost::weak_ptr<I>  thisRCP;
> };
>
> Then I export this class to C++ and in python I would do:
> class IDer(I):
>    def __init__(self):
>      I.__init__(self)
>      self.setThisRCP(self)
>
> And I would suspect that I can use that class now in C++ and in
> particular calling getThisRCP() from C++ returns a valid shared
> pointer. However, immediately after the call of
> `self.setThisRCP(self)` the weak_ptr is already invalid and getThisRCP
> returns an shared pointer to Null (None in python).
>
> Attached to this message is a very simple example that demonstrates
> how this failes for the above implementation, the ouptu for the
> snipped:
>
> print '\ntestcase for thisRCP as from python extended class:'
> i = IDer()
> assert i.getThisRCP() is not None
>
> fails the assertion. However if I do the equivalent to IDer purely in
> C++ (which is not really an option) then everything works (see
> attachment) with the putput
> testcase for thisRCP as set from C++:
> 42
> destructor for J called ...
>
> I suspect that the error comes from the fact that boost python create
> as temporary shared_ptr that is passes to setThisRCP() which is not
> the shared pointer that actually holds the instance of IDer in python
> (is it actually hold as a shared_ptr?).
>
> Do you have any proposal of how I can implement the above pattern with
> boost python? Or is this completly impossible at the moment?
>

I agree with your suspicion about what's going on here, though I haven't
(yet) tested your example.  One quick thing to try would be having
setThisRCP() take a shared_ptr by reference - that should force
Boost.Python to try to return the internal shared_ptr rather than create
a temporary one.

Even if it fails, you may learn something about what's going on.


> I also reported this issue before "storing weak_ptr to this in classes
> extended from python" but at that time I wasn't using shared_ptr and
> now I really suffer from the memory leak that was introduced by my
> workaround which was to store the thisRCP as a "shared_ptr" and I
> really need to fix that now. So here is a small little example that
> does not depend on any other parts of my library.
>

Thanks for the small example.  Hopefully we can get to the bottom of
this; you don't seem to be the only person who wants to do something
like this.

Jim

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Dave Abrahams
In reply to this post by Holger Brandsmeier

on Sun Apr 22 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:

> Dear list,
>
> how is it possible to have a class in C++ that can be extended from
> python and that stores a weak_ptr to itself?

Easy, I think: just like any other class you wrap.  Are you aware of
boost::enable_shared_from_this?  It does all of that internally.

> The reason why I want to do this:
>  - using a weak_ptr to itself does not introduce a memory leak.
>  - from a C++-member of that class I can call other functions that
> expect a shared_ptr<>, in that case I convert the weak_ptr to a
> shared_ptr and all the memory management works nicely.
>
> The way I would suspect this to work is:
> struct I {
>   void setThisRCP(boost::shared_ptr<I> ptr) {
>     thisRCP = ptr;
>   }
>
>   boost::shared_ptr<I> getThisRCP() const {
>     return thisRCP.lock();
>   }
>
>   boost::weak_ptr<I> thisRCP;
> };
>
> Then I export this class to C++ and in python I would do:
> class IDer(I):
>   def __init__(self):
>     I.__init__(self)
>     self.setThisRCP(self)
>
> And I would suspect that I can use that class now in C++ and in
> particular calling getThisRCP() from C++ returns a valid shared
> pointer. However, immediately after the call of
> `self.setThisRCP(self)` the weak_ptr is already invalid and getThisRCP
> returns an shared pointer to Null (None in python).

If you use boost::shared_ptr<I> as the Holder for your wrapper, you'll
be able to do that conversion.  I see you've done that with "J" in your
example, but not with "I".  Why not?

If you use enable_shared_from_this, the internal weak_ptr will be
initialized from the first shared_ptr that is constructed to manage the
object.  Everything should work.

--
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Holger Brandsmeier
Dear Dave, dear Jim,

thanks for your help in particular to `boost::enable_shared_from_this`
which I wasn't aware about in the beginning. Unfortunately your
solutions did not yet work.

In the meantime I figured out a different way of doing the shared
pointer from this in python. Namely, to just make getThisRCP() a
virtual function, add it to the Wrapper and to implement it in the
python by implementing it as

def getThisRCP(self):
  return self

this simply works.

As for the issue that I reported, it still persists so out of interest
I did try your suggestions and here is what I found out.

the difference between the lines
  class_<IWrapper, boost::noncopyable >("I", init<>() )
and
  class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable
>("I", init<>() )
didn't change anything, the getThisRCP() still doesn't work. In fact I
tried both variants before and send you just the first one. Can it be
that specifying the holder for IWrapper does not apply to the holder
for the subclass IDer from python?

In fact the statement
  class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable
>("I", init<>() )
has an issue. When I call getThisRCP() later, and the return value is
not a null shared pointer, then I in fact the error
TypeError: No to_python (by-value) converter found for C++ type:
boost::shared_ptr<I>
which is quite annoying. I also implemented Dave's suggestion with
boost::enable_shared_from_this and because of the above issue I used
this for the Wrapper and not the class itself (see K and KWrapper in
the attachment).

The behaviour of `enable_shared_from_this` is different from my
initual getThisRCP() / setThisRCP() solution but still has seriour
problems. Concider the code:

class KDer(K):
  def __init__(self):
    K.__init__(self)

k = KDer()

In contrast to my initial solution the lines
  assert k.getThisRCP2() is not None
  k2 = k.getThisRCP2()
now work, which is nice.

Also this call works now:
  print k2.foo();

However, the following two lines
  del k
  print k2.foo();
give a segmentation fault. Note that the destructor for the class `K`
has not been called, which  would have printed a statement. The `gdb`
backtrace is

#0  0x00007ffff64425d7 in boost::python::objects::instance_dealloc
(inst=0xe51788) at libs/python/src/object/class.cpp:335
#1  0x00007ffff7aba7f6 in subtype_dealloc (self=0xe51788) at
Objects/typeobject.c:1014
#2  0x00007ffff7a72413 in instancemethod_dealloc (im=0xae0730) at
Objects/classobject.c:2364
#3  0x00007ffff645387e in xdecref<_object> (p=<optimized out>) at
./boost/python/refcount.hpp:36
#4  ~handle (this=<optimized out>, __in_chrg=<optimized out>) at
./boost/python/handle.hpp:211
#5  boost::python::detail::wrapper_base::get_override (this=0xaec988,
name=0x7fffe41de992 "foo", class_object=0x71bb10)
    at libs/python/src/wrapper.cpp:21
#6  0x00007fffe41d95f1 in get_override (name=<optimized out>,
this=<optimized out>, name=<optimized out>)
    at /usr/include/boost/python/wrapper.hpp:29
#7  KWrapper::foo (this=0xaec980) at
/home/bholger/HolgerBrandsmeier/parfem.h/python/thisRCPPy.cpp:112
[...]

Any ideas?

-Holger

On Mon, Apr 23, 2012 at 18:01, Dave Abrahams <[hidden email]> wrote:

>
> on Sun Apr 22 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
>
>> Dear list,
>>
>> how is it possible to have a class in C++ that can be extended from
>> python and that stores a weak_ptr to itself?
>
> Easy, I think: just like any other class you wrap.  Are you aware of
> boost::enable_shared_from_this?  It does all of that internally.
>
>> The reason why I want to do this:
>>  - using a weak_ptr to itself does not introduce a memory leak.
>>  - from a C++-member of that class I can call other functions that
>> expect a shared_ptr<>, in that case I convert the weak_ptr to a
>> shared_ptr and all the memory management works nicely.
>>
>> The way I would suspect this to work is:
>> struct I {
>>   void setThisRCP(boost::shared_ptr<I> ptr) {
>>     thisRCP = ptr;
>>   }
>>
>>   boost::shared_ptr<I> getThisRCP() const {
>>     return thisRCP.lock();
>>   }
>>
>>   boost::weak_ptr<I> thisRCP;
>> };
>>
>> Then I export this class to C++ and in python I would do:
>> class IDer(I):
>>   def __init__(self):
>>     I.__init__(self)
>>     self.setThisRCP(self)
>>
>> And I would suspect that I can use that class now in C++ and in
>> particular calling getThisRCP() from C++ returns a valid shared
>> pointer. However, immediately after the call of
>> `self.setThisRCP(self)` the weak_ptr is already invalid and getThisRCP
>> returns an shared pointer to Null (None in python).
>
> If you use boost::shared_ptr<I> as the Holder for your wrapper, you'll
> be able to do that conversion.  I see you've done that with "J" in your
> example, but not with "I".  Why not?
>
> If you use enable_shared_from_this, the internal weak_ptr will be
> initialized from the first shared_ptr that is constructed to manage the
> object.  Everything should work.
>
> --
> Dave Abrahams
> BoostPro Computing
> http://www.boostpro.com
>
> _______________________________________________
> Cplusplus-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/cplusplus-sig

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig

test_rcp.py (1K) Download Attachment
thisRCPPy.cpp (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Dave Abrahams

on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:

> In fact the statement
>   class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable
>>("I", init<>() )
> has an issue. When I call getThisRCP() later, and the return value is
> not a null shared pointer, then I in fact the error
> TypeError: No to_python (by-value) converter found for C++ type:
> boost::shared_ptr<I>
> which is quite annoying. I also implemented Dave's suggestion with
> boost::enable_shared_from_this and because of the above issue I used
> this for the Wrapper and not the class itself (see K and KWrapper in
> the attachment).
>
> The behaviour of `enable_shared_from_this` is different from my
> initual getThisRCP() / setThisRCP() solution but still has seriour
> problems. Concider the code:
>
> class KDer(K):
>   def __init__(self):
>     K.__init__(self)
>
> k = KDer()
>
> In contrast to my initial solution the lines
>   assert k.getThisRCP2() is not None
>   k2 = k.getThisRCP2()
> now work, which is nice.
>
> Also this call works now:
>   print k2.foo();
>
> However, the following two lines
>   del k
>   print k2.foo();
> give a segmentation fault. Note that the destructor for the class `K`
> has not been called, which  would have printed a statement.

Please post a reduced example without any of the "J" business that
illustrates exactly the problem you're having for this one particular
case.  If doing so doesn't immediately reveal the problem to you, I'm
sure we can get to the bottom of it here.


--
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Holger Brandsmeier
Dave,

Here is the reduced example with only the
`boost::enable_shared_from_this` variant. The issues are all as
reported before.

-Holger

On Mon, Apr 23, 2012 at 22:25, Dave Abrahams <[hidden email]> wrote:

>
> on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
>
>> In fact the statement
>>   class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable
>>>("I", init<>() )
>> has an issue. When I call getThisRCP() later, and the return value is
>> not a null shared pointer, then I in fact the error
>> TypeError: No to_python (by-value) converter found for C++ type:
>> boost::shared_ptr<I>
>> which is quite annoying. I also implemented Dave's suggestion with
>> boost::enable_shared_from_this and because of the above issue I used
>> this for the Wrapper and not the class itself (see K and KWrapper in
>> the attachment).
>>
>> The behaviour of `enable_shared_from_this` is different from my
>> initual getThisRCP() / setThisRCP() solution but still has seriour
>> problems. Concider the code:
>>
>> class KDer(K):
>>   def __init__(self):
>>     K.__init__(self)
>>
>> k = KDer()
>>
>> In contrast to my initial solution the lines
>>   assert k.getThisRCP2() is not None
>>   k2 = k.getThisRCP2()
>> now work, which is nice.
>>
>> Also this call works now:
>>   print k2.foo();
>>
>> However, the following two lines
>>   del k
>>   print k2.foo();
>> give a segmentation fault. Note that the destructor for the class `K`
>> has not been called, which  would have printed a statement.
>
> Please post a reduced example without any of the "J" business that
> illustrates exactly the problem you're having for this one particular
> case.  If doing so doesn't immediately reveal the problem to you, I'm
> sure we can get to the bottom of it here.
>
>
> --
> Dave Abrahams
> BoostPro Computing
> http://www.boostpro.com
>
> _______________________________________________
> Cplusplus-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/cplusplus-sig

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig

test_rcp.py (598 bytes) Download Attachment
thisRCPPy.cpp (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Dave Abrahams

I don't understand; all the enable_shared_from_this stuff is commented
out.  Can you please post the complete, correct example that you tried?

on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:

> Dave,
>
> Here is the reduced example with only the
> `boost::enable_shared_from_this` variant. The issues are all as
> reported before.
>
> -Holger
>
> On Mon, Apr 23, 2012 at 22:25, Dave Abrahams <[hidden email]> wrote:
>>
>> on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
>>
>>> In fact the statement
>>>   class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable
>>>>("I", init<>() )
>>> has an issue. When I call getThisRCP() later, and the return value is
>>> not a null shared pointer, then I in fact the error
>>> TypeError: No to_python (by-value) converter found for C++ type:
>>> boost::shared_ptr<I>
>>> which is quite annoying. I also implemented Dave's suggestion with
>>> boost::enable_shared_from_this and because of the above issue I used
>>> this for the Wrapper and not the class itself (see K and KWrapper in
>>> the attachment).
>>>
>>> The behaviour of `enable_shared_from_this` is different from my
>>> initual getThisRCP() / setThisRCP() solution but still has seriour
>>> problems. Concider the code:
>>>
>>> class KDer(K):
>>>   def __init__(self):
>>>     K.__init__(self)
>>>
>>> k = KDer()
>>>
>>> In contrast to my initial solution the lines
>>>   assert k.getThisRCP2() is not None
>>>   k2 = k.getThisRCP2()
>>> now work, which is nice.
>>>
>>> Also this call works now:
>>>   print k2.foo();
>>>
>>> However, the following two lines
>>>   del k
>>>   print k2.foo();
>>> give a segmentation fault. Note that the destructor for the class `K`
>>> has not been called, which  would have printed a statement.
>>
>> Please post a reduced example without any of the "J" business that
>> illustrates exactly the problem you're having for this one particular
>> case.  If doing so doesn't immediately reveal the problem to you, I'm
>> sure we can get to the bottom of it here.
>>
>>
>> --
>> Dave Abrahams
>> BoostPro Computing
>> http://www.boostpro.com
>>
>> _______________________________________________
>> Cplusplus-sig mailing list
>> [hidden email]
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
> from thisRCPPy import *
>
> print '\ntestcase for thisRCP as from python extended class (using enable_shared_from_this):'
> class KDer(K):
>   def __init__(self):
>     K.__init__(self)
>
> k = KDer()
> assert k.getThisRCP2() is not None
> k2 = k.getThisRCP2()
> if 1:
>   print k2.foo();
>   del k
>   print 'with the `del k` statement the following statement will crash the program:';
>   print k2.foo();
>   print 'this line is never reached';
>   #del k
>   del k2
>
>
> #include <boost/python.hpp>
> #include "iostream"
> #include "string"
> #include "boost/weak_ptr.hpp"
> #include "boost/enable_shared_from_this.hpp"
>
> using namespace boost::python;
> using namespace std;
> //using Teuchos::RCP;
>
> struct K
> // : public boost::enable_shared_from_this<K>
> {
>   virtual int foo() = 0;
>
>   virtual ~K() {
>     std::cout << "destructor for K called ..." << std::endl;
>   }
>
>   /*boost::shared_ptr<K> getThisRCP() {
>     return shared_from_this();
>   }*/
> };
>
> struct KWrapper : public K, public boost::python::wrapper<K>,
>   public boost::enable_shared_from_this<KWrapper>
> {
>
>   KWrapper() {
>
>   }
>
>   virtual int foo() {
>     if( override f = this->get_override("foo") ) {
>       return f();
>     } else {
>       return base_foo();
>     }
>   }
>
>   boost::shared_ptr<KWrapper> getThisRCP2() {
>     return shared_from_this();
>   }
>
>   int base_foo() {
>     return 0;
>   }
> };
>
> BOOST_PYTHON_MODULE(thisRCPPy)
> {
>   {
>     typedef K ClassT;
>     class_<KWrapper, boost::shared_ptr<KWrapper>, boost::noncopyable >("K", init<>() )
>       //.def("getThisRCP", &ClassT::getThisRCP)
>       .def("getThisRCP2", &KWrapper::getThisRCP2)
>       .def("foo", &ClassT::foo)
>       ;
>   }
> }
> _______________________________________________
> Cplusplus-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/cplusplus-sig

--
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Holger Brandsmeier
Not all of it is commented out, it is enabled for the wrapper. You can
try to enable it for the class itself, but then I get a different
error, namely:

> In fact the statement
>  class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable>("I", init<>() )
> has an issue. When I call getThisRCP() later, and the return value is
> not a null shared pointer, then I in fact the error
> TypeError: No to_python (by-value) converter found for C++ type: boost::shared_ptr<I>
> which is quite annoying. I also implemented Dave's suggestion with
> boost::enable_shared_from_this and because of the above issue I used
> this for the Wrapper and not the class itself (see K and KWrapper in
> the attachment).

-Holger

On Wed, Apr 25, 2012 at 15:32, Dave Abrahams <[hidden email]> wrote:

>
> I don't understand; all the enable_shared_from_this stuff is commented
> out.  Can you please post the complete, correct example that you tried?
>
> on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
>
>> Dave,
>>
>> Here is the reduced example with only the
>> `boost::enable_shared_from_this` variant. The issues are all as
>> reported before.
>>
>> -Holger
>>
>> On Mon, Apr 23, 2012 at 22:25, Dave Abrahams <[hidden email]> wrote:
>>>
>>> on Mon Apr 23 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:
>>>
>>>> In fact the statement
>>>>   class_<IWrapper, boost::shared_ptr<IWrapper>, boost::noncopyable
>>>>>("I", init<>() )
>>>> has an issue. When I call getThisRCP() later, and the return value is
>>>> not a null shared pointer, then I in fact the error
>>>> TypeError: No to_python (by-value) converter found for C++ type:
>>>> boost::shared_ptr<I>
>>>> which is quite annoying. I also implemented Dave's suggestion with
>>>> boost::enable_shared_from_this and because of the above issue I used
>>>> this for the Wrapper and not the class itself (see K and KWrapper in
>>>> the attachment).
>>>>
>>>> The behaviour of `enable_shared_from_this` is different from my
>>>> initual getThisRCP() / setThisRCP() solution but still has seriour
>>>> problems. Concider the code:
>>>>
>>>> class KDer(K):
>>>>   def __init__(self):
>>>>     K.__init__(self)
>>>>
>>>> k = KDer()
>>>>
>>>> In contrast to my initial solution the lines
>>>>   assert k.getThisRCP2() is not None
>>>>   k2 = k.getThisRCP2()
>>>> now work, which is nice.
>>>>
>>>> Also this call works now:
>>>>   print k2.foo();
>>>>
>>>> However, the following two lines
>>>>   del k
>>>>   print k2.foo();
>>>> give a segmentation fault. Note that the destructor for the class `K`
>>>> has not been called, which  would have printed a statement.
>>>
>>> Please post a reduced example without any of the "J" business that
>>> illustrates exactly the problem you're having for this one particular
>>> case.  If doing so doesn't immediately reveal the problem to you, I'm
>>> sure we can get to the bottom of it here.
>>>
>>>
>>> --
>>> Dave Abrahams
>>> BoostPro Computing
>>> http://www.boostpro.com
>>>
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> [hidden email]
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>
>> from thisRCPPy import *
>>
>> print '\ntestcase for thisRCP as from python extended class (using enable_shared_from_this):'
>> class KDer(K):
>>   def __init__(self):
>>     K.__init__(self)
>>
>> k = KDer()
>> assert k.getThisRCP2() is not None
>> k2 = k.getThisRCP2()
>> if 1:
>>   print k2.foo();
>>   del k
>>   print 'with the `del k` statement the following statement will crash the program:';
>>   print k2.foo();
>>   print 'this line is never reached';
>>   #del k
>>   del k2
>>
>>
>> #include <boost/python.hpp>
>> #include "iostream"
>> #include "string"
>> #include "boost/weak_ptr.hpp"
>> #include "boost/enable_shared_from_this.hpp"
>>
>> using namespace boost::python;
>> using namespace std;
>> //using Teuchos::RCP;
>>
>> struct K
>> // : public boost::enable_shared_from_this<K>
>> {
>>   virtual int foo() = 0;
>>
>>   virtual ~K() {
>>     std::cout << "destructor for K called ..." << std::endl;
>>   }
>>
>>   /*boost::shared_ptr<K> getThisRCP() {
>>     return shared_from_this();
>>   }*/
>> };
>>
>> struct KWrapper : public K, public boost::python::wrapper<K>,
>>   public boost::enable_shared_from_this<KWrapper>
>> {
>>
>>   KWrapper() {
>>
>>   }
>>
>>   virtual int foo() {
>>     if( override f = this->get_override("foo") ) {
>>       return f();
>>     } else {
>>       return base_foo();
>>     }
>>   }
>>
>>   boost::shared_ptr<KWrapper> getThisRCP2() {
>>     return shared_from_this();
>>   }
>>
>>   int base_foo() {
>>     return 0;
>>   }
>> };
>>
>> BOOST_PYTHON_MODULE(thisRCPPy)
>> {
>>   {
>>     typedef K ClassT;
>>     class_<KWrapper, boost::shared_ptr<KWrapper>, boost::noncopyable >("K", init<>() )
>>       //.def("getThisRCP", &ClassT::getThisRCP)
>>       .def("getThisRCP2", &KWrapper::getThisRCP2)
>>       .def("foo", &ClassT::foo)
>>       ;
>>   }
>> }
>> _______________________________________________
>> Cplusplus-sig mailing list
>> [hidden email]
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
> --
> Dave Abrahams
> BoostPro Computing
> http://www.boostpro.com
>
> _______________________________________________
> Cplusplus-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Bryan Catanzaro
In reply to this post by Holger Brandsmeier
Holger Brandsmeier <brandsmeier <at> gmx.de> writes:

>
> Dear list,
>
> how is it possible to have a class in C++ that can be extended from
> python and that stores a weak_ptr to itself?


Have you tried using boost::enable_shared_from_this?  The following example code
seems to do what you want.

------------  foo.cpp  ------------
#include <boost/python.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/enable_shared_from_this.hpp>
#include <iostream>

//Inheriting boost::enable_shared_from_this adds a weak_ptr to the object
//That gets automatically initialized when you construct a shared_ptr
//holding the object.

class A
    : public boost::enable_shared_from_this<A> {
    static int counter;
public:    
    int id;
    A() {
        id = counter++;
        std::cout << "Constructing A[" << id << "]" << std::endl;
    }
    ~A() {
        std::cout << "Destructing A[" << id << "]" << std::endl;
    }
//This function allows us to get a shared_ptr holding the object
    boost::shared_ptr<A> ptr() {
        return shared_from_this();
    }
};

int A::counter = 0;

using namespace boost::python;

class B {
    boost::shared_ptr<A> m_a;
    static int counter;
public:
    int id;
    //Notice that B accepts an A object directly
    //And asks it for the shared_ptr which holds it.
    B(A& a) : m_a(a.ptr()) {
        id = counter++;
        std::cout << "Constructing B[" << id << "]";
        std::cout << ", which holds A[" << m_a->id << "]" << std::endl;
    }
    ~B() {
        std::cout << "Destructing B[" << id << "]";
        std::cout << ", which holds A[" << m_a->id << "]" << std::endl;
    }
};

int B::counter = 0;

BOOST_PYTHON_MODULE(foo) {
    class_<A, boost::shared_ptr<A> >("A", init<>());
    class_<B>("B", init<A&>());
}
-----------------------------------

------------  test.py  ------------
import foo

class ExtendedA(foo.A):
    def special(self):
        return "successfully extended foo.A"
a0 = foo.A()
a = ExtendedA()
print a.special()
print "an ExtendedA object is a foo.A object: %s" % isinstance(a, foo.A)
b = foo.B(a)
-----------------------------------

On my system the output is:

Constructing A[0]
Constructing A[1]
successfully extended foo.A
an ExtendedA object is a foo.A object: True
Constructing B[0], which holds A[1]
Destructing B[0], which holds A[1]
Destructing A[1]
Destructing A[0]

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Holger Brandsmeier
Bryan,

my problems happen as soon as I need to use wrappers. I really need
the class that I extend in python to implement some pure virtual
function in the C++ class. I extended your code and refined what is
going wrong. I tried several Variants with different errors, I used
#defines to show you all 4 versions, next to each Variant I put a C++
comment of what is going wrong.

So here it is:

------------  foo.cpp  ------------
#include <boost/python.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/enable_shared_from_this.hpp>
#include <iostream>

using namespace boost::python;

class A
   : public boost::enable_shared_from_this<A> {
   static int counter;
public:
   int id;
   A() {
       id = counter++;
       std::cout << "Constructing A[" << id << "]" << std::endl;
   }
   ~A() {
       std::cout << "Destructing A[" << id << "]" << std::endl;
   }
   boost::shared_ptr<A> ptr() {
       return shared_from_this();
   }
#define VARIANT0
#ifdef VARIANT0
   // this variant works nicely with VARIANT1 below (compiles and
   // no error for `aa = a.ptr()`), but it produces wrong results,
   // i.e. `77` is returned for `aa.foo()` although it should have
   // been `33`.
   virtual int foo() {
      return 77;
   }
#else
   virtual int foo() = 0;
#endif
};

struct AWrapper : public A, public boost::python::wrapper<A> {
  virtual int foo() {
    if( override f = this->get_override("foo") ) {
      return f();
    } else {
      return 42;
    }
  }
};

int A::counter = 0;

using namespace boost::python;


BOOST_PYTHON_MODULE(foo) {
#define VARIANT1
//#define VARIANT2
//#define VARIANT3

#ifdef VARIANT1
   // The following line doesn't compites it gives the error:
   // error: cannot allocate an object of abstract type ‘A’
   class_<AWrapper, boost::shared_ptr<A>, boost::noncopyable >("A", init<>())
#endif
#ifdef VARIANT2
  // the following compiles, but produces an error in python, when executing
  // aa = a.ptr();
  // RuntimeError: tr1::bad_weak_ptr
   class_<AWrapper, boost::noncopyable  >("A", init<>())
#endif
#ifdef VARIANT3
   // the following compiles, but produces an error in python, when executing
   // aa = a.ptr();
   // TypeError: No to_python (by-value) converter found for C++ type:
boost::shared_ptr<A>
   class_<AWrapper, boost::shared_ptr<AWrapper>, boost::noncopyable
>("A", init<>())
#endif
     .def("ptr", &A::ptr)
     .def("foo", &A::foo)
     ;
}
-----------------------------------

------------  foo.py  ------------
import foo

class ExtendedA(foo.A):
   def foo(self):
     return 33;

a0 = foo.A()
a = ExtendedA()
aa = a.ptr();
print a.foo()
del a
print aa.foo()
-----------------------------------

Do you have any suggestion how to get this working?
-Holger

On Thu, Apr 26, 2012 at 18:43, Bryan Catanzaro <[hidden email]> wrote:

> Holger Brandsmeier <brandsmeier <at> gmx.de> writes:
>
>>
>> Dear list,
>>
>> how is it possible to have a class in C++ that can be extended from
>> python and that stores a weak_ptr to itself?
>
>
> Have you tried using boost::enable_shared_from_this?  The following example code
> seems to do what you want.
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Bryan Catanzaro
I think the problem is that aa is no longer an ExtendedA object,
because when you call ptr() it returns a pointer to the base C++ A
class, not the derived class.

I think at this point it's probably best to reconsider the problem and
come up with a simpler solution.  Why do you want to expose the ptr
method to Python anyway?

- bryan

On Thu, Apr 26, 2012 at 11:49 AM, Holger Brandsmeier <[hidden email]> wrote:

> Bryan,
>
> my problems happen as soon as I need to use wrappers. I really need
> the class that I extend in python to implement some pure virtual
> function in the C++ class. I extended your code and refined what is
> going wrong. I tried several Variants with different errors, I used
> #defines to show you all 4 versions, next to each Variant I put a C++
> comment of what is going wrong.
>
> So here it is:
>
> ------------  foo.cpp  ------------
> #include <boost/python.hpp>
> #include <boost/shared_ptr.hpp>
> #include <boost/enable_shared_from_this.hpp>
> #include <iostream>
>
> using namespace boost::python;
>
> class A
>   : public boost::enable_shared_from_this<A> {
>   static int counter;
> public:
>   int id;
>   A() {
>       id = counter++;
>       std::cout << "Constructing A[" << id << "]" << std::endl;
>   }
>   ~A() {
>       std::cout << "Destructing A[" << id << "]" << std::endl;
>   }
>   boost::shared_ptr<A> ptr() {
>       return shared_from_this();
>   }
> #define VARIANT0
> #ifdef VARIANT0
>   // this variant works nicely with VARIANT1 below (compiles and
>   // no error for `aa = a.ptr()`), but it produces wrong results,
>   // i.e. `77` is returned for `aa.foo()` although it should have
>   // been `33`.
>   virtual int foo() {
>      return 77;
>   }
> #else
>   virtual int foo() = 0;
> #endif
> };
>
> struct AWrapper : public A, public boost::python::wrapper<A> {
>  virtual int foo() {
>    if( override f = this->get_override("foo") ) {
>      return f();
>    } else {
>      return 42;
>    }
>  }
> };
>
> int A::counter = 0;
>
> using namespace boost::python;
>
>
> BOOST_PYTHON_MODULE(foo) {
> #define VARIANT1
> //#define VARIANT2
> //#define VARIANT3
>
> #ifdef VARIANT1
>   // The following line doesn't compites it gives the error:
>   // error: cannot allocate an object of abstract type ‘A’
>   class_<AWrapper, boost::shared_ptr<A>, boost::noncopyable >("A", init<>())
> #endif
> #ifdef VARIANT2
>  // the following compiles, but produces an error in python, when executing
>  // aa = a.ptr();
>  // RuntimeError: tr1::bad_weak_ptr
>   class_<AWrapper, boost::noncopyable  >("A", init<>())
> #endif
> #ifdef VARIANT3
>   // the following compiles, but produces an error in python, when executing
>   // aa = a.ptr();
>   // TypeError: No to_python (by-value) converter found for C++ type:
> boost::shared_ptr<A>
>   class_<AWrapper, boost::shared_ptr<AWrapper>, boost::noncopyable
>>("A", init<>())
> #endif
>     .def("ptr", &A::ptr)
>     .def("foo", &A::foo)
>     ;
> }
> -----------------------------------
>
> ------------  foo.py  ------------
> import foo
>
> class ExtendedA(foo.A):
>   def foo(self):
>     return 33;
>
> a0 = foo.A()
> a = ExtendedA()
> aa = a.ptr();
> print a.foo()
> del a
> print aa.foo()
> -----------------------------------
>
> Do you have any suggestion how to get this working?
> -Holger
>
> On Thu, Apr 26, 2012 at 18:43, Bryan Catanzaro <[hidden email]> wrote:
>> Holger Brandsmeier <brandsmeier <at> gmx.de> writes:
>>
>>>
>>> Dear list,
>>>
>>> how is it possible to have a class in C++ that can be extended from
>>> python and that stores a weak_ptr to itself?
>>
>>
>> Have you tried using boost::enable_shared_from_this?  The following example code
>> seems to do what you want.
> _______________________________________________
> Cplusplus-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Holger Brandsmeier
On Fri, Apr 27, 2012 at 08:22, Bryan Catanzaro <[hidden email]> wrote:
> I think the problem is that aa is no longer an ExtendedA object,
> because when you call ptr() it returns a pointer to the base C++ A
> class, not the derived class.

I think you got something confused. A boost::shared_pointer<A> can
very well be any pointer to any derived class, no copy should be
created. I can still call virtual functions from `A` and should get
the result from the implementation of `ExtendedA`.

> I think at this point it's probably best to reconsider the problem and
> come up with a simpler solution.  Why do you want to expose the ptr
> method to Python anyway?

I don't plan to call `ptr()` from python, but I will call the same
function from C++ and and at some point the result that C++ produced
can find its way back to python.

I did find a different solution to this problem:

On Mon, Apr 23, 2012 at 20:48, Holger Brandsmeier <[hidden email]> wrote:
> In the meantime I figured out a different way of doing the shared
> pointer from this in python. Namely, to just make getThisRCP() a
> virtual function, add it to the Wrapper and to implement it in the
> python by implementing it as

but I still expect the above mentioned approch with
`boost::enable_shared_from_this` _should_ have worked with boost
python. I seem to run into an issue that even Dave can not explain
(well lets wait for his reply).

Moreover note that in boost::python usually if C++ returns a
`shared_ptr<A>` which has previously been created as an instance of
`ExtendedA` inside python, than this class is available as
`ExtendedA`. There is some magic with custom deleters that usually
makes you able to treat that class as instance of `ExtendedA`. This
magic will also stop working when you use
`boost::enable_shared_from_this`. With that aspect I can live, though.

-Holger

> On Thu, Apr 26, 2012 at 11:49 AM, Holger Brandsmeier <[hidden email]> wrote:
>> Bryan,
>>
>> my problems happen as soon as I need to use wrappers. I really need
>> the class that I extend in python to implement some pure virtual
>> function in the C++ class. I extended your code and refined what is
>> going wrong. I tried several Variants with different errors, I used
>> #defines to show you all 4 versions, next to each Variant I put a C++
>> comment of what is going wrong.
>>
>> So here it is:
>>
>> ------------  foo.cpp  ------------
>> #include <boost/python.hpp>
>> #include <boost/shared_ptr.hpp>
>> #include <boost/enable_shared_from_this.hpp>
>> #include <iostream>
>>
>> using namespace boost::python;
>>
>> class A
>>   : public boost::enable_shared_from_this<A> {
>>   static int counter;
>> public:
>>   int id;
>>   A() {
>>       id = counter++;
>>       std::cout << "Constructing A[" << id << "]" << std::endl;
>>   }
>>   ~A() {
>>       std::cout << "Destructing A[" << id << "]" << std::endl;
>>   }
>>   boost::shared_ptr<A> ptr() {
>>       return shared_from_this();
>>   }
>> #define VARIANT0
>> #ifdef VARIANT0
>>   // this variant works nicely with VARIANT1 below (compiles and
>>   // no error for `aa = a.ptr()`), but it produces wrong results,
>>   // i.e. `77` is returned for `aa.foo()` although it should have
>>   // been `33`.
>>   virtual int foo() {
>>      return 77;
>>   }
>> #else
>>   virtual int foo() = 0;
>> #endif
>> };
>>
>> struct AWrapper : public A, public boost::python::wrapper<A> {
>>  virtual int foo() {
>>    if( override f = this->get_override("foo") ) {
>>      return f();
>>    } else {
>>      return 42;
>>    }
>>  }
>> };
>>
>> int A::counter = 0;
>>
>> using namespace boost::python;
>>
>>
>> BOOST_PYTHON_MODULE(foo) {
>> #define VARIANT1
>> //#define VARIANT2
>> //#define VARIANT3
>>
>> #ifdef VARIANT1
>>   // The following line doesn't compites it gives the error:
>>   // error: cannot allocate an object of abstract type ‘A’
>>   class_<AWrapper, boost::shared_ptr<A>, boost::noncopyable >("A", init<>())
>> #endif
>> #ifdef VARIANT2
>>  // the following compiles, but produces an error in python, when executing
>>  // aa = a.ptr();
>>  // RuntimeError: tr1::bad_weak_ptr
>>   class_<AWrapper, boost::noncopyable  >("A", init<>())
>> #endif
>> #ifdef VARIANT3
>>   // the following compiles, but produces an error in python, when executing
>>   // aa = a.ptr();
>>   // TypeError: No to_python (by-value) converter found for C++ type:
>> boost::shared_ptr<A>
>>   class_<AWrapper, boost::shared_ptr<AWrapper>, boost::noncopyable
>>>("A", init<>())
>> #endif
>>     .def("ptr", &A::ptr)
>>     .def("foo", &A::foo)
>>     ;
>> }
>> -----------------------------------
>>
>> ------------  foo.py  ------------
>> import foo
>>
>> class ExtendedA(foo.A):
>>   def foo(self):
>>     return 33;
>>
>> a0 = foo.A()
>> a = ExtendedA()
>> aa = a.ptr();
>> print a.foo()
>> del a
>> print aa.foo()
>> -----------------------------------
>>
>> Do you have any suggestion how to get this working?
>> -Holger
>>
>> On Thu, Apr 26, 2012 at 18:43, Bryan Catanzaro <[hidden email]> wrote:
>>> Holger Brandsmeier <brandsmeier <at> gmx.de> writes:
>>>
>>>>
>>>> Dear list,
>>>>
>>>> how is it possible to have a class in C++ that can be extended from
>>>> python and that stores a weak_ptr to itself?
>>>
>>>
>>> Have you tried using boost::enable_shared_from_this?  The following example code
>>> seems to do what you want.
>> _______________________________________________
>> Cplusplus-sig mailing list
>> [hidden email]
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weak_ptr to this in C++ classes extended from python

Dave Abrahams

on Fri Apr 27 2012, Holger Brandsmeier <brandsmeier-AT-gmx.de> wrote:

> On Mon, Apr 23, 2012 at 20:48, Holger Brandsmeier <[hidden email]> wrote:
>> In the meantime I figured out a different way of doing the shared
>> pointer from this in python. Namely, to just make getThisRCP() a
>> virtual function, add it to the Wrapper and to implement it in the
>> python by implementing it as
>
> but I still expect the above mentioned approch with
> `boost::enable_shared_from_this` _should_ have worked with boost
> python. I seem to run into an issue that even Dave can not explain
> (well lets wait for his reply).

Unfortunately I don't have the time to do the analysis right now, but I
agree with you that it should have worked.

--
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Loading...