how to override __setattr__ of an extension class

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

how to override __setattr__ of an extension class

Holger Joukl

Hi,

I'm trying to instrument a wrapped class with  custom __setattr__
mechanics:

        .def("__setattr__", &MyExtensionClass::setattr)

How can I call the base class __setattr__ from within setattr and not run
into infinite recursion?

I.e. the analogon in C++ to

>> class Foo(object):
...     def __setattr__(self, attr, value):
...         print "custom __setattr__"
...         object.__setattr__(self, attr, value)
...
>>> foo = Foo()
>>> foo.x = 23
custom __setattr__
>>> foo.x
23
>>>

To actually write to my instance dict I currently do

    PyObject* __dict__ = PyObject_GetAttrString(m_self, const_cast<char*>
("__dict__"));
    PyDict_SetItemString(__dict__, name, value.ptr());

inside MyExtensionClass::setattr (where m_self is PyObj*)

Is there a better way?

On a sidenote, I initially thought I could wrap the __dict__ in bp::dict()
to
use its nice Python-like API for item assignment but this seems to always
make a copy of the __dict__.
I.e something like

bp::dict bp_dict(__dict__);
bp_dict[name] = value;

won't work.

Regards
Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: how to override __setattr__ of an extension class

Jim Bosch-2
On 02/23/2012 12:04 PM, Holger Joukl wrote:

>
> Hi,
>
> I'm trying to instrument a wrapped class with  custom __setattr__
> mechanics:
>
>          .def("__setattr__",&MyExtensionClass::setattr)
>
> How can I call the base class __setattr__ from within setattr and not run
> into infinite recursion?
>
> I.e. the analogon in C++ to
>
>>> class Foo(object):
> ...     def __setattr__(self, attr, value):
> ...         print "custom __setattr__"
> ...         object.__setattr__(self, attr, value)
> ...
>>>> foo = Foo()
>>>> foo.x = 23
> custom __setattr__
>>>> foo.x
> 23
>>>>
>
> To actually write to my instance dict I currently do
>
>      PyObject* __dict__ = PyObject_GetAttrString(m_self, const_cast<char*>
> ("__dict__"));
>      PyDict_SetItemString(__dict__, name, value.ptr());
>
> inside MyExtensionClass::setattr (where m_self is PyObj*)
>
> Is there a better way?
>

Everything I can think of (e.g. extract the base class from __bases__[0]
and call its setattr) is functionally equivalent and not any prettier.


> On a sidenote, I initially thought I could wrap the __dict__ in bp::dict()
> to
> use its nice Python-like API for item assignment but this seems to always
> make a copy of the __dict__.
> I.e something like
>
> bp::dict bp_dict(__dict__);
> bp_dict[name] = value;
>
> won't work.
>

The trick is to do:

bp::dict bp_dict = bp::extract<bp::dict>(__dict__);
bp_dict[name] = value;

Just calling the bp::dict invokes Python's dict constructor, which does
a copy.


Jim
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: how to override __setattr__ of an extension class

Holger Joukl
> > To actually write to my instance dict I currently do
> >
> >      PyObject* __dict__ = PyObject_GetAttrString(m_self,
const_cast<char*>

> > ("__dict__"));
> >      PyDict_SetItemString(__dict__, name, value.ptr());
> >
> > inside MyExtensionClass::setattr (where m_self is PyObj*)
> >
> > Is there a better way?
> >
>
> Everything I can think of (e.g. extract the base class from __bases__[0]
> and call its setattr) is functionally equivalent and not any prettier.

Good to know. I'll stick to this, then.

> The trick is to do:
>
> bp::dict bp_dict = bp::extract<bp::dict>(__dict__);
> bp_dict[name] = value;
>
> Just calling the bp::dict invokes Python's dict constructor, which does
> a copy.

...and now that you've told me what to do I've found that it's actually
documented
here:

http://www.boost.org/doc/libs/1_48_0/libs/python/doc/tutorial/doc/html/python/object.html#python.extracting_c___objects:

"[...]
The astute reader might have noticed that the extract<T> facility in fact
solves the mutable copying problem:


dict d = extract<dict>(x.attr("__dict__"));
d["whatever"] = 3;          // modifies x.__dict__ !
[...]"


and there:



http://www.boost.org/doc/libs/1_48_0/libs/python/doc/v2/extract.html:
"[...] Because invoking a mutable python type with an argument of the same
type (e.g. list([1,2]) typically makes a copy of the argument object, this
may be the only way to access the ObjectWrapper's interface on the original
object.[...]"

Maybe that's only me but I don't think the info that gets you going here is
very obvious in the docs, more like
a casual side note. So for now I've added some info on this here:
http://wiki.python.org/moin/boost.python/extract

Many thanks
Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: how to override __setattr__ of an extension class

Wojciech Mamrak
Here is a part of my message posted here few months ago, which has
been left without any response (no offence - maybe it disappeared
somewhere):

The docs regarding the mutable copying problem are very misleading.
Following Python [1] and Boost [2] scripts do not produce the same results:


[1]
b = a = []
c = list(a)
a.append("s1")
print a.count("s1"), b.count("s1"), c.count("s1") #prints 1 1 0

[2]
list a, b, c;
b = a;
c = list(a);
a.append("s1");
list d = extract<list>(a);
printf("%i %i %i %i\n", a.count("s1"), b.count("s1"), c.count("s1"),
d.count("s1")); //prints 1 1 1 1

In [2] expected was 1 1 0 1 according to Pythonic behaviour (it states
in docs that such behaviour is mimicked). This is in conflict with
example provided in the tutorial.

regards


2012/2/28 Holger Joukl <[hidden email]>:

>> > To actually write to my instance dict I currently do
>> >
>> >      PyObject* __dict__ = PyObject_GetAttrString(m_self,
> const_cast<char*>
>> > ("__dict__"));
>> >      PyDict_SetItemString(__dict__, name, value.ptr());
>> >
>> > inside MyExtensionClass::setattr (where m_self is PyObj*)
>> >
>> > Is there a better way?
>> >
>>
>> Everything I can think of (e.g. extract the base class from __bases__[0]
>> and call its setattr) is functionally equivalent and not any prettier.
>
> Good to know. I'll stick to this, then.
>
>> The trick is to do:
>>
>> bp::dict bp_dict = bp::extract<bp::dict>(__dict__);
>> bp_dict[name] = value;
>>
>> Just calling the bp::dict invokes Python's dict constructor, which does
>> a copy.
>
> ...and now that you've told me what to do I've found that it's actually
> documented
> here:
>
> http://www.boost.org/doc/libs/1_48_0/libs/python/doc/tutorial/doc/html/python/object.html#python.extracting_c___objects:
>
> "[...]
> The astute reader might have noticed that the extract<T> facility in fact
> solves the mutable copying problem:
>
>
> dict d = extract<dict>(x.attr("__dict__"));
> d["whatever"] = 3;          // modifies x.__dict__ !
> [...]"
>
>
> and there:
>
>
>
> http://www.boost.org/doc/libs/1_48_0/libs/python/doc/v2/extract.html:
> "[...] Because invoking a mutable python type with an argument of the same
> type (e.g. list([1,2]) typically makes a copy of the argument object, this
> may be the only way to access the ObjectWrapper's interface on the original
> object.[...]"
>
> Maybe that's only me but I don't think the info that gets you going here is
> very obvious in the docs, more like
> a casual side note. So for now I've added some info on this here:
> http://wiki.python.org/moin/boost.python/extract
>
> Many thanks
> Holger
>
> Landesbank Baden-Wuerttemberg
> Anstalt des oeffentlichen Rechts
> Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
> HRA 12704
> Amtsgericht Stuttgart
>
> _______________________________________________
> Cplusplus-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: how to override __setattr__ of an extension class

Holger Joukl
Hi,

> The docs regarding the mutable copying problem are very misleading.
> Following Python [1] and Boost [2] scripts do not produce the same
results:

>
>
> [1]
> b = a = []
> c = list(a)
> a.append("s1")
> print a.count("s1"), b.count("s1"), c.count("s1") #prints 1 1 0
>
> [2]
> list a, b, c;
> b = a;
> c = list(a);
> a.append("s1");
> list d = extract<list>(a);
> printf("%i %i %i %i\n", a.count("s1"), b.count("s1"), c.count("s1"),
> d.count("s1")); //prints 1 1 1 1
>
> In [2] expected was 1 1 0 1 according to Pythonic behaviour (it states
> in docs that such behaviour is mimicked). This is in conflict with
> example provided in the tutorial.

Ok, so it seems behaviour is different when dealing with
boost::python::object vs
PyObject* as constructor arguments.

Using extract<> to modify mutable objects is only necessary when dealing
with PyObject*:

// boost_dict.cpp
/* build cmdline:

/apps/local/gcc/4.6.1/bin/g++ -g -Wall -R /apps/local/gcc/4.6.1/lib/
-R /var/tmp/boost/gcc/boost_1_47_0/stage/gcc-4.6.1/py2.7/boost/1.47.0/lib/
-L /var/tmp/boost/gcc/boost_1_47_0/stage/gcc-4.6.1/py2.7/boost/1.47.0/lib/
-I /var/tmp/boost/gcc/boost_1_47_0/
-I /apps/local/gcc/4.6.1/include/python2.7/  -L /apps/local/gcc/4.6.1/lib
boost_dict.cpp -lboost_python -lrt -lpython2.7
*/

#include <boost/python.hpp>
#include <iostream>

namespace bp = boost::python;


int main(void) {
    // otherwise we core dump
    Py_Initialize();

    bp::object value(1);

    std::cout << "arg is bp::dict:" << std::endl;
    {
        bp::dict d1, d2, d3;
        d2 = d1;
        d3 = bp::dict(d1);
        d1["k1"] = value;
        bp::dict d4 = bp::extract<bp::dict>(d1);

        std::cout << d1.has_key("k1")
                  << " " << d2.has_key("k1")
                  << " " << d3.has_key("k1")
                  << " " << d4.has_key("k1") << std::endl;
    }

    std::cout << std::endl;

    std::cout << "arg is bp::handle<(PyObj*)" << std::endl;
    {
        bp::dict d1, d2, d3;

        // must use a handle to feed PyObj* to bp::objects
        bp::handle<> dict_pyobj_handle(d1.ptr());

        // Note: no operator= for handle<>
        d2 = bp::dict(dict_pyobj_handle);
        d3 = bp::dict(dict_pyobj_handle);
        d1["k1"] = value;
        bp::dict d4 = bp::extract<bp::dict>(d1.ptr());
        std::cout << d1.has_key("k1")
                  << " " << d2.has_key("k1")
                  << " " << d3.has_key("k1")
                  << " " << d4.has_key("k1") << std::endl;
    }


    Py_Finalize();
    return 0;
}


$ ./a.out
arg is bp::dict:
1 1 1 1

arg is bp::handle<(PyObj*)
1 0 0 1

So I think the tutorial example (if that's what you meant?)
"
C++:


dict d(x.attr("__dict__"));  // copies x.__dict__
d['whatever'] = 3;           // modifies the copy
"


is pretty subtle, as - if I understand correctly - this


- retrieves an attribute proxy from x


- that returns the bp::object-wrapped __dict__ attribute (which copies the
original PyObject* __dict__)


- constructs d from this new bp::object (which increases the refcount but
doesn't make another copy)


and the assignment operates on this new bp::object.

Regards
Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: how to override __setattr__ of an extension class

Wojciech Mamrak
Yes, this is what I meant :)

I will precise my previous message. The docs are very concise on some
topics, hence misleading.
If you find such statement:
"Beware the common pitfall of forgetting that the constructors of most
of Python's mutable types make copies, just as in Python."

then it is confusing why:

                dict a;
                dict b = dict(a); //copy? in Python yes, here not
                b["key"] = 1;
                (a.has_key("key")) ? puts("KEY FOUND") : puts("KEY NOT FOUND"); //KEY FOUND

yields KEY FOUND, opposite to its Python equivalent (posted earlier).

But the docs are correct, since they provide more or less such
exemplary (and different) code:

                object x = import("__main__");
                dict a(x.attr("__dict__")); //copy
                dict b = extract<dict>(x.attr("__dict__")); //no copy
                x.attr("__dict__")["key"] = 1;
                (a.has_key("key")) ? puts("KEY FOUND") : puts("KEY NOT FOUND");
//KEY NOT FOUND
                (b.has_key("key")) ? puts("KEY FOUND") : puts("KEY NOT FOUND"); //KEY FOUND


regards





2012/2/28 Holger Joukl <[hidden email]>:

> Hi,
>
>> The docs regarding the mutable copying problem are very misleading.
>> Following Python [1] and Boost [2] scripts do not produce the same
> results:
>>
>>
>> [1]
>> b = a = []
>> c = list(a)
>> a.append("s1")
>> print a.count("s1"), b.count("s1"), c.count("s1") #prints 1 1 0
>>
>> [2]
>> list a, b, c;
>> b = a;
>> c = list(a);
>> a.append("s1");
>> list d = extract<list>(a);
>> printf("%i %i %i %i\n", a.count("s1"), b.count("s1"), c.count("s1"),
>> d.count("s1")); //prints 1 1 1 1
>>
>> In [2] expected was 1 1 0 1 according to Pythonic behaviour (it states
>> in docs that such behaviour is mimicked). This is in conflict with
>> example provided in the tutorial.
>
> Ok, so it seems behaviour is different when dealing with
> boost::python::object vs
> PyObject* as constructor arguments.
>
> Using extract<> to modify mutable objects is only necessary when dealing
> with PyObject*:
>
> // boost_dict.cpp
> /* build cmdline:
>
> /apps/local/gcc/4.6.1/bin/g++ -g -Wall -R /apps/local/gcc/4.6.1/lib/
> -R /var/tmp/boost/gcc/boost_1_47_0/stage/gcc-4.6.1/py2.7/boost/1.47.0/lib/
> -L /var/tmp/boost/gcc/boost_1_47_0/stage/gcc-4.6.1/py2.7/boost/1.47.0/lib/
> -I /var/tmp/boost/gcc/boost_1_47_0/
> -I /apps/local/gcc/4.6.1/include/python2.7/  -L /apps/local/gcc/4.6.1/lib
> boost_dict.cpp -lboost_python -lrt -lpython2.7
> */
>
> #include <boost/python.hpp>
> #include <iostream>
>
> namespace bp = boost::python;
>
>
> int main(void) {
>    // otherwise we core dump
>    Py_Initialize();
>
>    bp::object value(1);
>
>    std::cout << "arg is bp::dict:" << std::endl;
>    {
>        bp::dict d1, d2, d3;
>        d2 = d1;
>        d3 = bp::dict(d1);
>        d1["k1"] = value;
>        bp::dict d4 = bp::extract<bp::dict>(d1);
>
>        std::cout << d1.has_key("k1")
>                  << " " << d2.has_key("k1")
>                  << " " << d3.has_key("k1")
>                  << " " << d4.has_key("k1") << std::endl;
>    }
>
>    std::cout << std::endl;
>
>    std::cout << "arg is bp::handle<(PyObj*)" << std::endl;
>    {
>        bp::dict d1, d2, d3;
>
>        // must use a handle to feed PyObj* to bp::objects
>        bp::handle<> dict_pyobj_handle(d1.ptr());
>
>        // Note: no operator= for handle<>
>        d2 = bp::dict(dict_pyobj_handle);
>        d3 = bp::dict(dict_pyobj_handle);
>        d1["k1"] = value;
>        bp::dict d4 = bp::extract<bp::dict>(d1.ptr());
>        std::cout << d1.has_key("k1")
>                  << " " << d2.has_key("k1")
>                  << " " << d3.has_key("k1")
>                  << " " << d4.has_key("k1") << std::endl;
>    }
>
>
>    Py_Finalize();
>    return 0;
> }
>
>
> $ ./a.out
> arg is bp::dict:
> 1 1 1 1
>
> arg is bp::handle<(PyObj*)
> 1 0 0 1
>
> So I think the tutorial example (if that's what you meant?)
> "
> C++:
>
>
> dict d(x.attr("__dict__"));  // copies x.__dict__
> d['whatever'] = 3;           // modifies the copy
> "
>
>
> is pretty subtle, as - if I understand correctly - this
>
>
> - retrieves an attribute proxy from x
>
>
> - that returns the bp::object-wrapped __dict__ attribute (which copies the
> original PyObject* __dict__)
>
>
> - constructs d from this new bp::object (which increases the refcount but
> doesn't make another copy)
>
>
> and the assignment operates on this new bp::object.
>
> Regards
> Holger
>
> Landesbank Baden-Wuerttemberg
> Anstalt des oeffentlichen Rechts
> Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
> HRA 12704
> Amtsgericht Stuttgart
>
> _______________________________________________
> Cplusplus-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig