boost::python: marshalling buffers

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

boost::python: marshalling buffers

Christoph Rupp
Hi,

i have a C++ class that i'm wrapping with boost::python:

class Foo {
  // ...
  const char *get_buffer();
  size_t get_size() const;
  // ...
};

I want to send this class through a thrift interface (which uses the write() method to send the data to a file or a socket). For a few days now i'm trying to implement this. By now i understand that i have to implement the buffer protocol, and i know that i can create a buffer with boost::python (read-only access is sufficient):
inline object as_buffer(const Foo &f) {
  return object (handle<> (PyBuffer_FromMemory ((void*)f.get_buffer(), f.get_size())));
} 
(This is based on an email from 2009 that i found in this mailing list's archive.)

But i do not know how to glue it together to form a full module. This is how the module looks right now:

BOOST_PYTHON_MODULE(Foo)
{
  class_<Foo, boost::noncopyable>("Foo",
          init<int32_t, bool>())
    .def("add", afn)
    .def("finalize", &Foo::finalize)
    .def("empty", &Foo::empty)
    .def("clear", &Foo::clear)
  ;
}

How can i include that buffer conversion?

Thanks a lot,
Christoph

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re : boost::python: C++/Python function overload issue

jjchristophe

Hi,

here is my problem.

Let say have a C++ A class and a Ader class in Python, derived from A (through boost::python).
I have a list of functions f_i in Ader overloading the equivalent functions with same name in A.
The main is written in Python and call a method g of a C++ class B passing and argument of type Ader (it's written as A type under C++, but I don't have any type issue there).
Let say B::g call a function f_1, calling f_2, calling f_3. The error occur at f_3 as I could identify it uses A::f_3 instead of Ader::f_3.
By going down, I could confirm that, from f_1 to f_3, all the 3 called functions are the A version. 
So my question is, how can I fix this issue so that the B::g function uses Ader functions (meanning Python functions) and not A functions?
On a side, this is a C++/Python version of a code working fine when fully written in C++.

Christophe Jean-Joseph
 

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re : boost::python: C++/Python function overload issue

JS Unkn0wn
Does this (  http://www.boost.org/doc/libs/1_49_0/libs/python/doc/tutorial/doc/html/python/exposing.html  ) answer your question? specifically "Class Virtual Functions".

On Wed, Apr 18, 2012 at 11:55 PM, christophe jean-joseph <[hidden email]> wrote:

Hi,

here is my problem.

Let say have a C++ A class and a Ader class in Python, derived from A (through boost::python).
I have a list of functions f_i in Ader overloading the equivalent functions with same name in A.
The main is written in Python and call a method g of a C++ class B passing and argument of type Ader (it's written as A type under C++, but I don't have any type issue there).
Let say B::g call a function f_1, calling f_2, calling f_3. The error occur at f_3 as I could identify it uses A::f_3 instead of Ader::f_3.
By going down, I could confirm that, from f_1 to f_3, all the 3 called functions are the A version. 
So my question is, how can I fix this issue so that the B::g function uses Ader functions (meanning Python functions) and not A functions?
On a side, this is a C++/Python version of a code working fine when fully written in C++.

Christophe Jean-Joseph
 

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig


_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re : boost::python: C++/Python function overload issue

Holger Brandsmeier
In reply to this post by jjchristophe
Christophe,

You need to work with Wrappers when you export the class A to python.
This wrapper will make sure that when C++ calls a class that you
overloaded in python that then the correct python implementation is
called.

See also the sections:

Class Virtual Functions
Virtual Functions with Default Implementations

of the document that JS Unkn0wn gave you:
http://www.boost.org/doc/libs/1_49_0/libs/python/doc/tutorial/doc/html/python/exposing.html

-Holger
On Thu, Apr 19, 2012 at 00:55, christophe jean-joseph
<[hidden email]> wrote:

>
> Hi,
>
> here is my problem.
>
> Let say have a C++ A class and a Ader class in Python, derived from A
> (through boost::python).
> I have a list of functions f_i in Ader overloading the equivalent functions
> with same name in A.
> The main is written in Python and call a method g of a C++ class B passing
> and argument of type Ader (it's written as A type under C++, but I don't
> have any type issue there).
> Let say B::g call a function f_1, calling f_2, calling f_3. The error occur
> at f_3 as I could identify it uses A::f_3 instead of Ader::f_3.
> By going down, I could confirm that, from f_1 to f_3, all the 3 called
> functions are the A version.
> So my question is, how can I fix this issue so that the B::g function uses
> Ader functions (meanning Python functions) and not A functions?
> On a side, this is a C++/Python version of a code working fine when fully
> written in C++.
>
> Christophe Jean-Joseph
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> [hidden email]
> http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re : Re : boost::python: C++/Python function overload issue

jjchristophe
In reply to this post by JS Unkn0wn

Thank you very much.
I knew the link already but it made me read my code again and realize that I forgot to reflect the overloaded functions with boost::python.

Christophe Jean-Joseph



De : JS Unkn0wn <[hidden email]>
À : christophe jean-joseph <[hidden email]>; Development of Python/C++ integration <[hidden email]>
Envoyé le : Jeudi 19 avril 2012 1h21
Objet : Re: [C++-sig] Re : boost::python: C++/Python function overload issue

Does this (  http://www.boost.org/doc/libs/1_49_0/libs/python/doc/tutorial/doc/html/python/exposing.html  ) answer your question? specifically "Class Virtual Functions".

On Wed, Apr 18, 2012 at 11:55 PM, christophe jean-joseph <[hidden email]> wrote:

Hi,

here is my problem.

Let say have a C++ A class and a Ader class in Python, derived from A (through boost::python).
I have a list of functions f_i in Ader overloading the equivalent functions with same name in A.
The main is written in Python and call a method g of a C++ class B passing and argument of type Ader (it's written as A type under C++, but I don't have any type issue there).
Let say B::g call a function f_1, calling f_2, calling f_3. The error occur at f_3 as I could identify it uses A::f_3 instead of Ader::f_3.
By going down, I could confirm that, from f_1 to f_3, all the 3 called functions are the A version. 
So my question is, how can I fix this issue so that the B::g function uses Ader functions (meanning Python functions) and not A functions?
On a side, this is a C++/Python version of a code working fine when fully written in C++.

Christophe Jean-Joseph
 

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig




_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: boost::python: marshalling buffers

Jim Bosch-2
In reply to this post by Christoph Rupp
On 04/18/2012 01:57 PM, Christoph Rupp wrote:

> Hi,
>
> i have a C++ class that i'm wrapping with boost::python:
>
> class Foo {
>    // ...
>    const char *get_buffer();
>    size_t get_size() const;
>    // ...
> };
>
> I want to send this class through a thrift interface (which uses the
> write() method to send the data to a file or a socket). For a few days now
> i'm trying to implement this. By now i understand that i have to implement
> the buffer protocol, and i know that i can create a buffer with
> boost::python (read-only access is sufficient):
>
> inline object as_buffer(const Foo&f) {
>    return object (handle<>  (PyBuffer_FromMemory ((void*)f.get_buffer(),
> f.get_size())));
> }
>
> (This is based on an email from 2009 that i found in this mailing list's
> archive.)
>
> But i do not know how to glue it together to form a full module. This is
> how the module looks right now:
>
> BOOST_PYTHON_MODULE(Foo)
> {
>    class_<Foo, boost::noncopyable>("Foo",
>            init<int32_t, bool>())
>      .def("add", afn)
>      .def("finalize",&Foo::finalize)
>      .def("empty",&Foo::empty)
>      .def("clear",&Foo::clear)
>    ;
> }
>
> How can i include that buffer conversion?
>

Sorry about the long delay in a reply, but I'm afraid that right now it
looks *really* tricky (possibly impossible) to add the buffer protocol
to a Boost.Python class.  Python doesn't have a special named methods
for the buffer protocol:

http://stackoverflow.com/questions/2079272/which-methods-implement-the-buffer-interface-in-python

And Boost.Python doesn't give you access to the PyTypeObject structure
where you'd be able to add the PyBufferProcs slots.

I think you have two options:

  - Wrap your buffer-getter function with another name and find a way to
get your serialization code to use that.

  - Wrap this class class using the raw Python C-API and use
Boost.Python custom lvalue converters to make it work with any
Boost.Python-wrapped code you have.  You could use Boost.Python to wrap
all the member functions, for instance, and add them to the Python C-API
class's dict.

Neither option is great, but hopefully one will allow you to move forward.

HTH!

Jim
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Loading...