Quantcast

[C++-sig] Boost.Python embedded doubt

Previous Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[C++-sig] Boost.Python embedded doubt

Eric Jardim
Hi,

For QtDesigner integration, I need to embedded Python extended classes exposed back to C++. So I started with the tutorial and the Boost.Python test example. The test is ok.

Then I changed the example to my needs, and tryed to load the Qt module. The program compile and run, but output:
Traceback (most recent call last):
  File "<string>", line 1, in ?
  File "/u/cwd7/devel/boost/python-qt4-0.0.3rc1/Qt/__init__.py", line 8, in ?
    import dl
ImportError: /g0dv/Bin/Linux/python/lib/python2.3/lib-dynload/dl.so: undefined symbol: _Py_NoneStruct

This error happens inside the PyRun_String function. I have commented the line of the "ld" module import, and it complain about another symbol. It does not look like a "dl" problem, but a linking problem.

Reading the tutorial,
http://www.boost.org/libs/python/doc/tutorial/doc/html/python/embedding.htm
it says: "To be able to use embedding in your programs, they have to be linked to both Boost.Python's and Python's static link library."

Is it really necessary to both the libs be static? I then (curiously) tried to link with the boost_python library dynamically and it complained about another (thrid Py~) symbol.

I am building this app with Qt "qmake". I do not use bjam, because I need specific precompiling for Qt (easyly done by qmake). But this should not be so complex, shoud it? I am just using simple:
>>> -L/path/to/python/lib/python2.3/config -lpython2.3 -L/path/to/boost/lib -lboost_python-static -lutil

Am I missing something? Do I have to do anything else?

The code I am trying is:
    using namespace boost::python;

    object main_module((
         handle<>(borrowed(PyImport_AddModule("__main__")))));

    object main_namespace = main_module.attr("__dict__");

    // Define the derived class in Python.
    // (You'll normally want to put this in a .py file.)
    python::handle<> result(
        PyRun_String(
        "import Qt\n"
        "from Qt.Gui import QPushButton\n"
        "print QPushButton\n",
        Py_file_input, main_namespace.ptr(), main_namespace.ptr())
        );
    // Result is not needed
    result.reset();

I also call PyInitialize first.

Thanks for any help,

[Eric Jardim]

_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

Stefan Seefeld
Eric Jardim wrote:
> Hi,
>
> For QtDesigner integration, I need to embedded Python extended classes
> exposed back to C++. So I started with the tutorial and the Boost.Python
> test example. The test is ok.

Please note that I recently added some functions specifically to make
embedding more convenient. There are three new functions: 'exec', 'exec_file',
and 'import'.
I'm not sure where to look for the updated reference manual, as the official
one doesn't contain my additions yet.
In the process I removed the (outdated) 'embedding' test and replaced it
with 'exec', which may be useful as an example. FWIW.

> Then I changed the example to my needs, and tryed to load the Qt module.
> The program compile and run, but output:
> Traceback (most recent call last):
>   File "<string>", line 1, in ?
>   File "/u/cwd7/devel/boost/python-qt4-0.0.3rc1/Qt/__init__.py", line 8,
> in ?
>     import dl
> ImportError: /g0dv/Bin/Linux/python/lib/python2.3/lib-dynload/dl.so:
> undefined symbol: _Py_NoneStruct
>
> This error happens inside the PyRun_String function. I have commented
> the line of the "ld" module import, and it complain about another
> symbol. It does not look like a "dl" problem, but a linking problem.

As your C++ application now acts as the python interpreter itself, it
has to provide all the symbols from the python runtime.

> Reading the tutorial,
> http://www.boost.org/libs/python/doc/tutorial/doc/html/python/embedding.htm
> it says: "To be able to use embedding in your programs, they have to be
> linked to both Boost.Python's and Python's static link library."

I don't think so. But in the past only the static python library was provided.

> Is it really necessary to both the libs be static? I then (curiously)
> tried to link with the boost_python library dynamically and it
> complained about another (thrid Py~) symbol.
>
> I am building this app with Qt "qmake". I do not use bjam, because I
> need specific precompiling for Qt (easyly done by qmake). But this
> should not be so complex, shoud it? I am just using simple:
>> >> -L/path/to/python/lib/python2.3/config -lpython2.3
> -L/path/to/boost/lib -lboost_python-static -lutil
>
> Am I missing something? Do I have to do anything else?

The missing symbols are provided by the python runtime, so linking
to the python library (static or dynamic shouldn't matter) should
do the trick.

Regards,
                Stefan
_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

Stefan Seefeld
In reply to this post by Eric Jardim
Eric Jardim wrote:

> I am building this app with Qt "qmake". I do not use bjam, because I
> need specific precompiling for Qt (easyly done by qmake). But this
> should not be so complex, shoud it? I am just using simple:
>> >> -L/path/to/python/lib/python2.3/config -lpython2.3
> -L/path/to/boost/lib -lboost_python-static -lutil

The order of libraries on the command line is significant. As you list
libboost_python after libpython2.3, its references to python symbols
can't be resolved. What happens if you put '-lpython2.3 -lutil' last ?

Regards,
                Stefan
_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

Eric Jardim
In reply to this post by Stefan Seefeld
2005/8/31, Stefan Seefeld <[hidden email]>:
Please note that I recently added some functions specifically to make
embedding more convenient. There are three new functions: 'exec', 'exec_file',
and 'import'.
I'm not sure where to look for the updated reference manual, as the official
one doesn't contain my additions yet.
In the process I removed the (outdated) 'embedding' test and replaced it
with 'exec', which may be useful as an example. FWIW.

Fine, this is good. I'll search for it. 


As your C++ application now acts as the python interpreter itself, it
has to provide all the symbols from the python runtime.

?! What do you mean by that? Should I link to the "dl.so" manually? How should I do it? If I import the "sys" module, everything goes fine. I am a few confused of what to do (so many scopes).
 
Sorry, I never wrote an embedded python app before.


I don't think so. But in the past only the static python library was provided.

You can use "--enable-shared" at the configure.
 

>> >> -L/path/to/python/lib/python2.3/config -lpython2.3
> -L/path/to/boost/lib -lboost_python-static -lutil
The missing symbols are provided by the python runtime, so linking
to the python library (static or dynamic shouldn't matter) should
do the trick.

But I am linking to the python static lib (with "-lpython2.3").  There might me something else. The strange thing is that I am linking to the python static lib, and on the import it gives this error:
Traceback (most recent call last):
  File "<string>", line 3, in ?
  File "Qt/__init__.py", line 8, in ?
ImportError: /g0dv/Bin/Linux/python/lib/python2.4/lib-dynload/dl.so: undefined symbol: _Py_NoneStruct

Thanks,

[Eric Jardim]

_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

Eric Jardim
In reply to this post by Stefan Seefeld
2005/8/31, Stefan Seefeld <[hidden email]>:
The order of libraries on the command line is significant. As you list
libboost_python after libpython2.3, its references to python symbols
can't be resolved. What happens if you put '-lpython2.3 -lutil' last ?

Well, it was actually after. I tried mixing the orders here but things got the same.

I'll try making python a dynamic lib to see what happens.

Should I "dlopen" the dl.so manually or something?

Thanks, again...

[Eric Jardim]

_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

Eric Jardim
2005/8/31, Eric Jardim <[hidden email]>:
I'll try making python a dynamic lib to see what happens.

Hmmm, using the dynamic lib seems to work. I just have to setup things here because I compiled Python in another directory prefix.

Thanks,

[Eric Jardim]

_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

David Abrahams
In reply to this post by Eric Jardim
Eric Jardim <[hidden email]> writes:

> Reading the tutorial,
> http://www.boost.org/libs/python/doc/tutorial/doc/html/python/embedding.htm
> it says: "To be able to use embedding in your programs, they have to be
> linked to both Boost.Python's and Python's static link library."

That's wrong.

> Is it really necessary to both the libs be static?

Nope.

http://mail.python.org/pipermail/c++-sig/2002-December/003145.html
http://mail.python.org/pipermail/c++-sig/2005-March/008703.html

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com

_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

Joel de Guzman
David Abrahams wrote:

> Eric Jardim <[hidden email]> writes:
>
>
>>Reading the tutorial,
>>http://www.boost.org/libs/python/doc/tutorial/doc/html/python/embedding.htm
>>it says: "To be able to use embedding in your programs, they have to be
>>linked to both Boost.Python's and Python's static link library."
>
>
> That's wrong.
>
>
>>Is it really necessary to both the libs be static?
>
>
> Nope.
>
> http://mail.python.org/pipermail/c++-sig/2002-December/003145.html
> http://mail.python.org/pipermail/c++-sig/2005-March/008703.html

I'm not sure how to fix (update) the text. I am not the author
of the embedding part of the tutorial. If someone can provide
me with an updated content based on Dave's links above, I'd
gladly update the tutorial.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net
_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

Eric Jardim
In reply to this post by David Abrahams
2005/8/31, David Abrahams <[hidden email]>:
That's wrong.
Nope.

Taking a shift from this moment of "clarification" of the docs, is this claim that PyFinalize() does not work with Boost.Python, still valid?

I call it and nothing (like crashes) happens. I read about memory leaks, is that right? But not calling it, will prevent the leaks?

Thanks,

[Eric Jardim]

_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

David Abrahams
Eric Jardim <[hidden email]> writes:

> 2005/8/31, David Abrahams <[hidden email]>:
>>
>> That's wrong.
>> Nope.
>>
>
> Taking a shift from this moment of "clarification" of the docs, is this
> claim that PyFinalize() does not work with Boost.Python, still valid?

Yep, AFAIK.

> I call it and nothing (like crashes) happens.

Pure luck.

> I read about memory leaks, is
> that right? But not calling it, will prevent the leaks?

No, the problem is that if you call it, Boost.Python will still be
holding references to deallocated Python objects in C++ objects.
Those reference counts will be decremented again when those C++
objects are destroyed, stomping on memory and quite possibly calling
functions in the Python interpreter.

--
Dave Abrahams
Boost Consulting
www.boost-consulting.com

_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [C++-sig] Boost.Python embedded doubt

Eric Jardim
In reply to this post by Eric Jardim
2005/8/31, Eric Jardim <[hidden email]>:
I call it and nothing (like crashes) happens. I read about memory leaks, is that right? But not calling it, will prevent the leaks?

Big mouth, mine. I got a crash. I holded a global reference to a python::object and it crashed.

[Eric Jardim]

_______________________________________________
C++-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/c++-sig
Loading...