Problem with importing shared library into Python

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with importing shared library into Python

Edgardo C.
Hello everybody,

I am having problems with importing a .so shared library I just compiled following the example giving in the book:
API Design for C++ : www.apibook.com (Chapter 11)

I downloaded the Boost library and I compiled it (fullw) into a subdirectory called /compilation, where I get the /include and /lib directories. No error during the compilation (Ubuntu 11.10).
Then I downloaded the code from the book examples and I compiled it without errors and I get the phonebook.so in the same directory as where I launch python interpreter, but the import in python trigger this error:


ImportError: /home/pablo/phonebook/phonebook.so: undefined symbol: _ZN5boost6python6detail11init_moduleEPKcPFvvE

what I think is because python does not find the library.

I try to play around with the LD_LIBRARY_PATH variable, ldconfig, I copied the phonebook.so library into /usr/lib and /usr/local/lib, and nothing ... I get the same error.

Can anybody explain me what is the problem? thanks a lot in advance,

N.B: the version of python used during the boost compilation and .cpp compilation are the same, 2.7

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: Problem with importing shared library into Python

Jim Bosch-2
On 11/25/2011 06:54 AM, Edgardo C. wrote:

> Hello everybody,
>
> I am having problems with importing a .so shared library I just compiled
> following the example giving in the book:
> API Design for C++ : www.apibook.com <http://www.apibook.com> (Chapter 11)
>
> I downloaded the Boost library and I compiled it (fullw) into a
> subdirectory called /compilation, where I get the /include and /lib
> directories. No error during the compilation (Ubuntu 11.10).
> Then I downloaded the code from the book examples and I compiled it
> without errors and I get the phonebook.so in the same directory as where
> I launch python interpreter, but the import in python trigger this error:
>
>
> ImportError: /home/pablo/phonebook/phonebook.so: undefined symbol:
> _ZN5boost6python6detail11init_moduleEPKcPFvvE
>
> what I think is because python does not find the library.
>
> I try to play around with the LD_LIBRARY_PATH variable, ldconfig, I
> copied the phonebook.so library into /usr/lib and /usr/local/lib, and
> nothing ... I get the same error.
>
> Can anybody explain me what is the problem? thanks a lot in advance,
>

I think you just need to link your library against the boost_python
library, and make sure that's also somewhere on your dynamic library
path.  With gcc, that means adding a "-lboost_python" to the command
line options.

Jim

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: Problem with importing shared library into Python

Edgardo C.
Hello Jim,

I really appreciate your feedback. I did what you mention, and I did it downloading manually the boost library, compiling, etc, etc. Then, I thought, ok, maybe I did something wrong and I downloaded the already compiled boost packages using the synaptic tool in Ubuntu and then I compiled the source code against the Ubuntu boost isntalled packages. The compilation works fine, no errors but I get the same error.

What I do not know is whether the problem is the compilation of the shared library or python. I spent the whole day on this, agrrrrrrrrrrrrrrrrrrrrrrrrr

this guy :

http://www.mentby.com/vivek-60/


had a similar problem. But, this command:

$ nm /usr/local/lib/libboost_python3.so.1.46.1 | c++filt | grep init_module

show no symbols for me. Any other idea?

 

2011/11/25 Jim Bosch <[hidden email]>
On 11/25/2011 06:54 AM, Edgardo C. wrote:
Hello everybody,

I am having problems with importing a .so shared library I just compiled
following the example giving in the book:
API Design for C++ : www.apibook.com <http://www.apibook.com> (Chapter 11)


I downloaded the Boost library and I compiled it (fullw) into a
subdirectory called /compilation, where I get the /include and /lib
directories. No error during the compilation (Ubuntu 11.10).
Then I downloaded the code from the book examples and I compiled it
without errors and I get the phonebook.so in the same directory as where
I launch python interpreter, but the import in python trigger this error:


ImportError: /home/pablo/phonebook/phonebook.so: undefined symbol:
_ZN5boost6python6detail11init_moduleEPKcPFvvE

what I think is because python does not find the library.

I try to play around with the LD_LIBRARY_PATH variable, ldconfig, I
copied the phonebook.so library into /usr/lib and /usr/local/lib, and
nothing ... I get the same error.

Can anybody explain me what is the problem? thanks a lot in advance,


I think you just need to link your library against the boost_python library, and make sure that's also somewhere on your dynamic library path.  With gcc, that means adding a "-lboost_python" to the command line options.

Jim

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig


_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: Problem with importing shared library into Python

Edgardo C.
In reply to this post by Jim Bosch-2
We have this situation(remember that the source code is from the book API C++ design www.apibook.com):

Source code:

- phonebook.h
- phonebook.cpp
- phonebook_wrap.cpp

Boost is download, compiled and installed into:
- /Desktop/boost_1_48_0/compilation/

which means that /include and /lib are the subfolders of this with all the headers and libraries into it.

Now:

g++ -c phonebook.cpp                                                                    (compiles only phonebook.cpp, without error)
g++ -c phonebook_wrap.cpp -I<boost_includes> -I<python_include>  (compiles the wrapper against the /include subfolder mentioned before and the python2.7 headers in /usr/...)
g++ -shared -o phonebook.so phonebook.o phonebook_wrap.o -lboost_python -lpython (links the objects agains the /lib subfolder mentioned before and and python2.7 libraries in /usr/...)

After this I get the phonebook.so shared library, but when I try to import it in python I get the error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: ./phonebook.so: undefined symbol: _ZN5boost6python6detail11init_moduleEPKcPFvvE


Hope this can help better to understand (maybe) what is going on. Again, I follow the same procedure after installing the precompiled packages from Ubuntu using synaptic, and the same error. Hope that somebody can help.




















2011/11/25 Jim Bosch <[hidden email]>
On 11/25/2011 06:54 AM, Edgardo C. wrote:
Hello everybody,

I am having problems with importing a .so shared library I just compiled
following the example giving in the book:
API Design for C++ : www.apibook.com <http://www.apibook.com> (Chapter 11)


I downloaded the Boost library and I compiled it (fullw) into a
subdirectory called /compilation, where I get the /include and /lib
directories. No error during the compilation (Ubuntu 11.10).
Then I downloaded the code from the book examples and I compiled it
without errors and I get the phonebook.so in the same directory as where
I launch python interpreter, but the import in python trigger this error:


ImportError: /home/pablo/phonebook/phonebook.so: undefined symbol:
_ZN5boost6python6detail11init_moduleEPKcPFvvE

what I think is because python does not find the library.

I try to play around with the LD_LIBRARY_PATH variable, ldconfig, I
copied the phonebook.so library into /usr/lib and /usr/local/lib, and
nothing ... I get the same error.

Can anybody explain me what is the problem? thanks a lot in advance,


I think you just need to link your library against the boost_python library, and make sure that's also somewhere on your dynamic library path.  With gcc, that means adding a "-lboost_python" to the command line options.

Jim

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig


_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: Problem with importing shared library into Python

Stefan Seefeld-2
On 11/25/2011 10:06 AM, Edgardo C. wrote:

> We have this situation(remember that the source code is from the book
> API C++ design www.apibook.com <http://www.apibook.com>):
>
> Source code:
>
> - phonebook.h
> - phonebook.cpp
> - phonebook_wrap.cpp
>
> Boost is download, compiled and installed into:
> - /Desktop/boost_1_48_0/compilation/
>
> which means that /include and /lib are the subfolders of this with all
> the headers and libraries into it.
>
> Now:
>
> g++ -c
> phonebook.cpp                                                                  
> (compiles only phonebook.cpp, without error)
> g++ -c phonebook_wrap.cpp -I<boost_includes> -I<python_include>
> (compiles the wrapper against the /include subfolder mentioned before
> and the python2.7 headers in /usr/...)
> g++ -shared -o phonebook.so phonebook.o phonebook_wrap.o
> -lboost_python -lpython (links the objects agains the /lib subfolder
> mentioned before and and python2.7 libraries in /usr/...)

Not sure whether this has anything to do with the symptoms you are
observing, but I believe you should not link to the Python runtime, as
that will be provided by the interpreter. (Some exotic platforms such as
cygwin seem to require to link against -lpython anyhow, but certainly
not GNU/Linux)

FWIW,
        Stefan

--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: Problem with importing shared library into Python

Anders Wallin-2
In reply to this post by Edgardo C.
> Hope this can help better to understand (maybe) what is going on. Again, I
> follow the same procedure after installing the precompiled packages from
> Ubuntu using synaptic, and the same error. Hope that somebody can help.

FWIW with this CMakeLists.txt the example-code from the book compiles
and runs fine on my Ubuntu 11.10 system:
http://pastebin.com/YrWVEmxX

If I run make with "make VERBOSE=1" it gives the following output
(this might help if you don't want to use cmake...):

/usr/bin/c++   -Dphonebook_EXPORTS -fPIC -I/usr/include/python2.7
-o CMakeFiles/phonebook.dir/phonebook.o -c
/home/anders/Desktop/apibook-apibook-tip/11_Scripting/python/phonebook.cpp

/usr/bin/c++   -Dphonebook_EXPORTS -fPIC -I/usr/include/python2.7
-o CMakeFiles/phonebook.dir/phonebook_wrap.o -c
/home/anders/Desktop/apibook-apibook-tip/11_Scripting/python/phonebook_wrap.cpp

/usr/bin/c++  -fPIC    -shared -Wl,-soname,phonebook.so -o
phonebook.so CMakeFiles/phonebook.dir/phonebook.o
CMakeFiles/phonebook.dir/phonebook_wrap.o -lboost_python


hth,
Anders
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: Problem with importing shared library into Python

Edgardo C.
Hello to all again and I really appreciate the help. Finally it works now !!!!!!!!!!!!!!!

Unfortunately i happened just 10 minutes before leaving the office and until Monday I will not touch again the code. 
@Anders: I will try what you suggest on Monday.


What I did was:

$ g++ -c phonebook_wrap.cpp -I/usr/include/python2.7
$ g++ -shared  -o phonebook.so phonebook.o phonebook_wrap.o -lboost_python -L/usr/bin/python.27

Previously I used variables to access the folder (header and libraries), now I use the folder explicitly. Now I am compiling against the boot libraries installed using synaptic and against the python executable located in /usr/bin/python.27.
Honestly I expected to compiled against -lpython, but there is not python library like "libpython ... .so":

$ g++ -shared  -o phonebook.so phonebook.o phonebook_wrap.o -lboost_python -lpython
/usr/bin/ld: cannot find -lpython
collect2: ld returned 1 exit status


I compiled against the executable, which is an ELF file and shared library as well, as noticed after running "file" command and it worked perfectly. Now I can import the phonebook.so from Python and run all the code. I still want to review all what I did again because there is something that is not clear what went wrong. Until Monday I would not be able to touch the code again, but thanks a lot for all your suggestions, I really appreciate it.
Keep you informed on Monday.

Have a nice week end to you all,
Cheers,




 

 

2011/11/25 Anders Wallin <[hidden email]>
> Hope this can help better to understand (maybe) what is going on. Again, I
> follow the same procedure after installing the precompiled packages from
> Ubuntu using synaptic, and the same error. Hope that somebody can help.

FWIW with this CMakeLists.txt the example-code from the book compiles
and runs fine on my Ubuntu 11.10 system:
http://pastebin.com/YrWVEmxX

If I run make with "make VERBOSE=1" it gives the following output
(this might help if you don't want to use cmake...):

/usr/bin/c++   -Dphonebook_EXPORTS -fPIC -I/usr/include/python2.7
-o CMakeFiles/phonebook.dir/phonebook.o -c
/home/anders/Desktop/apibook-apibook-tip/11_Scripting/python/phonebook.cpp

/usr/bin/c++   -Dphonebook_EXPORTS -fPIC -I/usr/include/python2.7
-o CMakeFiles/phonebook.dir/phonebook_wrap.o -c
/home/anders/Desktop/apibook-apibook-tip/11_Scripting/python/phonebook_wrap.cpp

/usr/bin/c++  -fPIC    -shared -Wl,-soname,phonebook.so -o
phonebook.so CMakeFiles/phonebook.dir/phonebook.o
CMakeFiles/phonebook.dir/phonebook_wrap.o -lboost_python


hth,
Anders
_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig


_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Reply | Threaded
Open this post in threaded view
|

Re: Problem with importing shared library into Python

Stefan Seefeld-2
On 11/25/2011 02:51 PM, Edgardo C. wrote:
> Hello to all again and I really appreciate the help. Finally it works
> now !!!!!!!!!!!!!!!

Good !

>
> Unfortunately i happened just 10 minutes before leaving the office and
> until Monday I will not touch again the code.
> @Anders: I will try what you suggest on Monday.
>
>
> What I did was:
>
> $ g++ -c phonebook_wrap.cpp -I/usr/include/python2.7
> $ g++ -shared  -o phonebook.so phonebook.o phonebook_wrap.o
> -lboost_python -L/usr/bin/python.27

That last argument ("-L/usr/bin/python.27") doesn't make any sense. You
should remove it.

>
> Previously I used variables to access the folder (header and
> libraries), now I use the folder explicitly. Now I am compiling
> against the boot libraries installed using synaptic and against the
> python executable located in /usr/bin/python.27.
> Honestly I expected to compiled against -lpython, but there is not
> python library like "libpython ... .so":

As I said in my last mail, you shouldn't need to link against the Python
runtime itself when building extension modules, as it will already be
loaded by the interpreter.


>
> $ g++ -shared  -o phonebook.so phonebook.o phonebook_wrap.o
> -lboost_python -lpython
> /usr/bin/ld: cannot find -lpython
> collect2: ld returned 1 exit status
>
>
> I compiled against the executable, which is an ELF file and shared
> library as well, as noticed after running "file" command and it worked
> perfectly.

That doesn't make sense. You can't link against an executable. The link
command likely succeeded simply because it already found all symbols it
needed without that. Use `ldd phonebook.so` to see what libraries your
module depends on.


Regards,
        Stefan

--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Cplusplus-sig mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/cplusplus-sig