error building Boost.Python for multiple Python versions.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

error building Boost.Python for multiple Python versions.

Boost - Build mailing list

Hello,

looking into https://github.com/boostorg/python/issues/129, I think I understand the underlying issue, which, I think, is a Boost.Build issue. After adding

using python : 2.7 ;
using python : 3.5 : python3 : /usr/include/python3.5m : /usr/lib64/libpython3.5m.so ;

to my user-config.jam file, and running `./b2 --with-python -d2`, I observe the following output:


...

gcc.compile.c++ bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/list.o

    "g++"   -O3 -finline-functions -Wno-inline -Wall -pthread -m64  -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DBOOST_PYTHON_STATIC_LIB -DNDEBUG  -I"." -I"/usr/include/python2.7" -c -o "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/list.o" "libs/python/src/list.cpp"
...

gcc.archive bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/libboost_python.a

    "ar"  rc "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/libboost_python.a" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/list.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/long.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/dict.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/tuple.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/str.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/slice.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/from_python.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/registry.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/type_id.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/enum.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/class.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/function.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/inheritance.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/life_support.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/pickle_support.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/errors.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/module.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/builtin_converters.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/arg_to_python_base.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/iterator.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/stl_iterator.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object_protocol.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object_operators.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/wrapper.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/import.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/exec.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/function_doc_signature.o"

...

gcc.archive bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/libboost_python3.a

    "ar"  rc "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/libboost_python3.a" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/list.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/long.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/dict.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/tuple.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/str.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/slice.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/from_python.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/registry.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/type_id.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/enum.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/class.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/function.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/inheritance.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/life_support.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/pickle_support.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/errors.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/module.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/builtin_converters.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/converter/arg_to_python_base.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/iterator.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/stl_iterator.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object_protocol.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object_operators.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/wrapper.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/import.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/exec.o" "bin.v2/libs/python/build/gcc-gnu-7/release/link-static/threadapi-pthread/threading-multi/object/function_doc_signature.o"
...


Notice how the "link.cpp" source file (tracing an arbitrary source file from Boost.Python) gets compiled once (against Python 2.7 headers), then linked twice, once for libboost_python (i.e. Python 2.7), and once for libboost_python3 (i.e. Python 3.5).

This is clearly wrong. If Boost.Python is compiled for multiple Python versions in the same process, the build tree needs to differentiate between the intermediate object files the same way it differentiates between objects compiled with different toolchains. Instead, Boost.Build gets confused and links two Boost.Python libraries from the same object files.

I'm normally not building for multiple Python versions in the same process (and neither for multiple toolchains), but as the script that generates Windows binaries uses that process, the generated binaries are wrong.

What needs to happen to fix this ? My personal advice would be to build for one Python version at a time (even if the resulting binaries get bundled into the same package). To support what some believe to be an important use-case, the internal Boost.Build logic needs to be augmented to add a <python-version> component to the build path, to make sure objects get rebuild properly for each additional Python version.


Thoughts ?


Stefan
-- 

      ...ich hab' noch einen Koffer in Berlin...
    

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build