Re: [C++-sig] Boost 1.69 python3 module

Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list

Hi Alexander,


On 2019-01-28 9:02 p.m., Alexander Biddulph wrote:

Hopefully this is the right list for this.


It is, though the issue may be more related to the Boost build system (b2 / Boost.Build), so I'm cross-posting to try to get their attention, too.



I have been struggling to get Boost 1.69 to build Boost.Python against Python 3.6.3.

I have specified the location of the python binary, include, and library (due to a non-standard install location) in the project-config.jam file in the Boost source directory.

Running bjam and specifying  "--debug-configuration" results in "[python-cfg]" output indicating that boost has found my Python installation and is satisfied with it configuration, it also indicates that it found numpy.

notice: [python-cfg] Configuring python...
notice: [python-cfg]   user-specified version: "3.6"
notice: [python-cfg]   user-specified cmd-or-prefix: "/nubots/toolchain"
notice: [python-cfg] Checking interpreter command "/nubots/toolchain/bin/python3.6"...
notice: [python-cfg] running command '/nubots/toolchain/bin/python3.6 -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...requested configuration matched!
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "/nubots/toolchain/bin/python3.6"
notice: [python-cfg]   include path: "/nubots/toolchain/include/python3.6"
notice: [python-cfg]   library path: "/nubots/toolchain/lib/python3.6/config" "/nubots/toolchain/lib"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command '/nubots/toolchain/bin/python3.6 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy enabled
notice: [python-cfg] Configuring python...
notice: [python-cfg]   user-specified version: "3.6"
notice: [python-cfg]   user-specified cmd-or-prefix: "/nubots/toolchain/bin/python3"
notice: [python-cfg]   user-specified includes: "/nubots/toolchain/include/python3.6m"
notice: [python-cfg]   user-specified libraries: "/nubots/toolchain/lib"
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "/nubots/toolchain/bin/python3"
notice: [python-cfg]   include path: "/nubots/toolchain/include/python3.6m"
notice: [python-cfg]   library path: "/nubots/toolchain/lib"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command '/nubots/toolchain/bin/python3 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy enabled

During the component configuration output it says "- python                   : building".


OK, so far all looks good / expected.



Scanning through the build output there are numerous lines where boost.python header files are copied (lines starting with common.copy). However, the libboost_python appears to never be built or installed and there is no output indicating that there was an error with any stage of the build.


Can you please attach the full output of your build ? (Perhaps even rerun with `-d2` to make the output more verbose ?)



It looks as though all other Boost modules/libraries are built, just not Boost.Python. What is going on here?


Thanks,



Stefan
--

      ...ich hab' noch einen Koffer in Berlin...
    

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list

Hi,

Finally managed to get around to rebuilding this with "-d2" added to the command line. Here is a link to the build output
https://drive.google.com/open?id=1tlwj3-uGd6wsZ5qJcrD4Ms4RbaVMbDmC

As before, everything seems to be built and installed, except for the python library.
As before, Boost.Build seems to find the python installation and says it is going to build the python library, but just doesn't seem to get around to doing it and there don't appear to be any errors that the build system encountered.



On 30/1/19 1:08 am, stefan wrote:

Hi Alexander,


On 2019-01-28 9:02 p.m., Alexander Biddulph wrote:

Hopefully this is the right list for this.


It is, though the issue may be more related to the Boost build system (b2 / Boost.Build), so I'm cross-posting to try to get their attention, too.



I have been struggling to get Boost 1.69 to build Boost.Python against Python 3.6.3.

I have specified the location of the python binary, include, and library (due to a non-standard install location) in the project-config.jam file in the Boost source directory.

Running bjam and specifying  "--debug-configuration" results in "[python-cfg]" output indicating that boost has found my Python installation and is satisfied with it configuration, it also indicates that it found numpy.

notice: [python-cfg] Configuring python...
notice: [python-cfg]   user-specified version: "3.6"
notice: [python-cfg]   user-specified cmd-or-prefix: "/nubots/toolchain"
notice: [python-cfg] Checking interpreter command "/nubots/toolchain/bin/python3.6"...
notice: [python-cfg] running command '/nubots/toolchain/bin/python3.6 -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...requested configuration matched!
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "/nubots/toolchain/bin/python3.6"
notice: [python-cfg]   include path: "/nubots/toolchain/include/python3.6"
notice: [python-cfg]   library path: "/nubots/toolchain/lib/python3.6/config" "/nubots/toolchain/lib"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command '/nubots/toolchain/bin/python3.6 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy enabled
notice: [python-cfg] Configuring python...
notice: [python-cfg]   user-specified version: "3.6"
notice: [python-cfg]   user-specified cmd-or-prefix: "/nubots/toolchain/bin/python3"
notice: [python-cfg]   user-specified includes: "/nubots/toolchain/include/python3.6m"
notice: [python-cfg]   user-specified libraries: "/nubots/toolchain/lib"
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "/nubots/toolchain/bin/python3"
notice: [python-cfg]   include path: "/nubots/toolchain/include/python3.6m"
notice: [python-cfg]   library path: "/nubots/toolchain/lib"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command '/nubots/toolchain/bin/python3 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy enabled

During the component configuration output it says "- python                   : building".


OK, so far all looks good / expected.



Scanning through the build output there are numerous lines where boost.python header files are copied (lines starting with common.copy). However, the libboost_python appears to never be built or installed and there is no output indicating that there was an error with any stage of the build.


Can you please attach the full output of your build ? (Perhaps even rerun with `-d2` to make the output more verbose ?)



It looks as though all other Boost modules/libraries are built, just not Boost.Python. What is going on here?


Thanks,



Stefan
--

      ...ich hab' noch einen Koffer in Berlin...
    

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list
In reply to this post by Boost - Build mailing list


On 2019-02-04 8:54 p.m., Alexander Biddulph wrote:

Hi,

Finally managed to get around to rebuilding this with "-d2" added to the command line. Here is a link to the build output
https://drive.google.com/open?id=1tlwj3-uGd6wsZ5qJcrD4Ms4RbaVMbDmC

As before, everything seems to be built and installed, except for the python library.
As before, Boost.Build seems to find the python installation and says it is going to build the python library, but just doesn't seem to get around to doing it and there don't appear to be any errors that the build system encountered.

I don't see anything obvious either. Steve, can you help with this ?

Thanks,


Stefan
--

      ...ich hab' noch einen Koffer in Berlin...
    

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list
AMDG

On 2/6/19 7:22 AM, stefan via Boost-build wrote:

>
> On 2019-02-04 8:54 p.m., Alexander Biddulph wrote:
>>
>> Hi,
>>
>> Finally managed to get around to rebuilding this with "-d2" added to
>> the command line. Here is a link to the build output
>> https://drive.google.com/open?id=1tlwj3-uGd6wsZ5qJcrD4Ms4RbaVMbDmC
>>
>> As before, everything seems to be built and installed, except for the
>> python library.
>> As before, Boost.Build seems to find the python installation and says
>> it is going to build the python library, but just doesn't seem to get
>> around to doing it and there don't appear to be any errors that the
>> build system encountered.
>>
> I don't see anything obvious either. Steve, can you help with this ?
>

Try `--with-python --debug-building`  (--debug-building
generates way too much output if you build everything)
I suspect that you'll see "<build>no in common properties"
My best guess is that the problem is in python.require-py.
Maybe run under the debugger and set a breakpoint there.

In Christ,
Steven Watanabe
_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list

Alexander,

I'm replying by re-adding some addresses, just so you don't miss Steven's reply below. Can you please try that and report back what you get ?

On 2019-02-06 12:13 p.m., Steven Watanabe via Boost-build wrote:
AMDG

On 2/6/19 7:22 AM, stefan via Boost-build wrote:
On 2019-02-04 8:54 p.m., Alexander Biddulph wrote:
Hi,

Finally managed to get around to rebuilding this with "-d2" added to
the command line. Here is a link to the build output
https://drive.google.com/open?id=1tlwj3-uGd6wsZ5qJcrD4Ms4RbaVMbDmC

As before, everything seems to be built and installed, except for the
python library.
As before, Boost.Build seems to find the python installation and says
it is going to build the python library, but just doesn't seem to get
around to doing it and there don't appear to be any errors that the
build system encountered.

I don't see anything obvious either. Steve, can you help with this ?

Try `--with-python --debug-building`  (--debug-building
generates way too much output if you build everything)
I suspect that you'll see "<build>no in common properties"
My best guess is that the problem is in python.require-py.
Maybe run under the debugger and set a breakpoint there.

In Christ,
Steven Watanabe
_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build

Stefan
--

      ...ich hab' noch einen Koffer in Berlin...
    

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list

OK, I scrubbed the build folder and ran again. Here is a link to the output with -d2 --debug-configuration --with-python --debug-building on the command line (in that order)
https://drive.google.com/open?id=1hINO1jHr5ikkVQ4o3EDexGGhhtpy2RTe


On 7/2/19 4:26 am, stefan wrote:

Alexander,

I'm replying by re-adding some addresses, just so you don't miss Steven's reply below. Can you please try that and report back what you get ?

On 2019-02-06 12:13 p.m., Steven Watanabe via Boost-build wrote:
AMDG

On 2/6/19 7:22 AM, stefan via Boost-build wrote:
On 2019-02-04 8:54 p.m., Alexander Biddulph wrote:
Hi,

Finally managed to get around to rebuilding this with "-d2" added to
the command line. Here is a link to the build output
https://drive.google.com/open?id=1tlwj3-uGd6wsZ5qJcrD4Ms4RbaVMbDmC

As before, everything seems to be built and installed, except for the
python library.
As before, Boost.Build seems to find the python installation and says
it is going to build the python library, but just doesn't seem to get
around to doing it and there don't appear to be any errors that the
build system encountered.

I don't see anything obvious either. Steve, can you help with this ?

Try `--with-python --debug-building`  (--debug-building
generates way too much output if you build everything)
I suspect that you'll see "<build>no in common properties"
My best guess is that the problem is in python.require-py.
Maybe run under the debugger and set a breakpoint there.

In Christ,
Steven Watanabe
_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build

Stefan
--

      ...ich hab' noch einen Koffer in Berlin...
    

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list
AMDG

On 2/6/19 4:21 PM, Alexander Biddulph via Boost-build wrote:
> OK, I scrubbed the build folder and ran again. Here is a link to the output with -d2 --debug-configuration --with-python --debug-building on the command line (in that order)

It looks like python was built in that run.
I also noticed:
  project-config.jam:33: syntax error at EOF
which didn't appear in the previous log.
In addition, the new log shows configuration
of python 2.7 instead of python 3.6.

In Christ,
Steven Watanabe
_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list
In reply to this post by Boost - Build mailing list
> That's really strange.  I have no idea why that would happen.
> There's definitely some kind of hidden global state somewhere.
> That mostly confirms my suspicion that the offending rule is
> python.require-py.  Maybe if you add /debug/  into the call to
> main-target.select-alternatives, it will reveal something:
>
>     local py-ext-alternative = [ $(py-ext-target).select-alternatives
> $(property-set) debug ] ;

This also results in a successful building and installation of the Boost.Python library (along with all the others). Here is the build output
https://drive.google.com/open?id=1jNBu73JvFwcs8z0Om_o7bJ68x0KXCU4U

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list
AMDG

On 2/12/19 3:31 PM, Alex Biddulph via Boost-build wrote:

>> That's really strange.  I have no idea why that would happen.
>> There's definitely some kind of hidden global state somewhere.
>> That mostly confirms my suspicion that the offending rule is
>> python.require-py.  Maybe if you add /debug/  into the call to
>> main-target.select-alternatives, it will reveal something:
>>
>>     local py-ext-alternative = [ $(py-ext-target).select-alternatives
>> $(property-set) debug ] ;
>
> This also results in a successful building and installation of the Boost.Python library (along with all the others). Here is the build output
> https://drive.google.com/open?id=1jNBu73JvFwcs8z0Om_o7bJ68x0KXCU4U
>

That makes no sense to me.  The only thing that should be
affected by this change is a few ECHO calls.  Are you
sure that everything else is the same?  I see a new error:
/nubots/toolchain/bin/python3: error while loading shared libraries:
libpython3.6m.so.1.0: cannot open shared object file: No such file or
directory

In Christ,
Steven Watanabe
_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list
Not sure what happened there, I must have dirtied my build environment.

I cleaned everything up and ran it again. Here is the output.
https://drive.google.com/open?id=1u_8KHpjuAC2Cm0jGrVOdGvaqxFtslhNm

On 16/2/19 5:39 am, Steven Watanabe via Boost-build wrote:

> AMDG
>
> On 2/12/19 3:31 PM, Alex Biddulph via Boost-build wrote:
>>> That's really strange.  I have no idea why that would happen.
>>> There's definitely some kind of hidden global state somewhere.
>>> That mostly confirms my suspicion that the offending rule is
>>> python.require-py.  Maybe if you add /debug/  into the call to
>>> main-target.select-alternatives, it will reveal something:
>>>
>>>      local py-ext-alternative = [ $(py-ext-target).select-alternatives
>>> $(property-set) debug ] ;
>> This also results in a successful building and installation of the Boost.Python library (along with all the others). Here is the build output
>> https://drive.google.com/open?id=1jNBu73JvFwcs8z0Om_o7bJ68x0KXCU4U
>>
> That makes no sense to me.  The only thing that should be
> affected by this change is a few ECHO calls.  Are you
> sure that everything else is the same?  I see a new error:
> /nubots/toolchain/bin/python3: error while loading shared libraries:
> libpython3.6m.so.1.0: cannot open shared object file: No such file or
> directory
>
> In Christ,
> Steven Watanabe
> _______________________________________________
> Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: [C++-sig] Boost 1.69 python3 module

Boost - Build mailing list
AMDG

On 2/18/19 9:09 PM, Alex Biddulph wrote:
> Not sure what happened there, I must have dirtied my build environment.
>
> I cleaned everything up and ran it again. Here is the output.
> https://drive.google.com/open?id=1u_8KHpjuAC2Cm0jGrVOdGvaqxFtslhNm
>

The problem is that you are configuring python twice.
Please make sure that you only have one `using python`
statement in your project-config.jam (or any other
configuration file).  This is a really obscure failure
mode and I'll look into making a better message.  You
might file a ticket at https://github.com/boostorg/build/

In Christ,
Steven Watanabe
_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build