Boost › Boost - Testing

My master regression runners failing

‹ Previous Topic Next Topic ›
 
classic Classic list List threaded Threaded
♦
♦
 8 messages Options
Boost - Testing mailing list
Reply | Threaded
Open this post in threaded view
♦
♦
|

My master regression runners failing

Boost - Testing mailing list
So sometime on Dec 27, my test runners for master stopped running. It appears to be fine for other runners, so I guess it is something in my setup, but after hunting around for a while I don't have any idea what....

The end result that I'm seeing is a failure of process_jam_log to build. 

Here's part of of the output from run.py/regression.py (started from my own python scripts):

# Searching for "b2" in "/var/boost/run/boost_bb/src/engine"...
# b2 succesfully built in "/var/boost/run/boost_bb/src/engine/bin.linuxx86_64/b2" location
# Preinstalled "/var/boost/run/boost_regression/stage/bin/process_jam_log" not found; building one...
# Found "process_jam_log" source directory "/var/boost/run/boost_regression/testing/build"
# Building "process_jam_log" ("/var/boost/run/boost_bb/src/engine/bin.linuxx86_64/b2" "-sBOOST_BUILD_PATH=/var/boost/run:/var/boost/run/boost_bb/src" "-sBOOST_ROOT=/var/boost/run/boost_root" "--boost=/var/boost/run/boost_root" "--boost-root=/var/boost/run/boost_root" "--boost-build=/var/boost/run/boost_bb/src" "--debug-configuration" install -d2 -l300)...
notice: found boost-build.jam at /var/boost/run/boost_regression/boost-build.jam
notice: loading Boost.Build from /var/boost/run/boost_bb/src
notice: Searching '/etc' '/home/boost' '/var/boost/run/boost_bb/src' '/var/boost/run' '/var/boost/run/boost_bb/src' '/var/boost/run/boost_bb/src/kernel' '/var/boost/run/boost_bb/src/util' '/var/boost/run/boost_bb/src/build' '/var/boost/run/boost_bb/src/tools' '/var/boost/run/boost_bb/src/contrib' '/var/boost/run/boost_bb/src/.' for site-config configuration file 'site-config.jam'.
notice: Configuration file 'site-config.jam' not found in '/etc' '/home/boost' '/var/boost/run/boost_bb/src' '/var/boost/run' '/var/boost/run/boost_bb/src' '/var/boost/run/boost_bb/src/kernel' '/var/boost/run/boost_bb/src/util' '/var/boost/run/boost_bb/src/build' '/var/boost/run/boost_bb/src/tools' '/var/boost/run/boost_bb/src/contrib' '/var/boost/run/boost_bb/src/.'.
notice: Searching '/home/boost' '/var/boost/run/boost_bb/src' '/var/boost/run' '/var/boost/run/boost_bb/src' '/var/boost/run/boost_bb/src/kernel' '/var/boost/run/boost_bb/src/util' '/var/boost/run/boost_bb/src/build' '/var/boost/run/boost_bb/src/tools' '/var/boost/run/boost_bb/src/contrib' '/var/boost/run/boost_bb/src/.' for user-config configuration file 'user-config.jam'.
notice: Loading user-config configuration file 'user-config.jam' from '/home/boost'.
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7
notice: using gcc libraries :: <toolset>gcc-7 :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7 :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7 :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:1
notice: using rc compiler :: <toolset>gcc-7 :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~c++98
notice: using gcc libraries :: <toolset>gcc-7~c++98 :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~c++98 :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~c++98 :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:2
notice: using rc compiler :: <toolset>gcc-7~c++98 :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~gnu98
notice: using gcc libraries :: <toolset>gcc-7~gnu98 :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~gnu98 :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~gnu98 :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:3
notice: using rc compiler :: <toolset>gcc-7~gnu98 :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~c++11
notice: using gcc libraries :: <toolset>gcc-7~c++11 :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~c++11 :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~c++11 :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:4
notice: using rc compiler :: <toolset>gcc-7~c++11 :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~gnu11
notice: using gcc libraries :: <toolset>gcc-7~gnu11 :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~gnu11 :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~gnu11 :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:5
notice: using rc compiler :: <toolset>gcc-7~gnu11 :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~c++14
notice: using gcc libraries :: <toolset>gcc-7~c++14 :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~c++14 :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~c++14 :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:6
notice: using rc compiler :: <toolset>gcc-7~c++14 :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~gnu14
notice: using gcc libraries :: <toolset>gcc-7~gnu14 :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~gnu14 :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~gnu14 :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:7
notice: using rc compiler :: <toolset>gcc-7~gnu14 :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~c++1z
notice: using gcc libraries :: <toolset>gcc-7~c++1z :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~c++1z :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~c++1z :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:8
notice: using rc compiler :: <toolset>gcc-7~c++1z :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~gnu1z
notice: using gcc libraries :: <toolset>gcc-7~gnu1z :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~gnu1z :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~gnu1z :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:9
notice: using rc compiler :: <toolset>gcc-7~gnu1z :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~c++1z~O2
notice: using gcc libraries :: <toolset>gcc-7~c++1z~O2 :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~c++1z~O2 :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~c++1z~O2 :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:10
notice: using rc compiler :: <toolset>gcc-7~c++1z~O2 :: /usr/bin/as
notice: will use 'g++-7' for gcc, condition <toolset>gcc-7~c++1z~warn
notice: using gcc libraries :: <toolset>gcc-7~c++1z~warn :: /usr/bin /usr/lib /usr/lib32 /usr/lib64
notice: using gcc archiver :: <toolset>gcc-7~c++1z~warn :: /usr/bin/ar
notice: using gcc ranlib :: <toolset>gcc-7~c++1z~warn :: /usr/bin/ranlib
warning: toolset gcc initialization: can not find tool windres
warning: initialized from /home/boost/user-config.jam:11
notice: using rc compiler :: <toolset>gcc-7~c++1z~warn :: /usr/bin/as
notice: [python-cfg] Configuring python...
notice: [python-cfg]   user-specified version: "2.7"
notice: [python-cfg]   user-specified includes: "/usr/include/python2.7"
notice: [python-cfg]   user-specified libraries: "/usr/lib/x86_64-linux-gnu"
notice: [python-cfg] Checking interpreter command "python2.7"...
notice: [python-cfg] running command 'python2.7 -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...requested configuration matched!
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "python2.7"
notice: [python-cfg]   include path: "/usr/include/python2.7"
notice: [python-cfg]   library path: "/usr/lib/x86_64-linux-gnu"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command 'python2.7 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy disabled. Reason:
notice: [python-cfg]   python2.7 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())" aborted with 
notice: [python-cfg]   Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named numpy
notice: [python-cfg] Configuring python...
notice: [python-cfg]   user-specified version: "3.5"
notice: [python-cfg]   user-specified includes: "/usr/include/python3.5"
notice: [python-cfg]   user-specified libraries: "/usr/lib/x86_64-linux-gnu"
notice: [python-cfg] Checking interpreter command "python3.5"...
notice: [python-cfg] running command 'python3.5 -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...requested configuration matched!
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "python3.5"
notice: [python-cfg]   include path: "/usr/include/python3.5"
notice: [python-cfg]   library path: "/usr/lib/x86_64-linux-gnu"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command 'python3.5 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy disabled. Reason:
notice: [python-cfg]   python3.5 -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())" aborted with 
notice: [python-cfg]   Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named 'numpy'
notice: [zlib] Using pre-installed library
notice: [zlib] Condition
notice: [bzip2] Using pre-installed library
notice: [bzip2] Condition
notice: [lzma] Using pre-installed library
notice: [lzma] Condition
Performing configuration checks

    - 32-bit                   : no
    - 64-bit                   : yes
    - arm                      : no
    - mips1                    : no
    - power                    : no
    - sparc                    : no
    - x86                      : yes
    - symlinks supported       : yes
error: Cannot create link ../../../boost_root/boost/functional/hash_fwd.hpp to ../../../boost_root/libs/container_hash/include/boost/functional/hash_fwd.hpp.
error: Link previously defined to another file, ../../../boost_root/libs/functional/include/boost/functional/hash_fwd.hpp.

# Searching for "process_jam_log" in "/var/boost/run/boost_regression/stage/bin"...
Traceback (most recent call last):
  File "run.py", line 71, in <module>
    runner(root)
  File "/var/boost/run/boost_regression_src/regression.py", line 286, in __init__
    self.main()
  File "/var/boost/run/boost_regression_src/regression.py", line 623, in main
    getattr(self,action_m)()
  File "/var/boost/run/boost_regression_src/regression.py", line 580, in command_regression
    self.command_setup()
  File "/var/boost/run/boost_regression_src/regression.py", line 351, in command_setup
    self.build_if_needed(self.process_jam_log,self.pjl_toolset)
  File "/var/boost/run/boost_regression_src/regression.py", line 714, in build_if_needed
    tool[ 'build_path' ] = self.tool_path( tool )
  File "/var/boost/run/boost_regression_src/regression.py", line 739, in tool_path
    , '\n'.join( [ name_or_spec[ 'path' ], build_dir ] )
Exception: Cannot find "process_jam_log" in any of the following locations:
/var/boost/run/boost_regression/stage/bin/process_jam_log
/var/boost/run/boost_regression/stage/bin

Everything looks fine until it goes to start making the symlinks (this test was run on linux, but I'm seeing the same issue on windows). For instance a good version of this on develop looks like:

# b2 succesfully built in "/var/boost/run/boost_bb/src/engine/bin.linuxx86_64/b2" location
# Preinstalled "/var/boost/run/boost_regression/stage/bin/process_jam_log" not found; building one...
# Found "process_jam_log" source directory "/var/boost/run/boost_regression/testing/build"
# Building "process_jam_log" ("/var/boost/run/boost_bb/src/engine/bin.linuxx86_64/b2" "-sBOOST_BUILD_PATH=/var/boost/run:/var/boost/run/boost_bb/src" "-sBOOST_ROOT=/var/boost/run/boost_root" "--boost=/var/boost/run/boost_root" "--boost-root=/var/boost/run/boost_root" "--boost-build=/var/boost/run/boost_bb/src" "--debug-configuration" install -d2 -l300)...
notice: found boost-build.jam at /var/boost/run/boost_regression/boost-build.jam
notice: loading Boost.Build from /var/boost/run/boost_bb/src
[snip]
notice: [lzma] Using pre-installed library
notice: [lzma] Condition
Performing configuration checks

    - 32-bit                   : no
    - 64-bit                   : yes
    - arm                      : no
    - mips1                    : no
    - power                    : no
    - sparc                    : no
    - x86                      : yes
    - symlinks supported       : yes
...found 1345 targets...
...updating 592 targets...
common.mkdir ../../bin

        mkdir -p "../../bin"
    
common.mkdir ../../bin/common

        mkdir -p "../../bin/common"
    
common.mkdir ../../bin/common/build

        mkdir -p "../../bin/common/build"
    
common.mkdir ../../bin/common/build/gcc-gnu-7

        mkdir -p "../../bin/common/build/gcc-gnu-7"
    
common.mkdir ../../bin/common/build/gcc-gnu-7/release

        mkdir -p "../../bin/common/build/gcc-gnu-7/release"
    
mklink-or-dir ../../../boost_root/boost

    mkdir "../../../boost_root/boost"

mklink-or-dir ../../../boost_root/boost/xpressive

    ln -s ../libs/xpressive/include/boost/xpressive ../../../boost_root/boost/xpressive

mklink-or-dir ../../../boost_root/boost/winapi

    ln -s ../libs/winapi/include/boost/winapi ../../../boost_root/boost/winapi
[snip]
common.copy /var/boost/run/boost_regression/stage/bin/process_jam_log

    cp "../../bin/testing/build/gcc-gnu-7/release/process_jam_log"  "/var/boost/run/boost_regression/stage/bin/process_jam_log"

...updated 592 targets...
# process_jam_log succesfully built in "/var/boost/run/boost_regression/stage/bin/process_jam_log" location
# Making "/var/boost/run/results" directory...



Any ideas why this error would be happening only on master? Different version of boost build (master vs. develop)?

Thanks,
Tom

_______________________________________________
Boost-Testing mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-testing
Boost - Testing mailing list
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: My master regression runners failing

Boost - Testing mailing list
AMDG

On 01/01/2018 08:37 AM, Tom Kent via Boost-Testing wrote:
>
> error: Cannot create link ../../../boost_root/boost/functional/hash_fwd.hpp
> to ../../../boost_root/libs/container_hash/include/boost/
> functional/hash_fwd.hpp.
> error: Link previously defined to another file, ../../../boost_root/libs/
> functional/include/boost/functional/hash_fwd.hpp.
>

  master is in an inconsistent state.  This file
was moved from functional to container_hash, and
it's currently present in both.
Solution: merge functional to master.

In Christ,
Steven Watanabe
_______________________________________________
Boost-Testing mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-testing
Boost - Testing mailing list
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: My master regression runners failing

Boost - Testing mailing list
In reply to this post by Boost - Testing mailing list
Tom Kent wrote:
...

> error: Cannot create link
> ../../../boost_root/boost/functional/hash_fwd.hpp to
> ../../../boost_root/libs/container_hash/include/boost/functional/hash_fwd.hpp.
> error: Link previously defined to another file,
> ../../../boost_root/libs/functional/include/boost/functional/hash_fwd.hpp.

This happens because on develop, this file has been moved from `functional`
to `container_hash`.

Master is untouched; `container_hash` is not present there, and the file is
still in `functional`. However, what I think happens is that this is not a
clean clone of master, but a clone of develop, which is then switched to
master with `git checkout master`. In this case, the develop modules still
remain as directories.

_______________________________________________
Boost-Testing mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-testing
Boost - Testing mailing list
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: My master regression runners failing

Boost - Testing mailing list
In reply to this post by Boost - Testing mailing list
On 1 January 2018 at 15:53, Steven Watanabe via Boost-Testing
<[hidden email]> wrote:

> AMDG
>
> On 01/01/2018 08:37 AM, Tom Kent via Boost-Testing wrote:
>>
>> error: Cannot create link ../../../boost_root/boost/functional/hash_fwd.hpp
>> to ../../../boost_root/libs/container_hash/include/boost/
>> functional/hash_fwd.hpp.
>> error: Link previously defined to another file, ../../../boost_root/libs/
>> functional/include/boost/functional/hash_fwd.hpp.
>>
>
>   master is in an inconsistent state.  This file
> was moved from functional to container_hash, and
> it's currently present in both.
> Solution: merge functional to master.

Will do, but would be better to have a proper solution. It should be
okay for a header to be in different modules on master and develop.
I'd like to be in a more stable state before merging to master.
_______________________________________________
Boost-Testing mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-testing
Boost - Testing mailing list
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: My master regression runners failing

Boost - Testing mailing list
In reply to this post by Boost - Testing mailing list
Steven Watanabe wrote:

> AMDG
>
> On 01/01/2018 08:37 AM, Tom Kent via Boost-Testing wrote:
> >
> > error: Cannot create link
> > ../../../boost_root/boost/functional/hash_fwd.hpp
> > to ../../../boost_root/libs/container_hash/include/boost/
> > functional/hash_fwd.hpp.
> > error: Link previously defined to another file,
> > ../../../boost_root/libs/
> > functional/include/boost/functional/hash_fwd.hpp.
>
>
>   master is in an inconsistent state.

No, master is consistent. If you make a clean clone of it, everything's fine
because `container_hash` isn't present.

_______________________________________________
Boost-Testing mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-testing
Boost - Testing mailing list
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: My master regression runners failing

Boost - Testing mailing list
In reply to this post by Boost - Testing mailing list
AMDG

On 01/01/2018 08:58 AM, Peter Dimov via Boost-Testing wrote:

> Tom Kent wrote:
> ...
>
>> error: Cannot create link
>> ../../../boost_root/boost/functional/hash_fwd.hpp to
>> ../../../boost_root/libs/container_hash/include/boost/functional/hash_fwd.hpp.
>>
>> error: Link previously defined to another file,
>> ../../../boost_root/libs/functional/include/boost/functional/hash_fwd.hpp.
>>
>
> This happens because on develop, this file has been moved from
> `functional` to `container_hash`.
>
> Master is untouched; `container_hash` is not present there, and the file
> is still in `functional`. However, what I think happens is that this is
> not a clean clone of master, but a clone of develop, which is then
> switched to master with `git checkout master`. In this case, the develop
> modules still remain as directories.
>

regression.py tries to fix up the tree, but it seems
that it doesn't handle the case of removing a submodule.

https://github.com/boostorg/regression/blob/develop/testing/src/regression.py#L852

I don't know git well enough to know what the correct command
would be.

In Christ,
Steven Watanabe
_______________________________________________
Boost-Testing mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-testing
Boost - Testing mailing list
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: My master regression runners failing

Boost - Testing mailing list
On 1 January 2018 at 16:18, Steven Watanabe via Boost-Testing
<[hidden email]> wrote:
>
> regression.py tries to fix up the tree, but it seems
> that it doesn't handle the case of removing a submodule.
>
> https://github.com/boostorg/regression/blob/develop/testing/src/regression.py#L852
>
> I don't know git well enough to know what the correct command
> would be.

According to the man page you need to use '-f' twice with 'git clean'.
_______________________________________________
Boost-Testing mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-testing
Boost - Testing mailing list
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: My master regression runners failing

Boost - Testing mailing list


On Mon, Jan 1, 2018 at 10:22 AM, Daniel James via Boost-Testing <[hidden email]> wrote:
On 1 January 2018 at 16:18, Steven Watanabe via Boost-Testing
<[hidden email]> wrote:
>
> regression.py tries to fix up the tree, but it seems
> that it doesn't handle the case of removing a submodule.
>
> https://github.com/boostorg/regression/blob/develop/testing/src/regression.py#L852
>
> I don't know git well enough to know what the correct command
> would be.

According to the man page you need to use '-f' twice with 'git clean'.

Thanks all, I added a `git clean -xfdf` to my script[1] and it seems to work now.

Should this be added to regression.py? I feel like we could re-do some of the cleanup in regression.py as there appear that some (long-ish running) commands are duplicated. I'll try to find some time to poke around in there.

Tom


_______________________________________________
Boost-Testing mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-testing
Free forum by Nabble Edit this page