Quantcast

Building Boost with multiple python versions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Building Boost with multiple python versions

Boost - Build mailing list
Hello,

having learned a bit more about Boost.Build and its internals, I'm
trying to apply my understanding.


My user-config.jam file contains

  using python : 2.7 ;
  using pyhon : 3.5 ;

Then I issue this command:

  ./b2 --with-python python=2.7


Inside my Jamfile, I see that `feature.values python` reports "2.7 3.5",
independently on what I specify on the command-line. So it seems I'm
missing something:

What is b2's interpretation of the above user-config.jam file ?
And what does it mean to provide the `python=2.7` option on the
command-line ?

How can I specify that I want to

* build with Python 2.7 only
* build with Python 3.5 only
* build both variants at the same time



Thanks,
        Stefan


--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
On Mon, Feb 13, 2017 at 9:35 AM, Stefan Seefeld via Boost-build <[hidden email]> wrote:
Hello,

having learned a bit more about Boost.Build and its internals, I'm
trying to apply my understanding.


My user-config.jam file contains

  using python : 2.7 ;
  using pyhon : 3.5 ;

Then I issue this command:

  ./b2 --with-python python=2.7


Inside my Jamfile, I see that `feature.values python` reports "2.7 3.5",
independently on what I specify on the command-line. So it seems I'm
missing something:

What is b2's interpretation of the above user-config.jam file ?
And what does it mean to provide the `python=2.7` option on the
command-line ?

How can I specify that I want to

* build with Python 2.7 only
* build with Python 3.5 only
* build both variants at the same time

I would like to add to this question. On windows we often intermingle 32-bit and 64-bit builds. Is it correct to setup my user-config.jam as:

using python
: 2.7 # version
: C:\\Python27-32\\python.exe # Interperter/path to dir
: C:\\Python27-32\\include # includes
: C:\\Python27-32\\libs # libs
: <address-model>32 # conditions
;

using python
: 2.7 # version
: C:\\Python27-64\\python.exe # Interperter/path to dir
: C:\\Python27-64\\include # includes
: C:\\Python27-64\\libs # libs
: <address-model>64 # conditions
;

using python 
: 3.5 # version 
: C:\\Python35-32\\python.exe # Interperter/path to dir 
: C:\\Python35-32\\include # includes 
: C:\\Python35-32\\libs # libs 
: <address-model>32 # conditions 


using python 
: 3.5 # version 
: C:\\Python35-64\\python.exe # Interperter/path to dir 
: C:\\Python35-64\\include # includes 
: C:\\Python35-64\\libs # libs 
: <address-model>64 # conditions 
;

I've fought with both of these issues before (multiple versions and multiple architectures on the same PC) but never had a good understanding of if I was doing things correctly or if they were working as I assumed.

Thanks,
Tom

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
In reply to this post by Boost - Build mailing list
Warning: This is only a quick reply after some investigation. I would have to delve deeper to figure out more of the why things are the way they are (for better or worse)..

On Mon, Feb 13, 2017 at 9:35 AM, Stefan Seefeld via Boost-build <[hidden email]> wrote:
Hello,

having learned a bit more about Boost.Build and its internals, I'm
trying to apply my understanding.


My user-config.jam file contains

  using python : 2.7 ;
  using pyhon : 3.5 ;

Then I issue this command:

  ./b2 --with-python python=2.7


Inside my Jamfile, I see that `feature.values python` reports "2.7 3.5",
independently on what I specify on the command-line.

The "python" feature is composed of *all* the configured version of python. If you had no python installed or configured it would be empty. 
 
So it seems I'm
missing something:

What is b2's interpretation of the above user-config.jam file ?

It configures two python interpreters, and potentially libraries and headers.
 
And what does it mean to provide the `python=2.7` option on the
command-line ?

It means to add the "<python>2.7" as a build request to match against the targets to build.
 
How can I specify that I want to

* build with Python 2.7 only

You can't [1].
 
* build with Python 3.5 only

You can't [1]
 
* build both variants at the same time

[1] The way the BPL build file is written by default it builds both a 2 and 3 version of the library if both a 2 and 3 python version are configured. Now.. Theoretically it would be possible that specifying the version on the command would limit it to just that version to build. But it seems that the way the python.jam module sets up python libraries it fails to trigger the b2 target requirements selection mechanics. Or simply put.. It's broken. And I don't know entirely why yet. I'll keep looking at it so see if I can resolve this better eventually.

--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
On 13.02.2017 23:53, Rene Rivera via Boost-build wrote:

> Warning: This is only a quick reply after some investigation. I would
> have to delve deeper to figure out more of the why things are the way
> they are (for better or worse)..
>
> On Mon, Feb 13, 2017 at 9:35 AM, Stefan Seefeld via Boost-build
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hello,
>
>     having learned a bit more about Boost.Build and its internals, I'm
>     trying to apply my understanding.
>
>
>     My user-config.jam file contains
>
>       using python : 2.7 ;
>       using pyhon : 3.5 ;
>
>     Then I issue this command:
>
>       ./b2 --with-python python=2.7
>
>
>     Inside my Jamfile, I see that `feature.values python` reports "2.7
>     3.5",
>     independently on what I specify on the command-line.
>
>
> The "python" feature is composed of *all* the configured version of
> python.

I find that quite counter-intuitive.

> If you had no python installed or configured it would be empty.
>  
>
>     So it seems I'm
>     missing something:
>
>     What is b2's interpretation of the above user-config.jam file ?
>
>
> It configures two python interpreters, and potentially libraries and
> headers.

That makes sense.

>  
>
>     And what does it mean to provide the `python=2.7` option on the
>     command-line ?
>
>
> It means to add the "<python>2.7" as a build request to match against
> the targets to build.

OK, makes sense, too. So let me paraphrase, to make sure I understand
the idea:

Having the two "using python" lines above means that those two python
configurations are made available for future reference (in Jamfiles, as
well as command-line). It does *not* imply that both are being used (as
feature values) automatically (i.e. not without some other code
elsewhere to specify that "all" is indeed what the Jamfile requests.)


>  
>
>     How can I specify that I want to
>
>     * build with Python 2.7 only
>
>
> You can't [1].
>  
>
>     * build with Python 3.5 only
>
>
> You can't [1]
>  
>
>     * build both variants at the same time
>
>
> [1] The way the BPL build file is written by default it builds both a
> 2 and 3 version of the library if both a 2 and 3 python version are
> configured. Now.. Theoretically it would be possible that specifying
> the version on the command would limit it to just that version to
> build. But it seems that the way the python.jam module sets up python
> libraries it fails to trigger the b2 target requirements selection
> mechanics. Or simply put.. It's broken. And I don't know entirely why
> yet. I'll keep looking at it so see if I can resolve this better
> eventually.

OK, so can we agree that this is a bug (no matter where) ? After all I
have read about b2, it would seem logical that the above amounts to
declaring a "python" feature, with two possible values, such that a
command-line option of "python=2.7" would then pick one of the two.

However, this example raises another question (already asked by Tom): As
in the above the discriminating label is the version number, it isn't
clear how one can declare multiple python configurations that use the
same version. So if, as per Tom's example, I configure two Python 2.7
variants (one for 32-bit and one for 64-bit builds), how can I refer to
them unambiguously ? The version string clearly isn't enough. Is that
also a bug / limitation in the python.jam file ?

Thanks,
        Stefan


--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
On 2/14/2017 6:15 PM, Stefan Seefeld via Boost-build wrote:

> On 13.02.2017 23:53, Rene Rivera via Boost-build wrote:
>> Warning: This is only a quick reply after some investigation. I would
>> have to delve deeper to figure out more of the why things are the way
>> they are (for better or worse)..
>>
>> On Mon, Feb 13, 2017 at 9:35 AM, Stefan Seefeld via Boost-build
>> <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>     Hello,
>>
>>     having learned a bit more about Boost.Build and its internals, I'm
>>     trying to apply my understanding.
>>
>>
>>     My user-config.jam file contains
>>
>>       using python : 2.7 ;
>>       using pyhon : 3.5 ;
>>
>>     Then I issue this command:
>>
>>       ./b2 --with-python python=2.7
>>
>>
>>     Inside my Jamfile, I see that `feature.values python` reports "2.7
>>     3.5",
>>     independently on what I specify on the command-line.
>>
>>
>> The "python" feature is composed of *all* the configured version of
>> python.
>
> I find that quite counter-intuitive.
>
>> If you had no python installed or configured it would be empty.
>>
>>
>>     So it seems I'm
>>     missing something:
>>
>>     What is b2's interpretation of the above user-config.jam file ?
>>
>>
>> It configures two python interpreters, and potentially libraries and
>> headers.
>
> That makes sense.
>
>>
>>
>>     And what does it mean to provide the `python=2.7` option on the
>>     command-line ?
>>
>>
>> It means to add the "<python>2.7" as a build request to match against
>> the targets to build.
>
> OK, makes sense, too. So let me paraphrase, to make sure I understand
> the idea:
>
> Having the two "using python" lines above means that those two python
> configurations are made available for future reference (in Jamfiles, as
> well as command-line). It does *not* imply that both are being used (as
> feature values) automatically (i.e. not without some other code
> elsewhere to specify that "all" is indeed what the Jamfile requests.)
>
>
>>
>>
>>     How can I specify that I want to
>>
>>     * build with Python 2.7 only
>>
>>
>> You can't [1].
>>
>>
>>     * build with Python 3.5 only
>>
>>
>> You can't [1]
>>
>>
>>     * build both variants at the same time
>>
>>
>> [1] The way the BPL build file is written by default it builds both a
>> 2 and 3 version of the library if both a 2 and 3 python version are
>> configured. Now.. Theoretically it would be possible that specifying
>> the version on the command would limit it to just that version to
>> build. But it seems that the way the python.jam module sets up python
>> libraries it fails to trigger the b2 target requirements selection
>> mechanics. Or simply put.. It's broken. And I don't know entirely why
>> yet. I'll keep looking at it so see if I can resolve this better
>> eventually.
>
> OK, so can we agree that this is a bug (no matter where) ? After all I
> have read about b2, it would seem logical that the above amounts to
> declaring a "python" feature, with two possible values, such that a
> command-line option of "python=2.7" would then pick one of the two.
>
> However, this example raises another question (already asked by Tom): As
> in the above the discriminating label is the version number, it isn't
> clear how one can declare multiple python configurations that use the
> same version. So if, as per Tom's example, I configure two Python 2.7
> variants (one for 32-bit and one for 64-bit builds), how can I refer to
> them unambiguously ? The version string clearly isn't enough. Is that
> also a bug / limitation in the python.jam file ?

For compilers one can always do:

using some_compiler : n.n ; some_path : some_requirements ;
using some_compiler : n.nsome_string ; some_path : some_requirements ;

as in:

using gcc : 6.3 :
C:/Utilities/mingw-w64/i686-6.3.0-posix-dwarf-rt_v5-rev1/mingw32/bin/g++ ;
using gcc : 6.3c03 :
C:/Utilities/mingw-w64/i686-6.3.0-posix-dwarf-rt_v5-rev1/mingw32/bin/g++
: <cxxflags>-std=c++03 ;

and if I run 'b2 toolset=gcc-6.3 etc.' it will use the first of the
using statements and if I run 'b2 toolset=gcc-6.3c03 etc.' it will use
the second of the two using statements.

Ideally one should be able to do something similar with python.

I have no idea how these things work internally, but just mention what I
have noticed works for me.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
On 14.02.2017 19:20, Edward Diener via Boost-build wrote:

>
> For compilers one can always do:
>
> using some_compiler : n.n ; some_path : some_requirements ;
> using some_compiler : n.nsome_string ; some_path : some_requirements ;
>
> as in:
>
> using gcc : 6.3 :
> C:/Utilities/mingw-w64/i686-6.3.0-posix-dwarf-rt_v5-rev1/mingw32/bin/g++
> ;
> using gcc : 6.3c03 :
> C:/Utilities/mingw-w64/i686-6.3.0-posix-dwarf-rt_v5-rev1/mingw32/bin/g++
> : <cxxflags>-std=c++03 ;
>
> and if I run 'b2 toolset=gcc-6.3 etc.' it will use the first of the
> using statements and if I run 'b2 toolset=gcc-6.3c03 etc.' it will use
> the second of the two using statements.

Interesting, but also quite counter-intuitive: A line such as

using gcc : 6.3 ;

suggests the build system is instantiating a gcc toolchain with version
6.3. (Assume a platform with multiple compiler versions, it would have
to validate that it really picks a matching version !)
Now, with

using gcc : 6.3c03 ;

that pattern is broken, as there is no compiler that would report
"6.3c03" as version string.
So the obvious question is : what is the second argument to the "using"
rule above ? Is it a version or is it not a version ?

>
> Ideally one should be able to do something similar with python.
>
> I have no idea how these things work internally, but just mention what
> I have noticed works for me.

Understood. So the question above goes to the boost.build developers. :-)

Thanks,
        Stefan


--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
On 2/14/2017 7:51 PM, Stefan Seefeld via Boost-build wrote:

> On 14.02.2017 19:20, Edward Diener via Boost-build wrote:
>>
>> For compilers one can always do:
>>
>> using some_compiler : n.n ; some_path : some_requirements ;
>> using some_compiler : n.nsome_string ; some_path : some_requirements ;
>>
>> as in:
>>
>> using gcc : 6.3 :
>> C:/Utilities/mingw-w64/i686-6.3.0-posix-dwarf-rt_v5-rev1/mingw32/bin/g++
>> ;
>> using gcc : 6.3c03 :
>> C:/Utilities/mingw-w64/i686-6.3.0-posix-dwarf-rt_v5-rev1/mingw32/bin/g++
>> : <cxxflags>-std=c++03 ;
>>
>> and if I run 'b2 toolset=gcc-6.3 etc.' it will use the first of the
>> using statements and if I run 'b2 toolset=gcc-6.3c03 etc.' it will use
>> the second of the two using statements.
>
> Interesting, but also quite counter-intuitive: A line such as
>
> using gcc : 6.3 ;
>
> suggests the build system is instantiating a gcc toolchain with version
> 6.3. (Assume a platform with multiple compiler versions, it would have
> to validate that it really picks a matching version !)
> Now, with
>
> using gcc : 6.3c03 ;
>
> that pattern is broken, as there is no compiler that would report
> "6.3c03" as version string.
> So the obvious question is : what is the second argument to the "using"
> rule above ? Is it a version or is it not a version ?

I don't know those answers, but I am happy that at least it works for
me. This is because the feature/subfeature way of doing the above does
not work with the check-target-builds used by config, predef, and any
other library that uses the config and predef functionality for
configuring builds.

>
>>
>> Ideally one should be able to do something similar with python.
>>
>> I have no idea how these things work internally, but just mention what
>> I have noticed works for me.
>
> Understood. So the question above goes to the boost.build developers. :-)

You have much better luck in getting answers from them than I usually
do. One of the real problems with Boost Build is not that it does not
work well in the vast majority of situations, which it clearly does, but
that in those situations where it does not work as you expect it is
almost entirely on you to figure out why, since the internals of Boost
Build are understood by only a very few and they are often too busy with
other things to answer.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
The "using" rule is a convenience rule:
 
using MODULE_NAME : field1 : field 2 : ... : field9 ;
 
The above essentially does this:
 
import MODULE_NAME ;
MODULE_NAME.init field1 : field2 : ... : field9 ;
 
So, if you look at the gcc.jam module, you'll notice that its init() rule's signature is this:
 
rule init ( version ? : command * : options * )
 
Thus, the using rule would be used like this:
 
using gcc : $(valid-gcc-module-version-string) : $(optional-command-to-gcc-exe) : $(options) ;
 
Where $(options) can be something like <root>root/directory or <flavor>mingw.
 
You can find the using() rule's definition in the toolset module.
 
Hope that helps,
Aaron
 

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
Hi Aaron,

On 15.02.2017 11:18, Aaron Boman via Boost-build wrote:

> The "using" rule is a convenience rule:
>  
> using MODULE_NAME : field1 : field 2 : ... : field9 ;
>  
> The above essentially does this:
>  
> import MODULE_NAME ;
> MODULE_NAME.init field1 : field2 : ... : field9 ;
>  
> So, if you look at the gcc.jam module, you'll notice that its init()
> rule's signature is this:
>  
> rule init ( version ? : command * : options * )
>  
> Thus, the using rule would be used like this:
>  
> using gcc : $(valid-gcc-module-version-string) :
> $(optional-command-to-gcc-exe) : $(options) ;
>  
> Where $(options) can be something like <root>root/directory or
> <flavor>mingw.
>  
> You can find the using() rule's definition in the toolset module.

thanks, but I don't think that answers any of the questions.
Specifically, my last question was about the 'version' argument to the
gcc.init() function. Is it used to identify a compiler by version ? Or
can it be an arbitrary string / discriminator I use to configure
compilers with different flags (e.g. `--std=c++11` vs. `--std=c++03`) ?


>  
> Hope that helps,
> Aaron

Thanks,
        Stefan

--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list


On Wed, Feb 15, 2017 at 10:23 AM, Stefan Seefeld via Boost-build <[hidden email]> wrote:
Hi Aaron,

On 15.02.2017 11:18, Aaron Boman via Boost-build wrote:
> The "using" rule is a convenience rule:
>
> using MODULE_NAME : field1 : field 2 : ... : field9 ;
>
> The above essentially does this:
>
> import MODULE_NAME ;
> MODULE_NAME.init field1 : field2 : ... : field9 ;
>
> So, if you look at the gcc.jam module, you'll notice that its init()
> rule's signature is this:
>
> rule init ( version ? : command * : options * )
>
> Thus, the using rule would be used like this:
>
> using gcc : $(valid-gcc-module-version-string) :
> $(optional-command-to-gcc-exe) : $(options) ;
>
> Where $(options) can be something like <root>root/directory or
> <flavor>mingw.
>
> You can find the using() rule's definition in the toolset module.

thanks, but I don't think that answers any of the questions.
Specifically, my last question was about the 'version' argument to the
gcc.init() function. Is it used to identify a compiler by version ? Or
can it be an arbitrary string / discriminator I use to configure
compilers with different flags (e.g. `--std=c++11` vs. `--std=c++03`) ?

For gcc it can be an arbitrary string. If I understand the python init code correctly, which is a challenge even for me, for python it must be the version of the interpreter (ie it checks that it matches).

--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
On 15.02.2017 11:25, Rene Rivera wrote:

>
>
> On Wed, Feb 15, 2017 at 10:23 AM, Stefan Seefeld via Boost-build
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hi Aaron,
>
>     On 15.02.2017 11:18, Aaron Boman via Boost-build wrote:
>     > The "using" rule is a convenience rule:
>     >
>     > using MODULE_NAME : field1 : field 2 : ... : field9 ;
>     >
>     > The above essentially does this:
>     >
>     > import MODULE_NAME ;
>     > MODULE_NAME.init field1 : field2 : ... : field9 ;
>     >
>     > So, if you look at the gcc.jam module, you'll notice that its init()
>     > rule's signature is this:
>     >
>     > rule init ( version ? : command * : options * )
>     >
>     > Thus, the using rule would be used like this:
>     >
>     > using gcc : $(valid-gcc-module-version-string) :
>     > $(optional-command-to-gcc-exe) : $(options) ;
>     >
>     > Where $(options) can be something like <root>root/directory or
>     > <flavor>mingw.
>     >
>     > You can find the using() rule's definition in the toolset module.
>
>     thanks, but I don't think that answers any of the questions.
>     Specifically, my last question was about the 'version' argument to the
>     gcc.init() function. Is it used to identify a compiler by version ? Or
>     can it be an arbitrary string / discriminator I use to configure
>     compilers with different flags (e.g. `--std=c++11` vs.
>     `--std=c++03`) ?
>
>
> For gcc it can be an arbitrary string.

How then does

using gcc : 6.3 ;
using gcc : 5.3 ;

work in practice ? How does b2 look up the appropriate compilers, if it
doesn't somehow match the version strings in the above snippet with the
output of `gcc --version` ?

> If I understand the python init code correctly, which is a challenge
> even for me, for python it must be the version of the interpreter (ie
> it checks that it matches).

OK. So for Python the proposed trick of using a "labeled version" to
name a specific Python instance unambiguously (e.g. in the presence of a
32-bit Python and a 64-bit Python of the same version) wouldn't work, yes ?

Thanks,
        Stefan


--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
In reply to this post by Boost - Build mailing list
On Tue, Feb 14, 2017 at 5:15 PM, Stefan Seefeld via Boost-build <[hidden email]> wrote:
On 13.02.2017 23:53, Rene Rivera via Boost-build wrote:
> Warning: This is only a quick reply after some investigation. I would
> have to delve deeper to figure out more of the why things are the way
> they are (for better or worse)..
>
> On Mon, Feb 13, 2017 at 9:35 AM, Stefan Seefeld via Boost-build
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hello,
>
>     having learned a bit more about Boost.Build and its internals, I'm
>     trying to apply my understanding.
>
>
>     My user-config.jam file contains
>
>       using python : 2.7 ;
>       using pyhon : 3.5 ;
>
>     Then I issue this command:
>
>       ./b2 --with-python python=2.7
>
>
>     Inside my Jamfile, I see that `feature.values python` reports "2.7
>     3.5",
>     independently on what I specify on the command-line.
>
>
> The "python" feature is composed of *all* the configured version of
> python.

I find that quite counter-intuitive.

It's how non-free features work. They are a set of values. 

>     And what does it mean to provide the `python=2.7` option on the
>     command-line ?
>
>
> It means to add the "<python>2.7" as a build request to match against
> the targets to build.

OK, makes sense, too. So let me paraphrase, to make sure I understand
the idea:

Having the two "using python" lines above means that those two python
configurations are made available for future reference (in Jamfiles, as
well as command-line). It does *not* imply that both are being used (as
feature values) automatically (i.e. not without some other code
elsewhere to specify that "all" is indeed what the Jamfile requests.)

Precisely correct. 

>     How can I specify that I want to
>
>     * build with Python 2.7 only
>
>
> You can't [1].
>
>
>     * build with Python 3.5 only
>
>
> You can't [1]
>
>
>     * build both variants at the same time
>
>
> [1] The way the BPL build file is written by default it builds both a
> 2 and 3 version of the library if both a 2 and 3 python version are
> configured. Now.. Theoretically it would be possible that specifying
> the version on the command would limit it to just that version to
> build. But it seems that the way the python.jam module sets up python
> libraries it fails to trigger the b2 target requirements selection
> mechanics. Or simply put.. It's broken. And I don't know entirely why
> yet. I'll keep looking at it so see if I can resolve this better
> eventually.

OK, so can we agree that this is a bug (no matter where) ? After all I
have read about b2, it would seem logical that the above amounts to
declaring a "python" feature, with two possible values, such that a
command-line option of "python=2.7" would then pick one of the two.

After more investigation.. Calling it a bug is an understatement! It's so broken I haven't figured out why it's broken. The current symptoms, even after some cleanup on my part of the python build file is as follows:

(a) If a py2 interpreter is configured BPL builds py2 libraries that link against python2 runtime.
(b) If a py2 and py3 interpreter is configured BPL builds (a) and py3 libraries that link against **python2** runtime!
(c) If only a py3 interpreter is configured BPL builds py3 libraries that link against python3 runtime.

Read that at least twice and contemplate the implications of how broken that makes everything be. I'm trying to fix it. But the results defy expectations of how b2 is supposed to work. So it's taking some effort to track down the root of the problems.

However, this example raises another question (already asked by Tom): As
in the above the discriminating label is the version number, it isn't
clear how one can declare multiple python configurations that use the
same version. So if, as per Tom's example, I configure two Python 2.7
variants (one for 32-bit and one for 64-bit builds), how can I refer to
them unambiguously ? The version string clearly isn't enough. Is that
also a bug / limitation in the python.jam file ?

You can specify multiple features in the command like to indicate the collective requirement request of what you want to build. Hence if you configure the python interpreter with conditions that match that build request it should use that interpreter only. That's assuming it wasn't broken though :-(

--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
In reply to this post by Boost - Build mailing list


On Wed, Feb 15, 2017 at 10:32 AM, Stefan Seefeld <[hidden email]> wrote:
On 15.02.2017 11:25, Rene Rivera wrote:
>
>
> On Wed, Feb 15, 2017 at 10:23 AM, Stefan Seefeld via Boost-build
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hi Aaron,
>
>     On 15.02.2017 11:18, Aaron Boman via Boost-build wrote:
>     > The "using" rule is a convenience rule:
>     >
>     > using MODULE_NAME : field1 : field 2 : ... : field9 ;
>     >
>     > The above essentially does this:
>     >
>     > import MODULE_NAME ;
>     > MODULE_NAME.init field1 : field2 : ... : field9 ;
>     >
>     > So, if you look at the gcc.jam module, you'll notice that its init()
>     > rule's signature is this:
>     >
>     > rule init ( version ? : command * : options * )
>     >
>     > Thus, the using rule would be used like this:
>     >
>     > using gcc : $(valid-gcc-module-version-string) :
>     > $(optional-command-to-gcc-exe) : $(options) ;
>     >
>     > Where $(options) can be something like <root>root/directory or
>     > <flavor>mingw.
>     >
>     > You can find the using() rule's definition in the toolset module.
>
>     thanks, but I don't think that answers any of the questions.
>     Specifically, my last question was about the 'version' argument to the
>     gcc.init() function. Is it used to identify a compiler by version ? Or
>     can it be an arbitrary string / discriminator I use to configure
>     compilers with different flags (e.g. `--std=c++11` vs.
>     `--std=c++03`) ?
>
>
> For gcc it can be an arbitrary string.

How then does

using gcc : 6.3 ;
using gcc : 5.3 ;

work in practice ? How does b2 look up the appropriate compilers, if it
doesn't somehow match the version strings in the above snippet with the
output of `gcc --version` ?

> If I understand the python init code correctly, which is a challenge
> even for me, for python it must be the version of the interpreter (ie
> it checks that it matches).

OK. So for Python the proposed trick of using a "labeled version" to
name a specific Python instance unambiguously (e.g. in the presence of a
32-bit Python and a 64-bit Python of the same version) wouldn't work, yes ?

Probably not.. As in I'm still trying to decipher the python.jam code.


--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
In reply to this post by Boost - Build mailing list
On 15.02.2017 11:39, Rene Rivera wrote:

> On Tue, Feb 14, 2017 at 5:15 PM, Stefan Seefeld via Boost-build
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     On 13.02.2017 23:53, Rene Rivera via Boost-build wrote:
>     > Warning: This is only a quick reply after some investigation. I
>     would
>     > have to delve deeper to figure out more of the why things are
>     the way
>     > they are (for better or worse)..
>     >
>     > On Mon, Feb 13, 2017 at 9:35 AM, Stefan Seefeld via Boost-build
>     > <[hidden email] <mailto:[hidden email]>
>     <mailto:[hidden email]
>     <mailto:[hidden email]>>> wrote:
>     >
>     >     Hello,
>     >
>     >     having learned a bit more about Boost.Build and its
>     internals, I'm
>     >     trying to apply my understanding.
>     >
>     >
>     >     My user-config.jam file contains
>     >
>     >       using python : 2.7 ;
>     >       using pyhon : 3.5 ;
>     >
>     >     Then I issue this command:
>     >
>     >       ./b2 --with-python python=2.7
>     >
>     >
>     >     Inside my Jamfile, I see that `feature.values python`
>     reports "2.7
>     >     3.5",
>     >     independently on what I specify on the command-line.
>     >
>     >
>     > The "python" feature is composed of *all* the configured version of
>     > python.
>
>     I find that quite counter-intuitive.
>
>
> It's how non-free features work. They are a set of values.
>
>     >     And what does it mean to provide the `python=2.7` option on the
>     >     command-line ?
>     >
>     >
>     > It means to add the "<python>2.7" as a build request to match
>     against
>     > the targets to build.
>
>     OK, makes sense, too. So let me paraphrase, to make sure I understand
>     the idea:
>
>     Having the two "using python" lines above means that those two python
>     configurations are made available for future reference (in
>     Jamfiles, as
>     well as command-line). It does *not* imply that both are being
>     used (as
>     feature values) automatically (i.e. not without some other code
>     elsewhere to specify that "all" is indeed what the Jamfile requests.)
>
>
> Precisely correct.
>
>     >     How can I specify that I want to
>     >
>     >     * build with Python 2.7 only
>     >
>     >
>     > You can't [1].
>     >
>     >
>     >     * build with Python 3.5 only
>     >
>     >
>     > You can't [1]
>     >
>     >
>     >     * build both variants at the same time
>     >
>     >
>     > [1] The way the BPL build file is written by default it builds
>     both a
>     > 2 and 3 version of the library if both a 2 and 3 python version are
>     > configured. Now.. Theoretically it would be possible that specifying
>     > the version on the command would limit it to just that version to
>     > build. But it seems that the way the python.jam module sets up
>     python
>     > libraries it fails to trigger the b2 target requirements selection
>     > mechanics. Or simply put.. It's broken. And I don't know
>     entirely why
>     > yet. I'll keep looking at it so see if I can resolve this better
>     > eventually.
>
>     OK, so can we agree that this is a bug (no matter where) ? After all I
>     have read about b2, it would seem logical that the above amounts to
>     declaring a "python" feature, with two possible values, such that a
>     command-line option of "python=2.7" would then pick one of the two.
>
>
> After more investigation.. Calling it a bug is an understatement! It's
> so broken I haven't figured out why it's broken. The current symptoms,
> even after some cleanup on my part of the python build file is as follows:
>
> (a) If a py2 interpreter is configured BPL builds py2 libraries that
> link against python2 runtime.
> (b) If a py2 and py3 interpreter is configured BPL builds (a) and py3
> libraries that link against **python2** runtime!

That's exactly the bug that was reported to me in a couple of places,
and which I had no idea how to solve.


> (c) If only a py3 interpreter is configured BPL builds py3 libraries
> that link against python3 runtime.
>
> Read that at least twice and contemplate the implications of how
> broken that makes everything be. I'm trying to fix it. But the results
> defy expectations of how b2 is supposed to work. So it's taking some
> effort to track down the root of the problems.

Thanks a lot for looking into this !

>
>     However, this example raises another question (already asked by
>     Tom): As
>     in the above the discriminating label is the version number, it isn't
>     clear how one can declare multiple python configurations that use the
>     same version. So if, as per Tom's example, I configure two Python 2.7
>     variants (one for 32-bit and one for 64-bit builds), how can I
>     refer to
>     them unambiguously ? The version string clearly isn't enough. Is that
>     also a bug / limitation in the python.jam file ?
>
>
> You can specify multiple features in the command like to indicate the
> collective requirement request of what you want to build. Hence if you
> configure the python interpreter with conditions that match that build
> request it should use that interpreter only. That's assuming it wasn't
> broken though :-(

OK. But all existing bugs in the implementation aside, I think this is
more of a UI concern: If I understand correctly, the "python" feature
expects values that express versions (so I can select a python config on
the command line with "<python>2.7".
my question is about what happens if the version alone isn't an
unambiguous discriminator, i.e. if I need to distinguish between a
64-bit python and a 32-bit python, both using the same version 2.7 ? How
can I express that, both on the command line as well as in my
user-config.jam configuration with the `using` rule ?

Thanks,
        Stefan

--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
On Wed, Feb 15, 2017 at 10:50 AM, Stefan Seefeld <[hidden email]> wrote:
On 15.02.2017 11:39, Rene Rivera wrote:
>

> You can specify multiple features in the command like to indicate the
> collective requirement request of what you want to build. Hence if you
> configure the python interpreter with conditions that match that build
> request it should use that interpreter only. That's assuming it wasn't
> broken though :-(

OK. But all existing bugs in the implementation aside, I think this is
more of a UI concern: If I understand correctly, the "python" feature
expects values that express versions (so I can select a python config on
the command line with "<python>2.7".
my question is about what happens if the version alone isn't an
unambiguous discriminator, i.e. if I need to distinguish between a
64-bit python and a 32-bit python, both using the same version 2.7 ? How
can I express that, both on the command line as well as in my
user-config.jam configuration with the `using` rule ?

Given Tom's example user-config you can: 

b2 python=2.7 address-model=32
b2 python=2.7 address-model=64
b2 python=3.5 address-model=32
b2 python=3.5 address-model=64

To build them individually. And if you plan on installing you'd have to install to different locations as address-model doesn't tag the built products. You can also build multiple combinations in one go with:

b2 python=2.7,3.5 address-model=32
b2 python=2.7,3.5 address-model=64

Or all the combinations with:

b2 python=2.7,3.5 address-model=32,64

I believe the general build request option syntax is documented. But I leave finding that doc as an exercise for the reader ;-) 


--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
On 15.02.2017 12:00, Rene Rivera wrote:

> On Wed, Feb 15, 2017 at 10:50 AM, Stefan Seefeld <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On 15.02.2017 11:39, Rene Rivera wrote:
>     >
>     > You can specify multiple features in the command like to
>     indicate the
>     > collective requirement request of what you want to build. Hence
>     if you
>     > configure the python interpreter with conditions that match that
>     build
>     > request it should use that interpreter only. That's assuming it
>     wasn't
>     > broken though :-(
>
>     OK. But all existing bugs in the implementation aside, I think this is
>     more of a UI concern: If I understand correctly, the "python" feature
>     expects values that express versions (so I can select a python
>     config on
>     the command line with "<python>2.7".
>     my question is about what happens if the version alone isn't an
>     unambiguous discriminator, i.e. if I need to distinguish between a
>     64-bit python and a 32-bit python, both using the same version 2.7
>     ? How
>     can I express that, both on the command line as well as in my
>     user-config.jam configuration with the `using` rule ?
>
>
> Given Tom's example user-config you can:
>
> b2 python=2.7 address-model=32
> b2 python=2.7 address-model=64
> b2 python=3.5 address-model=32
> b2 python=3.5 address-model=64
>
> To build them individually. And if you plan on installing you'd have
> to install to different locations as address-model doesn't tag the
> built products. You can also build multiple combinations in one go with:
>
> b2 python=2.7,3.5 address-model=32
> b2 python=2.7,3.5 address-model=64
>
> Or all the combinations with:
>
> b2 python=2.7,3.5 address-model=32,64

Hmm, I'm not entirely convinced that means the same thing. The
"address-model" is a separate feature here, while what I had in mind was
more a composite feature (with 'version' and 'address-model' being
constituents). For this particular example this may not matter, as
address-models can't be mixed (or can they ?).

But consider the other part: defining python configurations:

using python : 2.7 : .... ;

defines a python "tool" with a "version" feature "2.7". There doesn't
appear to be a way to attach an address-model feature to it. So it's
impossible to express "here is a python configuration for which the
"version" is X.Y and the "address-model" is Z", and neither is there a
way to request a python configuration (such as done via "using python :
2.7") with both version and address-model fixed.

The "using ..." rule can be used both as a definition of a tool as well
as to request one. So it seems it might be useful to add an
"address-model" (optional) argument to the python.init() function (and
by extension, the `using python ...` rule.


>
> I believe the general build request option syntax is documented. But I
> leave finding that doc as an exercise for the reader ;-)

Hah. :-)

But to fully drive home my point, I think this entire discussion reveals
not so much shortcomings in the implementation but issues on the
conceptual level (and holes in the documentation), don't you think ?

Thanks,
        Stefan

--

      ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list
I think, in order to select 32 vs 64, you would need to pass in the command to that version yourself, then pass in your own condition (6th parameter to using, 5th parameter to python.init):
 
using python : 2.7 : /path/to/64/python : : : <address-model>64 ;
using python : 2.7 : /path/to/python : : : <address-model>32 <address-model> ; # the lack of value on <address-model> here indicates that in the event that the address-model is not present in the build request, this will be the default python chosen.
 
Correct me if I'm wrong, but this should select the correct python interpreter/library.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list


On Wed, Feb 15, 2017 at 11:24 AM, Aaron Boman via Boost-build <[hidden email]> wrote:
I think, in order to select 32 vs 64, you would need to pass in the command to that version yourself, then pass in your own condition (6th parameter to using, 5th parameter to python.init):
 
using python : 2.7 : /path/to/64/python : : : <address-model>64 ;
using python : 2.7 : /path/to/python : : : <address-model>32 <address-model> ; # the lack of value on <address-model> here indicates that in the event that the address-model is not present in the build request, this will be the default python chosen.
 
Correct me if I'm wrong, but this should select the correct python interpreter/library.

That is a very useful trick, I have definitely struggled trying to support the "<address-model>32" and "<address-model> ". I guess it is kinda obvious once you think about it. Do you know if this is anywhere in the docs so I can reference it?

Tom

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Building Boost with multiple python versions

Boost - Build mailing list

> That is a very useful trick, I have definitely struggled
> trying to support the "<address-model>32" and
> "<address-model> ". I guess it is kinda obvious once you
> think about it. Do you know if this is anywhere in the
> docs so I can reference it?

I did a quick search through the documentation and I can't
find anything on it. I may have learned about that particular
construct while playing with the code (I use the Python port
and tend to run the debugger to watch how the code executes).


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Loading...