CMake Modularization

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

CMake Modularization

Philip Woods
CMake is a widely used build system and has become for many the de facto
standard. Boost is a
widely used collection of modules and has become for many part of the de
facto standard.
However, the two do not play very well together. I know there is another
port of boost to CMake, but
it has lain dormant for years and uses an old version of boost; plus I
chose to take a different route.
The old project attempted to completely replace Boost.Build with CMake. My
example port simply
allows people to more easily use boost modules in their projects.

I created an example project
https://github.com/Elzair/cmake-boost-filesystem-test to demonstrate
my ideas. I added CMake build support to the minimum number of modules
needed to support this
demo: https://github.com/Elzair/assert https://github.com/Elzair/config
https://github.com/Elzair/core
https://github.com/Elzair/filesystem/tree/master and
https://github.com/Elzair/system

I even added CMake build support to the header only boost libraries. I did
this for two reasons.
First, I did not want users to have to fool with CMake's find_package().
Second, I wanted to add
helpful error messages in case a user has not included all of a module's
dependencies (or added
them in the wrong order).

Would the community be interested in modular CMake support for more
modules? Any comments, questions, critiques or suggestions are welcome.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Edward Diener-3
On 9/27/2016 11:04 AM, Philip Woods wrote:

> CMake is a widely used build system and has become for many the de facto
> standard. Boost is a
> widely used collection of modules and has become for many part of the de
> facto standard.
> However, the two do not play very well together. I know there is another
> port of boost to CMake, but
> it has lain dormant for years and uses an old version of boost; plus I
> chose to take a different route.
> The old project attempted to completely replace Boost.Build with CMake. My
> example port simply
> allows people to more easily use boost modules in their projects.

It is always beneficial to give a detail account of what you are
actually doing. Otherwise your chances of interesting people in what you
have done will be less than it could be despite the examples you have
given below.

>
> I created an example project
> https://github.com/Elzair/cmake-boost-filesystem-test to demonstrate
> my ideas. I added CMake build support to the minimum number of modules
> needed to support this
> demo: https://github.com/Elzair/assert https://github.com/Elzair/config
> https://github.com/Elzair/core
> https://github.com/Elzair/filesystem/tree/master and
> https://github.com/Elzair/system
>
> I even added CMake build support to the header only boost libraries. I did
> this for two reasons.
> First, I did not want users to have to fool with CMake's find_package().
> Second, I wanted to add
> helpful error messages in case a user has not included all of a module's
> dependencies (or added
> them in the wrong order).
>
> Would the community be interested in modular CMake support for more
> modules? Any comments, questions, critiques or suggestions are welcome.



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Philip Woods
> It is always beneficial to give a detail account of what you are actually
doing. Otherwise your chances of interesting
> people in what you have done will be less than it could be despite the
examples you have given below.

Alright, I want to make boost more modular and easier to use. Many modules
are header-only, so they are quite easy to
use: you can just copy the headers from their repository and drop them into
your project. However, a lot of interesting
functionality (i.e. the filesystem module, the fibers module, the regexp
module, etc.) requires building libraries. AFAIK this
requires having/learning about JAM, BJAM, Boost.Build and other
non-standard tools. It also requires downloading the
whole main boost repository. I guess the only way around that would be
including .dlls, .dylibs or .sos you got from your package manager into
your project.

Suppose we have a project named foo that wants to use Boost.Filesystem. It
has the following file structure.

foo/
  CMakeLists.txt
  3rdpartylibs/
    CMakeLists.txt
  bar.cpp

I want to make using Boost.Filesystem (or any other boost module) as simple
as dropping the module and all of its dependent modules into the
3rdpartylibs folder and updating 3rdpartylibs/CMakeLists.txt with the
following code.

    add_subdirectory(config)
    add_subdirectory(assert)
    add_subdirectory(core)
    add_subdirectory(system)
    add_subdirectory(filesystem)

Then the user only needs to add the following to the main CMakeLists.txt

add_subdirectory(3rdpartylibs)
project(foo)
include_directories(${BOOST_INCLUDE_DIRS})
add_executable(foo bar.cpp)
target_link_libraries(foo ${BOOST_LIBRARIES})

Then, the modules can be built along with the rest of your code.

On Tue, Sep 27, 2016 at 2:26 PM, Edward Diener <[hidden email]>
wrote:

> On 9/27/2016 11:04 AM, Philip Woods wrote:
>
>> CMake is a widely used build system and has become for many the de facto
>> standard. Boost is a
>> widely used collection of modules and has become for many part of the de
>> facto standard.
>> However, the two do not play very well together. I know there is another
>> port of boost to CMake, but
>> it has lain dormant for years and uses an old version of boost; plus I
>> chose to take a different route.
>> The old project attempted to completely replace Boost.Build with CMake. My
>> example port simply
>> allows people to more easily use boost modules in their projects.
>>
>
> It is always beneficial to give a detail account of what you are actually
> doing. Otherwise your chances of interesting people in what you have done
> will be less than it could be despite the examples you have given below.
>
>
>> I created an example project
>> https://github.com/Elzair/cmake-boost-filesystem-test to demonstrate
>> my ideas. I added CMake build support to the minimum number of modules
>> needed to support this
>> demo: https://github.com/Elzair/assert https://github.com/Elzair/config
>> https://github.com/Elzair/core
>> https://github.com/Elzair/filesystem/tree/master and
>> https://github.com/Elzair/system
>>
>> I even added CMake build support to the header only boost libraries. I did
>> this for two reasons.
>> First, I did not want users to have to fool with CMake's find_package().
>> Second, I wanted to add
>> helpful error messages in case a user has not included all of a module's
>> dependencies (or added
>> them in the wrong order).
>>
>> Would the community be interested in modular CMake support for more
>> modules? Any comments, questions, critiques or suggestions are welcome.
>>
>
>
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman
> /listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Peter Dimov-2
In reply to this post by Philip Woods
Philip Woods wrote:
> I created an example project
> https://github.com/Elzair/cmake-boost-filesystem-test to demonstrate my
> ideas. I added CMake build support to the minimum number of modules needed
> to support this demo: https://github.com/Elzair/assert 
> https://github.com/Elzair/config https://github.com/Elzair/core 
> https://github.com/Elzair/filesystem/tree/master and
> https://github.com/Elzair/system

A few questions come to mind... why is filesystem on master and the others
on develop?

In filesystem/CMakeLists.txt, shouldn't

set(BOOST_SYSTEM_INCLUDE_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/include" CACHE
STRING "Location of boost.system headers")

refer to FILESYSTEM instead?

As far as I can see libs/CMakeLists.txt requires the add_subdirectory calls
to be properly ordered. Isn't this inconvenient? I suppose it's fine for
this case where there are only five lines, but not all libraries are like
that.

Why does the Filesystem CMakeLists.txt file require C++11?

Does this actually work? When I do 'make' I get

D:/tmp/cmake-boost-filesystem-test/test.cpp:3:32: fatal error:
boost/filesystem.hpp: No such file or directory
#include <boost/filesystem.hpp>
                                ^

When I fix the above problem in filesystem/CMakeLists.txt, I get:

In file included from
D:/tmp/cmake-boost-filesystem-test/libs/filesystem/include/boost/filesystem/path.hpp:25:0,
                 from
D:/tmp/cmake-boost-filesystem-test/libs/filesystem/include/boost/filesystem.hpp:16,
                 from D:/tmp/cmake-boost-filesystem-test/test.cpp:3:
D:/tmp/cmake-boost-filesystem-test/libs/filesystem/include/boost/filesystem/path_traits.hpp:21:42:
fatal error: boost/type_traits/is_array.hpp: No such file or directory
#include <boost/type_traits/is_array.hpp>
                                          ^

which is correct, there is no type_traits.

You've probably tested in on a machine with a system Boost installation.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Peter Dimov-2
To add to my post, to use (or build) filesystem, you need at minimum the
following Boost modules:

assert
config
core
detail
functional
io
iterator
mpl
predef
preprocessor
range
smart_ptr
static_assert
system
throw_exception
type_traits

-----Original Message-----
From: Peter Dimov
Sent: Tuesday, September 27, 2016 23:41
To: [hidden email]
Subject: Re: [boost] CMake Modularization

Philip Woods wrote:
> I created an example project
> https://github.com/Elzair/cmake-boost-filesystem-test to demonstrate my
> ideas. I added CMake build support to the minimum number of modules needed
> to support this demo: https://github.com/Elzair/assert 
> https://github.com/Elzair/config https://github.com/Elzair/core 
> https://github.com/Elzair/filesystem/tree/master and
> https://github.com/Elzair/system

A few questions come to mind... why is filesystem on master and the others
on develop?

In filesystem/CMakeLists.txt, shouldn't

set(BOOST_SYSTEM_INCLUDE_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/include" CACHE
STRING "Location of boost.system headers")

refer to FILESYSTEM instead?

As far as I can see libs/CMakeLists.txt requires the add_subdirectory calls
to be properly ordered. Isn't this inconvenient? I suppose it's fine for
this case where there are only five lines, but not all libraries are like
that.

Why does the Filesystem CMakeLists.txt file require C++11?

Does this actually work? When I do 'make' I get

D:/tmp/cmake-boost-filesystem-test/test.cpp:3:32: fatal error:
boost/filesystem.hpp: No such file or directory
#include <boost/filesystem.hpp>
                                ^

When I fix the above problem in filesystem/CMakeLists.txt, I get:

In file included from
D:/tmp/cmake-boost-filesystem-test/libs/filesystem/include/boost/filesystem/path.hpp:25:0,
                 from
D:/tmp/cmake-boost-filesystem-test/libs/filesystem/include/boost/filesystem.hpp:16,
                 from D:/tmp/cmake-boost-filesystem-test/test.cpp:3:
D:/tmp/cmake-boost-filesystem-test/libs/filesystem/include/boost/filesystem/path_traits.hpp:21:42:
fatal error: boost/type_traits/is_array.hpp: No such file or directory
#include <boost/type_traits/is_array.hpp>
                                          ^

which is correct, there is no type_traits.

You've probably tested in on a machine with a system Boost installation.


_______________________________________________
Unsubscribe & other changes:
http://lists.boost.org/mailman/listinfo.cgi/boost



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Peter Dimov-2
In reply to this post by Philip Woods
Philip Woods wrote:
> I created an example project
> https://github.com/Elzair/cmake-boost-filesystem-test to demonstrate my
> ideas. I added CMake build support to the minimum number of modules needed
> to support this demo: https://github.com/Elzair/assert 
> https://github.com/Elzair/config https://github.com/Elzair/core 
> https://github.com/Elzair/filesystem/tree/master and
> https://github.com/Elzair/system

I know next to nothing about CMake, but this doesn't seem quite the right
way.

This:

set(BOOST_FILESYSTEM_INCLUDE_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/include"
CACHE STRING "Location of boost.filesystem headers")

should perhaps just be

target_include_directories(boost_filesystem PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}/include)

The manual dependency checking at the top:

if(NOT BOOST_CONFIG_INCLUDE_DIRS)
   message(FATAL_ERROR "Cannot find boost.config!")
endif(NOT BOOST_CONFIG_INCLUDE_DIRS)

should probably be replaced by

target_link_libraries(boost_config)

boost_config, being a header-only library, should be

add_library(boost_config INTERFACE)

The various BOOST_*_INCLUDE_DIRS and BOOST_*_LIBRARIES won't be needed;
CMake will figure it out automatically. And the order of add_subdirectory
will no longer matter.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Philip Woods
In reply to this post by Peter Dimov-2
> In filesystem/CMakeLists.txt, shouldn't

> set(BOOST_SYSTEM_INCLUDE_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/include" CACHE
STRING "Location of
> boost.system headers")

> refer to FILESYSTEM instead?

Yes it should. I just updated the codebases with those fixes.

> You've probably tested in on a machine with a system Boost installation.

Yes I did. Thank you for catching that.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Louis Dionne
In reply to this post by Philip Woods
> CMake is a widely used build system and has become for many the de facto
> standard. Boost is a widely used collection of modules and has become for
> many part of the de facto standard. However, the two do not play very well
> together.

I am very much interested in a port to CMake, and so are a few other members
of the community (even if they sometimes stay quiet). There's also the silent
majority of Boost users that want something like this.

> I know there is another port of boost to CMake, but it has lain dormant for
> years and uses an old version of boost; plus I chose to take a different
> route.
> [...]
> The old project attempted to completely replace Boost.Build with CMake. My
> example port simply allows people to more easily use boost modules in their
> projects.

Are you talking about https://github.com/boost-cmake/boost-cmake ?
If not, then I would suggest you have a look at it. I think any CMake
effort should start from there, since the largest part of the work is
already done. If you are aware of this project, I think a clarification
of why you think it's not fit for what you want to do would be good.

Regards,
Louis
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Philip Woods
In reply to this post by Peter Dimov-2
> I know next to nothing about CMake, but this doesn't seem quite the right
way.

> This:

>   set(BOOST_FILESYSTEM_INCLUDE_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/include"
CACHE STRING
     "Location of boost.filesystem headers")

> should perhaps just be

>    target_include_directories(boost_filesystem PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}/include)

> The manual dependency checking at the top:

>    if(NOT BOOST_CONFIG_INCLUDE_DIRS)
        message(FATAL_ERROR "Cannot find boost.config!")
      endif(NOT BOOST_CONFIG_INCLUDE_DIRS)

> should probably be replaced by

>    target_link_libraries(boost_config)

> boost_config, being a header-only library, should be

>    add_library(boost_config INTERFACE)

> The various BOOST_*_INCLUDE_DIRS and BOOST_*_LIBRARIES won't be needed;
CMake will figure it out automatically. And the order of add_subdirectory
will no longer matter.

Thanks for that! I did not know CMake had that functionality.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Egor Pugin
Hi,

What you are doing manually is already done in automatic way.
(But only for headers/sources. It does not handle tests and docs.)

C++ Archive Network https://cppan.org/ generates CMake files during its work.
https://cppan.org/projects/pvt.cppan.demo.boost
Boost 1.61.0 is fully there and almost all libs (except some compiled
libs with external deps) work fine.

--
Egor Pugin

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

David Sankel
In reply to this post by Philip Woods
On Tue, Sep 27, 2016 at 11:04 AM, Philip Woods <
[hidden email]> wrote:
>
> Would the community be interested in modular CMake support for more
> modules? Any comments, questions, critiques or suggestions are welcome.
>

First and foremost, thank you for your interest and for working on this! It
has been said many times before, but it bears repeating: good CMake support
would lower Boost adoption barriers for a large number of C++ developers.

CMake's FindBoost.cmake interface is awkward to use sometimes, but it is
how most CMake applications use Boost these days. If we could provide a
drop-in replacement that builds from a source repository on an as-needed
basis, it would get a ton of immediate use. Something worth considering. It
may also be useful to check out this talk
<https://www.youtube.com/watch?v=3eH4hMKl7XE> where I discuss designing
CMake builds in a way that facilitates inter-operation between different
library builds.

Good luck,

David Sankel

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Peter Dimov-2
David Sankel wrote:
> CMake's FindBoost.cmake interface is awkward to use sometimes, but it is
> how most CMake applications use Boost these days. If we could provide a
> drop-in replacement that builds from a source repository on an as-needed
> basis, it would get a ton of immediate use. Something worth considering.

After thinking about this a bit, I'm still not sure how are we to resolve
the conflict between in-Boost use and standalone use of, f.ex. Hana.

The use case in which Boost libraries are brought into a project via git
submodules requires a CMakeLists.txt file that is different from the one
that standalone libraries currently use. As one example of the differences,
standalone libraries find the system Boost installation and use that for
their dependencies.

I suppose my earlier suggestion of putting the submodule CMakeLists into
libs/*/build instead of libs/* could work. But there'll still be a potential
conflict with libs/*/test/CMakeLists.txt.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Paul Fultz II

> On Oct 2, 2016, at 5:16 PM, Peter Dimov <[hidden email]> wrote:
>
> David Sankel wrote:
>> CMake's FindBoost.cmake interface is awkward to use sometimes, but it is how most CMake applications use Boost these days. If we could provide a drop-in replacement that builds from a source repository on an as-needed basis, it would get a ton of immediate use. Something worth considering.
>
> After thinking about this a bit, I'm still not sure how are we to resolve the conflict between in-Boost use and standalone use of, f.ex. Hana.
>
> The use case in which Boost libraries are brought into a project via git submodules requires a CMakeLists.txt file that is different from the one that standalone libraries currently use. As one example of the differences, standalone libraries find the system Boost installation and use that for their dependencies.

The libraries will need to some coordination and cannot use the `FindBoost` module provided by cmake(and it really doesn’t make sense if boost actually supports cmake). Instead a top-level cmake can bring the packages together, something like:

    set(BOOST_ALL TRUE)
    # Whatever cmake modulues boost provide
    include(cmake/BoostModules.cmake)
    # Include the libraries
    # Perhaps this could be globbed
    add_subdirectory(libs/fit)
    add_subdirectory(libs/hana)
    ...

Then in the library, the cmake would do something like this:

    # We are building standalone
    if(NOT BOOST_ALL)
        # BoostModules must be installed first, before building standalone
        find_package(BoostModulues)
    endif()

    # Find the dependent boost libraries
    find_boost_lib(BOOST_LIBS asio hana)

So now, when building standalone `find_boost_lib` will call `find_package` to retrieve the exported targets. If its building in-tree, then `find_boost_lib` will just find the target name, and let cmake resolve the target during build.

This will let boost libraries to be built in-tree, which is useful for libraries with circular dependencies, and also can be built per-modulue. The only thing is that the boost cmake modules would need to be installed first.

Now if libraries want to be even more standalone, and not require the boost cmake modules, then the author could provide more conditionals in their cmake to provide that, but i think that is up to the author. I don’t really see it necessary.

Paul


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Peter Dimov-2
Paul Fultz II wrote:
> Instead a top-level cmake can bring the packages together, something like:
>
>     set(BOOST_ALL TRUE)
>     # Whatever cmake modulues boost provide
>     include(cmake/BoostModules.cmake)

Something like that, although BOOST_ALL does not strike me as quite correct
a name, I'd call that BOOST_IS_LOCAL. (It doesn't have to be all of Boost,
could be a subset.)

We'll probably also need

set(BOOST_ROOT ${CMAKE_SOURCE_DIR})

>     # Include the libraries
>     # Perhaps this could be globbed
>     add_subdirectory(libs/fit)
>     add_subdirectory(libs/hana)

It has to be globbed, for the same reason. Subsets should be supported.

> Then in the library, the cmake would do something like this:
>
>     # We are building standalone
>     if(NOT BOOST_ALL)
>         # BoostModules must be installed first, before building standalone
>         find_package(BoostModulues)
>     endif()
>
>     # Find the dependent boost libraries
>     find_boost_lib(BOOST_LIBS asio hana)
>
> So now, when building standalone `find_boost_lib` will call `find_package`
> to retrieve the exported targets. If its building in-tree, then
> `find_boost_lib` will just find the target name, and let cmake resolve the
> target during build.

Yes, this could work for possibly-standalone libraries.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Paul Fultz II

> On Oct 2, 2016, at 8:22 PM, Peter Dimov <[hidden email]> wrote:
>
> Paul Fultz II wrote:
>> Instead a top-level cmake can bring the packages together, something like:
>>
>>    set(BOOST_ALL TRUE)
>>    # Whatever cmake modulues boost provide
>>    include(cmake/BoostModules.cmake)
>
> Something like that, although BOOST_ALL does not strike me as quite correct a name, I'd call that BOOST_IS_LOCAL. (It doesn't have to be all of Boost, could be a subset.)

Yes, of course, thats probably a better name. I really couldn’t think of a good name, so I just put that.

>
> We'll probably also need
>
> set(BOOST_ROOT ${CMAKE_SOURCE_DIR})

This shouldn’t be needed. `CMAKE_SOURCE_DIR` is always the directory of the top root. `PROJECT_SOURCE_DIR` will give the directory of the current project. There can be multiple projects in cmake, and, ideally, there would be a project for each library or submodule.

Paul


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Peter Dimov-2
Paul Fultz II wrote:
> > We'll probably also need
> >
> > set(BOOST_ROOT ${CMAKE_SOURCE_DIR})
>
> This shouldn’t be needed. `CMAKE_SOURCE_DIR` is always the directory of
> the top root.

I was thinking of the case where a library is built out of tree but there
does exist a complete Boost source tree and BOOST_ROOT (from the
environment, for instance) points to it. Some libraries seem to support
that, at least in theory judging by their Jamfiles.

I'm also not quite clear on what the equivalent of our current stage and
install targets should be, and how we'll handle tools/*.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: CMake Modularization

Peter Dimov-2
> I was thinking of the case where a library is built out of tree but there
> does exist a complete Boost source tree and BOOST_ROOT (from the
> environment, for instance) points to it. Some libraries seem to support
> that, at least in theory judging by their Jamfiles.

Although now that I think of it more, this case probably doesn't translate
well to CMake.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost