About all these metaprogramming libraries

[Boost - Dev mailing list]

> However, Boost needs one such library, not four. I think we can't just do 4
> reviews and include all the libraries that pass in Boost; that would be a
> huge disservice to our users, who will be left with the burden of choosing.
> Heck, if we can't even make our mind, how can they?
>
> So, how do we pick? Have we ever been in a similar situation where we have
> multiple competing libraries solving _exactly_ the same problem?

Can I ask a stupid question?

If Hana is the slow, "full fat" metaprogramming library, is there space
remaining for exactly one fast, "lightweight" metaprogramming library?

You seem to assume this de facto to be the case, but I'd like to know why?

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On 18 March 2017 at 14:32, Louis Dionne via Boost <[hidden email]>
wrote:

> However, Boost needs one such library, not four.


Maybe, maybe not. What would help though if there would be a (ONE) document
outlining what the differences are and in what domain/context one should
prefer on library over the other, complemented with pros and cons of each
lib. This (TMP) is not simple stuff unless one is heavilly involved (are a
library writer). An 'average' boost user is not capable within reasonable
time and with reasonable effort to make an informed choice from amongst all
these similar libraries. I must say, I like Peters' 'simple' mp11 though,
providing building blocks to more advanced stuff (I would say (generally)
the STL-spirit).

degski

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On Sun, Mar 19, 2017 at 4:22 PM, degski via Boost <[hidden email]>
wrote:

> I must say, I like Peters' 'simple' mp11 though,
> providing building blocks to more advanced stuff (I would say (generally)
> the STL-spirit).
>

I agree being simple is a plus, but IMO there is nothing exceptionally
simple about mp11.
Metal was designed to be as a simple building block as possible as well
[1], the only difference is that it approaches this goal in different way.

[1]: http://brunocodutra.github.io/metal/#concepts

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

Le 19/03/2017 à 16:22, degski via Boost a écrit :
> On 18 March 2017 at 14:32, Louis Dionne via Boost <[hidden email]>
> wrote:
>
>> However, Boost needs one such library, not four.
>
> Maybe, maybe not. What would help though if there would be a (ONE) document
> outlining what the differences are and in what domain/context one should
> prefer on library over the other, complemented with pros and cons of each
> lib.

Including Hana.

The best would be only one library for all TMP needs, but if there is
more than one, such a document is of the essence.

---
Loïc


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On 2017-03-19 17:13, Bruno Dutra via Boost wrote:

> On Sun, Mar 19, 2017 at 4:22 PM, degski via Boost
> <[hidden email]>
> wrote:
>
>> I must say, I like Peters' 'simple' mp11 though,
>> providing building blocks to more advanced stuff (I would say
>> (generally)
>> the STL-spirit).
>>
>
> I agree being simple is a plus, but IMO there is nothing exceptionally
> simple about mp11.

Hi,

I would disagree. Conceptionally it is a lot easier to grasp as
everything is defined either in language primitives or logical
constraints.

Here are the concepts:

A list is any type of the form L<T...>.
A set is a list where all elements are unique
A map is a set of keys zipped with an arbitrary list of same size.
A metafunction is a template alias with unspecified template parameters.

The definition of the concepts do not need to reference any primitive
inside the library and thus it can work well with many other of the
proposed libraries and libraries outside of boost, without having to
jump through additional hoops (e.g. renaming/"copying" arguments).

For me the biggest question is: will it work natively with mpl::vector?
This would make it a clear winner in my book.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

Oswin Krause wrote:

> For me the biggest question is: will it work natively with mpl::vector?

Not very well at the moment; mpl::vector<int, void, float> is a list of
fixed size 20, whereas mpl::vector<int, void, float>::type is of size 3, but
still fixed. It will only work with algorithms that don't need to change the
size of the list.

It would be possible in principle to make the algorithms take mpl::vector
and return mp_list, I suppose.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On Sun, Mar 19, 2017 at 7:43 PM, Oswin Krause <
[hidden email]> wrote:

> On 2017-03-19 17:13, Bruno Dutra via Boost wrote:
>
>> On Sun, Mar 19, 2017 at 4:22 PM, degski via Boost <[hidden email]>
>> wrote:
>>
>> I must say, I like Peters' 'simple' mp11 though,
>>> providing building blocks to more advanced stuff (I would say (generally)
>>> the STL-spirit).
>>>
>>>
>> I agree being simple is a plus, but IMO there is nothing exceptionally
>> simple about mp11.
>>
>
> Hi,
>
> I would disagree. Conceptionally it is a lot easier to grasp as everything
> is defined either in language primitives or logical constraints.
>
> Here are the concepts:
>
> A list is any type of the form L<T...>.
> A set is a list where all elements are unique
> A map is a set of keys zipped with an arbitrary list of same size.
> A metafunction is a template alias with unspecified template parameters.
>
> The definition of the concepts do not need to reference any primitive
> inside the library and thus it can work well with many other of the
> proposed libraries and libraries outside of boost, without having to jump
> through additional hoops (e.g. renaming/"copying" arguments).
>

You've pretty much described Metal, except that Metal requires lists to be
specializations of metal::list, but that is far from a shortcoming, in fact
there is a very good reason behind that design choice.

What do you expect to happen if you try to erase a value from a template,
such as std::unique_ptr, that has a default parameter using mp11? What
about inserting an element? Templates with default parameters are valid
lists if the template that represents a List is not specified, so
algorithms must be well defined for all these cases, which is very
cumbersome to handle on the library side for very little gain, not to
mention that it often leads to surprising and unexpected behavior.

For me the biggest question is: will it work natively with mpl::vector?
> This would make it a clear winner in my book.
>

Working natively with mpl::vector is only possible through adaptors, either
implicit in the library, or explicit to the user, because the type of the
concept behind mpl::vector is unspecified by design. Personally I think
there is very little gain in embedding such adaptors implicitly in any
library, because they bring along a lot of legacy and make the life of
maintainers miserable for basically no gain, since a explicit adaptor, such
as from_mpl<>, has zero cost to the user.

My choice for Metal was to provide metal::from_mpl, a metafunction that
converts any MPL concept to its equivalent in Metal parlance.

Bruno

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On Sun, Mar 19, 2017 at 8:09 PM, Peter Dimov via Boost <
[hidden email]> wrote:

> Oswin Krause wrote:
>
> For me the biggest question is: will it work natively with mpl::vector?
>>
>
> Not very well at the moment; mpl::vector<int, void, float> is a list of
> fixed size 20, whereas mpl::vector<int, void, float>::type is of size 3,
> but still fixed. It will only work with algorithms that don't need to
> change the size of the list.
>
> It would be possible in principle to make the algorithms take mpl::vector
> and return mp_list, I suppose.
>

You forgot to take into account that algorithms such as mpl::insert and
mpl::erase return proxies that inherit from vectors and don't expose the
elements in their types signatures, while being themselves valid instances
of a Vector. This is the real deal breaker.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Louis Dionne]

If Hana is the slow, "full fat" metaprogramming library,

Hana is not a "slow, full fat" metaprogramming library. Quite far from that. It's just that for pure type-level metaprogramming (which by the way is quite rarely needed nowadays with deduced return types), we can't possibly be as fast as Brigand, Metal or such type-level only solutions that is highly optimized for that precise use case.

is there space remaining for exactly one fast, "lightweight" metaprogramming library?

The problem is that the term metaprogramming is overloaded to mean several things. It can either mean pure type-level metaprogramming (MPL world), or algorithms on tuples (Fusion world). It HAPPENS that the latter is powerful enough to perform the former; this is what Hana allows and then you only need one metaprogramming library. However, that is heavier compile-time wise and does not scale as well as a library that was crafter with type computations only from the start. And if that was your question, I don't think you can do what Hana does (i.e. handle Fusion + MPL) while being significantly faster. You might be able to be some small constant factor faster, but that's it.

By the way, these libraries for pure type-level metaprogramming have the same problem, it's just that their constant factors are smaller than Hana's. Take any of these libraries and try to process a type list with 200K elements while doing something non-trivial. The fundamental problem is that we need language-level support for metaprogramming if we want to go completely crazy with it, and some of us are working exactly on this right now for C++next (or maybe the one after).

Louis

[Boost - Dev mailing list]

> On Mar 18, 2017, at 7:02 PM, Peter Dimov via Boost <[hidden email]> wrote:
>
> Paul Fultz II wrote:
>
>> It would be nice to see better collaboration between authors to build an unifying metaprogramming library, rather than competing against each other. Of course, this can be somewhat difficult as they have different fundamental concepts.
>
> As I see it, it's the fundamental concepts competing against each other, and that's how it should be; how else we'll find out which approach is best?

Well brigand and mpl11 seem very similar conceptually. There is a conceptual difference with Metal where everything is explicit. However, I don’t think we have to choose between a implicit and explicit API. Rather a library could provide both APIs, much like Boost.PP provides an API for both explicit and implicit(ie deduced) re-entrance.


Furthermore, the simple API of mpl11 very likely could be implemented using Metal underneath. Perhaps instead of providing two different libraries we provide one library with metal::explicit_ and metal::implicit where the implicit API could follow mpl11 very closely(if thats what seems to be the best approach to the implicit part considering other APIs).

Paul



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

Bruno Dutra wrote:
> > It would be possible in principle to make the algorithms take
> > mpl::vector and return mp_list, I suppose.
>
> You forgot to take into account that algorithms such as mpl::insert and
> mpl::erase return proxies that inherit from vectors and don't expose the
> elements in their types signatures, while being themselves valid instances
> of a Vector. This is the real deal breaker.

On MSVC, insert/erase return numbered vectors, but on g++ and clang++,
you're right, they return some weird v_item sequences.

What is our use case here though? From where would those mpl types come?


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On Sun, Mar 19, 2017 at 9:29 PM, Peter Dimov via Boost <
[hidden email]> wrote:

> Bruno Dutra wrote:
>
>> > It would be possible in principle to make the algorithms take >
>> mpl::vector and return mp_list, I suppose.
>>
>> You forgot to take into account that algorithms such as mpl::insert and
>> mpl::erase return proxies that inherit from vectors and don't expose the
>> elements in their types signatures, while being themselves valid instances
>> of a Vector. This is the real deal breaker.
>>
>
> On MSVC, insert/erase return numbered vectors, but on g++ and clang++,
> you're right, they return some weird v_item sequences.
>
> What is our use case here though? From where would those mpl types come?
>

I don't have a use case, I'm just trying to argue against the idea of
implementing any implicit native support for MPL sequences, which besides
non-trivial is also fruitless.

Moreover, providing support for numbered instances of mpl::vector only and
not for its proxies would really be a bad idea, because it doesn't solve
the common use case, where the user has some legacy metaprogram that
manipulates mpl::vectors, certainly also through insert and erase, and
wants to pass this on to the algorithms of a new library that advertises
itself as interoperating with MPL natively, only to realize the hard way
that it doesn't really.

I stand by the opinion that it can't go wrong with explicit converters.

Bruno

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On 3/19/17 12:31, Bruno Dutra via Boost wrote:
> On Sun, Mar 19, 2017 at 7:43 PM, Oswin Krause <
> [hidden email]> wrote:
>
>> On 2017-03-19 17:13, Bruno Dutra via Boost wrote:
>>
>>> On Sun, Mar 19, 2017 at 4:22 PM, degski via Boost <[hidden email]>
>>> wrote:

<snip features and criticisms of various libraries>

>
> You've pretty much described Metal, except that Metal requires lists to be
> specializations of metal::list, but that is far from a shortcoming, in fact
> there is a very good reason behind that design choice.
>

<snip more things why-I-designed-my-library-this-way>

Here is a suggestions. I don't think this thread is the place for the
authors to explain the benefits of their design/implementation/usage.
There will be a lot of time for that.

In this thread, we are trying (as a community) to figure out the best
approach to deal with multiple submissions of TMP libraries. Lets
concentrate on that concern.

Don't worry ... there will be plenty of opportunity for critique and
defense  (o;

michael

--
Michael Caisse
Ciere Consulting
ciere.com

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On 19/03/2017 19:57, Louis Dionne via Boost wrote:
>
>> If Hana is the slow, "full fat" metaprogramming library,
>
> Hana is not a "slow, full fat" metaprogramming library. Quite far from that.
> It's just that for pure type-level metaprogramming (which by the way is
> quite rarely needed nowadays with deduced return types), we can't possibly
> be as fast as Brigand, Metal or such type-level only solutions that is
> highly optimized for that precise use case.

Surely you forgot to append the condition "we can't possibly be as fast
as ... *with current compilers*"?

>> is there space remaining for exactly one fast, "lightweight"
>> metaprogramming library?
>
> The problem is that the term metaprogramming is overloaded to mean several
> things. It can either mean pure type-level metaprogramming (MPL world), or
> algorithms on tuples (Fusion world). It HAPPENS that the latter is powerful
> enough to perform the former; this is what Hana allows and then you only
> need one metaprogramming library. However, that is heavier compile-time wise
> and does not scale as well as a library that was crafter with type
> computations only from the start. And if that was your question, I don't
> think you can do what Hana does (i.e. handle Fusion + MPL) while being
> significantly faster. You might be able to be some small constant factor
> faster, but that's it.

Again, Hana is slower than these other libraries because *current
compilers* have been heavily optimised for type based metaprogramming,
and not yet for deduced return type metaprogramming.

But this won't be the situation soon. The Visual Studio team are
actively targeting MSVC at Hana. Knowing them, even after it's compiling
100% clean they'll then go on to make MSVC *fast* with Hana. I don't
doubt the clang guys are also thinking the same thing, that a Hana based
benchmark could be real useful to shining a spotlight on code which is
currently highly suboptimal in their compilers.

(As an aside, a Hana based compiler performance benchmark would be
awesome, you should write one with all that free time you have :) and
put it online with graphs or something)

> By the way, these libraries for pure type-level metaprogramming have the
> same problem, it's just that their constant factors are smaller than Hana's.
> Take any of these libraries and try to process a type list with 200K
> elements while doing something non-trivial. The fundamental problem is that
> we need language-level support for metaprogramming if we want to go
> completely crazy with it, and some of us are working exactly on this right
> now for C++next (or maybe the one after).

So I've got to ask the question: Are we not jumping the gun here with
adding any metaprogramming library to Boost beyond Hana? If we wait a
few years, the performance gap between Hana and any of these other
libraries will probably close very significantly to the point of
inconsequence.

Let me put this another way: do any of the other metaprogramming
libraries add *in pure API or capability terms* anything over Hana?
Because if they add nothing new in terms of a better API, or able to do
extra stuff, there is a *strong* argument that we should just wait and
see what future compilers look like before adding more metaprogramming
libraries.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On Mar 20, 2017 12:42 AM, "Niall .


Let me put this another way: do any of the other metaprogramming
libraries add *in pure API or capability terms* anything over Hana?


I don't mean to derail the discussion again, but since you asked, Metal
provides SFINAE-friendliness guarantees that Hana does not. Think
std::enable_if on steroids.

Bruno,

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

>     Let me put this another way: do any of the other metaprogramming
>     libraries add *in pure API or capability terms* anything over Hana?
>
>
> I don't mean to derail the discussion again, but since you asked, Metal
> provides SFINAE-friendliness guarantees that Hana does not. Think
> std::enable_if on steroids.

Thanks for the info.

However to continue to play devil's advocate, is that a highly
motivating reason for an additional metaprogramming library over Hana in
Boost when Concepts will ship in every major compiler this year or next?

If someone were proposing a metaprogramming library which allowed me to
write code which uses Concepts on a supporting compiler, but fell back
to a C++ 14 emulation on older compilers, that I'd find that a
compelling reason for an additional Boost metaprogramming library.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

Am 19.03.2017 um 23:00 schrieb Michael Caisse via Boost:

> On 3/19/17 12:31, Bruno Dutra via Boost wrote:
>> On Sun, Mar 19, 2017 at 7:43 PM, Oswin Krause <
>> [hidden email]> wrote:
>>
>>> On 2017-03-19 17:13, Bruno Dutra via Boost wrote:
>>>
>>>> On Sun, Mar 19, 2017 at 4:22 PM, degski via Boost <[hidden email]>
>>>> wrote:
>
> <snip features and criticisms of various libraries>
>
>>
>> You've pretty much described Metal, except that Metal requires lists to be
>> specializations of metal::list, but that is far from a shortcoming, in fact
>> there is a very good reason behind that design choice.
>>
>
> <snip more things why-I-designed-my-library-this-way>
>
> Here is a suggestions. I don't think this thread is the place for the
> authors to explain the benefits of their design/implementation/usage.
> There will be a lot of time for that.
>
> In this thread, we are trying (as a community) to figure out the best
> approach to deal with multiple submissions of TMP libraries. Lets
> concentrate on that concern.

As mentioned before, I think some kind of comparison between such TMP
libraries is required. A comparison between their features/concepts and
especially the users' needs they try to address with these
features/concepts.

For example, from a user's point of view I would pretty much like to see
a "drop-in" replacement for Boost.MPL which compiles (much) faster (and
uses less RAM doing so).
However, I am not really interested in using such a replacement by
myself. I am instead interested in all existing Boost libraries that
currently use Boost.MPL to "magically" become/compile faster (and use
less RAM).

Of course, if all Boost-libraries that use Boost.MPL are actively
maintained and their maintainers would apply internal changes so that
these libraries would use another (faster) TMP library (e.g. Boost.Hana
or what might be adopted next), that would be fine, too.
However, it probably is less work for the maintainer to only adjust some
includes and maybe some typedefs to use a "drop-in" replacement instead
of changing a lot of code to use another TMP library which is completely
different from Boost.MPL.

>
> Don't worry ... there will be plenty of opportunity for critique and
> defense  (o;
>
> michael
>

Just my "user's view".

Deniz

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

Deniz Bahadir wrote:

> As mentioned before, I think some kind of comparison between such TMP
> libraries is required.

Error C6301: passive voice used: "is required"

Who should produce this comparison?


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

On Mon, Mar 20, 2017 at 4:00 PM, Peter Dimov via Boost
<[hidden email]> wrote:
>> As mentioned before, I think some kind of comparison between such TMP
>> libraries is required.
>
> Error C6301: passive voice used: "is required"
>
> Who should produce this comparison?

I agree with Deniz if we can rephrase his words slightly, changing
required to desirable.
I have myself some knowledge of MPL and a little bit of knowledge of
Brigand. But my knowledge is fragmented, having used only what I use
in my projects. It would be quite nice to have an overview of the
capabilities and differences of the different libraries. For metal and
brigand that should not be to difficult as I know that the authors
have followed both libraries.
For your library, I read your excellent posts on metaprogramming, but
I was unaware that it had become a public library before you submitted
it to boost.

/Peter

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Boost - Dev mailing list]

Am 20.03.2017 um 16:00 schrieb Peter Dimov via Boost:
> Deniz Bahadir wrote:
>
>> As mentioned before, I think some kind of comparison between such TMP
>> libraries is required.
>
> Error C6301: passive voice used: "is required"
>
> Who should produce this comparison?

The answer to that question is one of the goals of the current
discussion, if I understood Michael correctly. ;-)

But jokes aside:

I would recommend, the authors of the different TMP libraries create a
short list of the concepts/features used and the problems and use-cases
they want to address with their libraries. (Probably, such a list should
also be available for Boost.Hana and possibly Boost.MPL. Although I
doubt that anybody would be pleased to just continue using Boost.MPL
instead of one of the new TMP-libraries.)
Experienced/interested TMP developers/users could probably help doing so.

Additionally, users of TMP libraries (others or some of the
to-be-reviewed ones) should mention their use-cases or problems they
would like to see addressed. (E.g. as I did in my former email; hoping
for a "drop-in" replacement of Boost.MPL for other Boost libs.)

Checking which of the libraries addresses what needs might help to find
consensus in what really should and can be addressed and which of the
new libraries does the best job.
Maybe we will realize their might be two TMP libraries that best become
joined into a single one to address almost all of the user's needs? Then
such a joined TMP lib might be the best candidate to become reviewed and
adopted by Boost?

And just in case, you might ask me the same question. ;-)
The last step, the checking of addressed needs should probably be done
by all readers of the mailing-list that are interested in this topic via
discussion on this mailing-list.

As a result we might get a nice pre-selection for a review candidate and
as a side-effect gather more reviewers, because they already
participated in the pre-selection so that the review-barrier is/feels
lowered.

I hope that is not too unrealistic.
Deniz


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost