[vmd] Library Review

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

[vmd] Library Review

Paul Mensonides
(apologies for the late review)

-- Design --

Many of the utility components of the library have decent design.  There
are, however, a couple of aspects of the design that I dislike, and they
touch on a large chunk of the library.  These concerns all revolve
around the registration, recognition, parsing, and limitations of
various "v-types" and "v-sequences".

I do not yet see a strong use case for the "v-key"/"v-identifier"
recognition scenario, as opposed to just registering a "v-identifier".
For example,

     #define BOOST_VMD_MAP_<v-key><v-identifier>
     // ...

     BOOST_VMD_IS_IDENTIFIER(<v-seq>, <tuple-of-v-keys>)

Perhaps Edward can enlighten me about the use-cases for which this is
ideal.  I do see some use-cases, but in most cases it requires dispatch
based on what the particular "v-identifier" it is anyway.

One could do better for this low-level part with registrations such as
(without a BOOST_VMD_ prefix for brevity)

     #define IDENTIFIER_CIRCLE (CIRCLE),
     #define IDENTIFIER_SQUARE (SQUARE),
     #define IDENTIFIER_TRIANGLE (TRIANGLE),
     #define IDENTIFIER_RECTANGLE (RECTANGLE),

These definitions don't allow one to directly test for specific
"v-identifiers", but they do allow one to extract registered
"v-identifiers" from a "v-sequence".

If all such registrations are required to be the same, there is no need
for separate per-library prefixes.  Macros can be defined multiple times
if they are defined the same way.

Similarly, the "v-numbers" could be pre-registered by the library as

     #define NUMBER_0 (0),
     #define NUMBER_1 (1),
     #define NUMBER_2 (2),
     // ...

Tests for specific "v-identifiers" can be done via extraction from the
above and using macro definitions such as

     #define COMPARE_CIRCLE_CIRCLE ()
     #define COMPARE_SQUARE_SQUARE ()
     // ...

In many cases this sort of comparison is not needed, as one would
directly dispatch off of the "v-identifier" itself.

In any case, one could "parse" a "v-sequence" into "v-types" without
many of the limitations of the current design.  The following example
uses Chaos, but I do not think there is anything in the design that
fundamentally cannot be made to work on (e.g.) VC++.

     #include <chaos/preprocessor.h>

     #define NUMBER_1 (1),
     #define NUMBER_2 (2),
     #define NUMBER_3 (3),
     #define NUMBER_4 (4),
     #define NUMBER_5 (5),
     // ...

     // parser implementation...
     #define PARSE(vseq) PARSE_S(CHAOS_PP_STATE(), vseq)
     #define PARSE_S(s, vseq) \
         CHAOS_PP_SPLIT(0, CHAOS_PP_EXPR_S(s)(CHAOS_PP_WHILE_S( \
             s, PARSE_P, PARSE_O,, vseq \
         ))) \
         /**/
     #define PARSE_P(s, seq, vseq) \
         CHAOS_PP_COMPL(CHAOS_PP_IS_EMPTY_NON_FUNCTION(vseq)) \
         /**/
     #define PARSE_O(s, seq, vseq) \
         seq CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(vseq))( \
             PARSE_O_A, PARSE_O_B \
         )(vseq) \
         /**/
     #define PARSE_O_A(vseq) PARSE_O_A_A vseq
     #define PARSE_O_A_A(...) (3, (__VA_ARGS__)),
     #define PARSE_O_B(vseq) \
         CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(NUMBER_ ## vseq))( \
             PARSE_O_B_A, \
             CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(IDENTIFIER_ ## vseq))( \
                 PARSE_O_B_B, PARSE_O_B_C \
             ) \
         )(vseq) \
         /**/
     #define PARSE_O_B_A(vseq) \
         (1, CHAOS_PP_REM_CTOR( \
             CHAOS_PP_SPLIT(0, NUMBER_ ## vseq) \
         )), \
         CHAOS_PP_SPLIT(1, NUMBER_ ## vseq) \
         /**/
     #define PARSE_O_B_B(vseq) \
         (2, CHAOS_PP_REM_CTOR( \
             CHAOS_PP_SPLIT(0, IDENTIFIER_ ## vseq) \
         )), \
         CHAOS_PP_SPLIT(1, IDENTIFIER_ ## vseq) \
         /**/
     #define PARSE_O_B_C(vseq) (0, vseq),

     // registrations...
     #define IDENTIFIER_CIRCLE (CIRCLE),
     #define IDENTIFIER_SQUARE (SQUARE),
     #define IDENTIFIER_TRIANGLE (TRIANGLE),
     #define IDENTIFIER_RECTANGLE (RECTANGLE),

     PARSE(
         CIRCLE
         RECTANGLE
         (a, b, c)
         TRIANGLE
         3
         (xyz)
         UNREGISTERED
         SQUARE
     )

(Apologies for any mistakes in the above.  I wrote it up quickly.)

This particular example results in key-value pairs of the form:
(0, <unrecognized>)
(1, <v-number>)
(2, <registered-identifier>)
(3, <tuple>)

(2, CIRCLE) (2, RECTANGLE) (3, (a, b, c)) (2, TRIANGLE) (1, 3) (3,
(xyz)) (0, UNREGISTERED SQUARE)

There are a number of things that could be done instead of producing a
binary sequence like this (which Boost.Preprocessor cannot handle).  One
could fold the results via a user-supplied macro, for example.

Aside from the massive amount of clutter introduced by workarounds, many
of the pieces of the above are already available either in
Boost.Preprocessor or in other parts of the VMD library.  For example,
BOOST_VMD_IS_EMPTY is essentially CHAOS_PP_IS_EMPTY_NON_FUNCTION.

-- Implementation --

I have not looked closely at the implementation, but I do know what a
herculean effort it is to get this type of stuff working with VC++ in
particular.

-- Documentation --

There are few inaccuracies in the documentation related to motivations
for various Boost.Preprocessor things, but those could be fixed.

The documentation for BOOST_VMD_IS_EMPTY is decent except that
essentially any input that ends with a function-like macro name will
cause weird results or errors.  So much so that the documentation should
just disallow input that ends with a function-like macro name across the
board.

Other than that, the documentation needs some clean-up and organization
(particularly of definitions), but does a reasonable job of documenting
the library.  I do feel that a lot of documentation complexity comes
from the way that v-keys/v-identifiers (etc) are handled.

-- Usefulness --

There is a definite use-case for many of the utilities in the library,
and the fundamental concept v-sequence parsing.

-- Testing --

I tested the library with GCC and everything functioned as advertised.

-- Effort --

I did a fairly significant study of the design and interface of the library.

-- Domain Familiarity --

I had my skull opened up and had "preprocessor metaprogramming" tattooed
directly onto my brain.

--

There are many good parts of the library--including the herculean effort
to make it work on sub-par preprocessors such as VC++.  However, at this
point I would be hesitant to recommend acceptance based on the design of
the v-type recognition and v-sequence parsing part.

I am open to argument about why the current design is superior based on
use-cases or technical limitations introduced by (e.g.) VC++ or, if it
is not superior, a further review at a later date after a redesign
because the underlying concept is sound.

--

Regards,
Paul Mensonides

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Edward Diener-3
On 9/13/2014 9:44 AM, Paul Mensonides wrote:
> (apologies for the late review)

Your review is most welcome.

>
> -- Design --
>
> Many of the utility components of the library have decent design.  There
> are, however, a couple of aspects of+ the design that I dislike, and they
> touch on a large chunk of the library.  These concerns all revolve
> around the registration, recognition, parsing, and limitations of
> various "v-types" and "v-sequences".
>
> I do not yet see a strong use case for the "v-key"/"v-identifier"
> recognition scenario, as opposed to just registering a "v-identifier".
> For example,
>
>      #define BOOST_VMD_MAP_<v-key><v-identifier>
>      // ...
>
>      BOOST_VMD_IS_IDENTIFIER(<v-seq>, <tuple-of-v-keys>)
>
> Perhaps Edward can enlighten me about the use-cases for which this is
> ideal.  I do see some use-cases, but in most cases it requires dispatch
> based on what the particular "v-identifier" it is anyway.

The use case is to be able to determine if some preprocessor input
matches a v-identifier and then to logically work off of that decision.
The idea is to parse some input and, for instance, if the input is X,
create output A, and if the input is Y, create output B etc. etc., where
X and Y are v-identifiers.

In any given situation since the v-keys are being
supplied, the end-user knows that if 1 is returned identifier X has been
matched and if 2 is returned identifier Y has been matched, and of
course if 0 is returned no identifier match has been found. This enables
the end-user to dispatch on the result and to know which identifier has
been found. Admittedly it does not extract the identifier itself as
output from the macro.

>
> One could do better for this low-level part with registrations such as
> (without a BOOST_VMD_ prefix for brevity)
>
>      #define IDENTIFIER_CIRCLE (CIRCLE),
>      #define IDENTIFIER_SQUARE (SQUARE),
>      #define IDENTIFIER_TRIANGLE (TRIANGLE),
>      #define IDENTIFIER_RECTANGLE (RECTANGLE),

Is the ending comma there for a reason ?

>
> These definitions don't allow one to directly test for specific
> "v-identifiers", but they do allow one to extract registered
> "v-identifiers" from a "v-sequence".
>
> If all such registrations are required to be the same, there is no need
> for separate per-library prefixes.  Macros can be defined multiple times
> if they are defined the same way.
>
> Similarly, the "v-numbers" could be pre-registered by the library as
>
>      #define NUMBER_0 (0),
>      #define NUMBER_1 (1),
>      #define NUMBER_2 (2),
>      // ...

Ditto about the ending commas.

>
> Tests for specific "v-identifiers" can be done via extraction from the
> above and using macro definitions such as
>
>      #define COMPARE_CIRCLE_CIRCLE ()
>      #define COMPARE_SQUARE_SQUARE ()
>      // ...

Sure.

> In many cases this sort of comparison is not needed, as one would
> directly dispatch off of the "v-identifier" itself.
>

I appreciate your technique of matching a v-identifier/v-number by
returning a registered identifier/number and then dispatching off of the
identifier itself. It appears to make parsing a v-sequence much easier,
but I need to consider this further.

> In any case, one could "parse" a "v-sequence" into "v-types" without
> many of the limitations of the current design.  The following example
> uses Chaos, but I do not think there is anything in the design that
> fundamentally cannot be made to work on (e.g.) VC++.

This is a bit unfair. You are asking me to understand Chaos code as
opposed to Boost PP and VMD. But I think I get the idea from your
original suggestion above and will study it further.

>
>      #include <chaos/preprocessor.h>
>
>      #define NUMBER_1 (1),
>      #define NUMBER_2 (2),
>      #define NUMBER_3 (3),
>      #define NUMBER_4 (4),
>      #define NUMBER_5 (5),
>      // ...
>
>      // parser implementation...
>      #define PARSE(vseq) PARSE_S(CHAOS_PP_STATE(), vseq)
>      #define PARSE_S(s, vseq) \
>          CHAOS_PP_SPLIT(0, CHAOS_PP_EXPR_S(s)(CHAOS_PP_WHILE_S( \
>              s, PARSE_P, PARSE_O,, vseq \
>          ))) \
>          /**/
>      #define PARSE_P(s, seq, vseq) \
>          CHAOS_PP_COMPL(CHAOS_PP_IS_EMPTY_NON_FUNCTION(vseq)) \
>          /**/
>      #define PARSE_O(s, seq, vseq) \
>          seq CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(vseq))( \
>              PARSE_O_A, PARSE_O_B \
>          )(vseq) \
>          /**/
>      #define PARSE_O_A(vseq) PARSE_O_A_A vseq
>      #define PARSE_O_A_A(...) (3, (__VA_ARGS__)),
>      #define PARSE_O_B(vseq) \
>          CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(NUMBER_ ## vseq))( \
>              PARSE_O_B_A, \
>              CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(IDENTIFIER_ ## vseq))( \
>                  PARSE_O_B_B, PARSE_O_B_C \
>              ) \
>          )(vseq) \
>          /**/
>      #define PARSE_O_B_A(vseq) \
>          (1, CHAOS_PP_REM_CTOR( \
>              CHAOS_PP_SPLIT(0, NUMBER_ ## vseq) \
>          )), \
>          CHAOS_PP_SPLIT(1, NUMBER_ ## vseq) \
>          /**/
>      #define PARSE_O_B_B(vseq) \
>          (2, CHAOS_PP_REM_CTOR( \
>              CHAOS_PP_SPLIT(0, IDENTIFIER_ ## vseq) \
>          )), \
>          CHAOS_PP_SPLIT(1, IDENTIFIER_ ## vseq) \
>          /**/
>      #define PARSE_O_B_C(vseq) (0, vseq),
>
>      // registrations...
>      #define IDENTIFIER_CIRCLE (CIRCLE),
>      #define IDENTIFIER_SQUARE (SQUARE),
>      #define IDENTIFIER_TRIANGLE (TRIANGLE),
>      #define IDENTIFIER_RECTANGLE (RECTANGLE),
>
>      PARSE(
>          CIRCLE
>          RECTANGLE
>          (a, b, c)
>          TRIANGLE
>          3
>          (xyz)
>          UNREGISTERED
>          SQUARE
>      )
>
> (Apologies for any mistakes in the above.  I wrote it up quickly.)
>
> This particular example results in key-value pairs of the form:
> (0, <unrecognized>)
> (1, <v-number>)
> (2, <registered-identifier>)
> (3, <tuple>)
>
> (2, CIRCLE) (2, RECTANGLE) (3, (a, b, c)) (2, TRIANGLE) (1, 3) (3,
> (xyz)) (0, UNREGISTERED SQUARE)
>
> There are a number of things that could be done instead of producing a
> binary sequence like this (which Boost.Preprocessor cannot handle).  One
> could fold the results via a user-supplied macro, for example.
>
> Aside from the massive amount of clutter introduced by workarounds, many
> of the pieces of the above are already available either in
> Boost.Preprocessor or in other parts of the VMD library.  For example,
> BOOST_VMD_IS_EMPTY is essentially CHAOS_PP_IS_EMPTY_NON_FUNCTION.
>
> -- Implementation --
>
> I have not looked closely at the implementation, but I do know what a
> herculean effort it is to get this type of stuff working with VC++ in
> particular.

Thanks ! VC++ is horrible to make it work "correctly". But you already
know that.

>
> -- Documentation --
>
> There are few inaccuracies in the documentation related to motivations
> for various Boost.Preprocessor things, but those could be fixed.

Please feel free to point them out and I will update the documentation
accordingly.

>
> The documentation for BOOST_VMD_IS_EMPTY is decent except that
> essentially any input that ends with a function-like macro name will
> cause weird results or errors.  So much so that the documentation should
> just disallow input that ends with a function-like macro name across the
> board.

I don't think that input that ends with a function-like macro name, when
that macro takes 0 or 1 parameter, causes incorrect results on a C++
standard conforming compiler. Can you explain why it would cause an
incorrect result or an error ?

>
> Other than that, the documentation needs some clean-up and organization
> (particularly of definitions), but does a reasonable job of documenting
> the library.  I do feel that a lot of documentation complexity comes
> from the way that v-keys/v-identifiers (etc) are handled.

I agree that if I use your methodology of registering v-identifiers and
v-numbers then the documentation, as well as the code, for parsing
v-sequences might be much simplified and I may be automatically able to
parse any v-sequence into its v-types. I will have to take a look at
this a bit more.

>
> -- Usefulness --
>
> There is a definite use-case for many of the utilities in the library,
> and the fundamental concept v-sequence parsing.

Thanks ! I felt the ability to specify/parse a v-sequence would add
flexibility when designing macros. Also as well the ability to identify
v-types.

>
> -- Testing --
>
> I tested the library with GCC and everything functioned as advertised.
>
> -- Effort --
>
> I did a fairly significant study of the design and interface of the
> library.
>
> -- Domain Familiarity --
>
> I had my skull opened up and had "preprocessor metaprogramming" tattooed
> directly onto my brain.

Who coined that term anyway <g> ?

>
> --
>
> There are many good parts of the library--including the herculean effort
> to make it work on sub-par preprocessors such as VC++.  However, at this
> point I would be hesitant to recommend acceptance based on the design of
> the v-type recognition and v-sequence parsing part.
>
> I am open to argument about why the current design is superior based on
> use-cases or technical limitations introduced by (e.g.) VC++ or, if it
> is not superior, a further review at a later date after a redesign
> because the underlying concept is sound.

Thanks very much for your review. I will definitely revisit the
implementation of v-sequences and the identifying of v-identifiers and
v-numbers to see if I can make the library better and easier to use
based on your suggestions.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Edward Diener-3
In reply to this post by Paul Mensonides
On 9/13/2014 9:44 AM, Paul Mensonides wrote:

> (apologies for the late review)
>
> -- Design --
>
> Many of the utility components of the library have decent design.  There
> are, however, a couple of aspects of the design that I dislike, and they
> touch on a large chunk of the library.  These concerns all revolve
> around the registration, recognition, parsing, and limitations of
> various "v-types" and "v-sequences".
>
> I do not yet see a strong use case for the "v-key"/"v-identifier"
> recognition scenario, as opposed to just registering a "v-identifier".
> For example,
>
>      #define BOOST_VMD_MAP_<v-key><v-identifier>
>      // ...
>
>      BOOST_VMD_IS_IDENTIFIER(<v-seq>, <tuple-of-v-keys>)
>
> Perhaps Edward can enlighten me about the use-cases for which this is
> ideal.  I do see some use-cases, but in most cases it requires dispatch
> based on what the particular "v-identifier" it is anyway.
>
> One could do better for this low-level part with registrations such as
> (without a BOOST_VMD_ prefix for brevity)
>
>      #define IDENTIFIER_CIRCLE (CIRCLE),
>      #define IDENTIFIER_SQUARE (SQUARE),
>      #define IDENTIFIER_TRIANGLE (TRIANGLE),
>      #define IDENTIFIER_RECTANGLE (RECTANGLE),
>
> These definitions don't allow one to directly test for specific
> "v-identifiers", but they do allow one to extract registered
> "v-identifiers" from a "v-sequence".
>
> If all such registrations are required to be the same, there is no need
> for separate per-library prefixes.  Macros can be defined multiple times
> if they are defined the same way.
>
> Similarly, the "v-numbers" could be pre-registered by the library as
>
>      #define NUMBER_0 (0),
>      #define NUMBER_1 (1),
>      #define NUMBER_2 (2),
>      // ...

I realized why the registrations have a trailing comma. Brilliant. Thanks !



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Paul Mensonides
In reply to this post by Edward Diener-3
On 9/13/2014 1:51 PM, Edward Diener wrote:
> On 9/13/2014 9:44 AM, Paul Mensonides wrote:

>> Perhaps Edward can enlighten me about the use-cases for which this is
>> ideal.  I do see some use-cases, but in most cases it requires dispatch
>> based on what the particular "v-identifier" it is anyway.
>
> The use case is to be able to determine if some preprocessor input
> matches a v-identifier and then to logically work off of that decision.
> The idea is to parse some input and, for instance, if the input is X,
> create output A, and if the input is Y, create output B etc. etc., where
> X and Y are v-identifiers.
>
> In any given situation since the v-keys are being
> supplied, the end-user knows that if 1 is returned identifier X has been
> matched and if 2 is returned identifier Y has been matched, and of
> course if 0 is returned no identifier match has been found. This enables
> the end-user to dispatch on the result and to know which identifier has
> been found. Admittedly it does not extract the identifier itself as
> output from the macro.

What I meant WRT dispatching directly off of the v-identifier something
along the lines of extracting the identifier and then using it to form
the suffix of another macro.  I.e. a conditional without going through
an intermediate integer.  However, there are still cases where something
like

     #define COMPARE_ABC_ABC ()

is useful and direct comparison can be implemented with something like:

#include <chaos/preprocessor.h>

#define IDENTIFIER_CIRCLE (CIRCLE),
#define IDENTIFIER_SQUARE (SQUARE),
#define IDENTIFIER_TRIANGLE (TRIANGLE),
#define IDENTIFIER_RECTANGLE (RECTANGLE),

#define COMPARE_CIRCLE_CIRCLE ()
#define COMPARE_RECTANGLE_RECTANGLE ()
#define COMPARE_TRIANGLE_TRIANGLE ()
#define COMPARE_RECTANGLE_RECTANGLE ()

#define IS_IDENTIFIER(...) \
     CHAOS_PP_QUICK_OVERLOAD(IS_IDENTIFIER_, __VA_ARGS__)(__VA_ARGS__) \
     /**/
#define IS_IDENTIFIER_1(vseq) \
     CHAOS_PP_BITAND \
         (CHAOS_PP_COMPL(CHAOS_PP_IS_VARIADIC(vseq))) \
         (IS_IDENTIFIER_1A(vseq)) \
     /**/
#define IS_IDENTIFIER_1A(vseq) \
     CHAOS_PP_BITAND \
         (CHAOS_PP_IS_VARIADIC(IDENTIFIER_ ## vseq)) \
         (CHAOS_PP_IS_EMPTY_NON_FUNCTION( \
             CHAOS_PP_SPLIT(1, IDENTIFIER_ ## vseq) \
         )) \
     /**/
#define IS_IDENTIFIER_2(vseq, tuple) \
     IS_IDENTIFIER_3(CHAOS_PP_STATE(), vseq, tuple) \
     /**/
#define IS_IDENTIFIER_3(s, vseq, tuple) \
     CHAOS_PP_BITAND \
         (IS_IDENTIFIER_1(vseq)) \
         (CHAOS_PP_VARIADIC_ELEM(0, \
             CHAOS_PP_EXPR_S(s)(CHAOS_PP_TUPLE_FOLD_LEFT_S( \
                 s, IS_IDENTIFIER_3A, tuple, 0, \
                 CHAOS_PP_REM_CTOR( \
                     CHAOS_PP_SPLIT(0, IDENTIFIER_ ## vseq) \
                 ) \
             )) \
         )) \
        /**/
#define IS_IDENTIFIER_3A(s, e, b, id) \
     CHAOS_PP_BITOR \
         (b) \
         (CHAOS_PP_IS_VARIADIC(COMPARE_ ## e ## _ ## id)), id \
        /**/

IS_IDENTIFIER(RECTANGLE, (CIRCLE, SQUARE, RECTANGLE))

Note this isn't finding the particular index, but it could be made to do so.

>> One could do better for this low-level part with registrations such as
>> (without a BOOST_VMD_ prefix for brevity)
>>
>>      #define IDENTIFIER_CIRCLE (CIRCLE),
>>      #define IDENTIFIER_SQUARE (SQUARE),
>>      #define IDENTIFIER_TRIANGLE (TRIANGLE),
>>      #define IDENTIFIER_RECTANGLE (RECTANGLE),
>
> Is the ending comma there for a reason ?

Yes, though I think you already figured it out.  The parentheses are
there so that you can detect that your prefix token-pasting did
something, the "v-identifier" is there so you can extract it, and the
comma is there to separate the "v-identifier" from the rest of a
"v-sequence".

>> These definitions don't allow one to directly test for specific
>> "v-identifiers", but they do allow one to extract registered
>> "v-identifiers" from a "v-sequence".
>>
>> If all such registrations are required to be the same, there is no need
>> for separate per-library prefixes.  Macros can be defined multiple times
>> if they are defined the same way.
>>
>> Similarly, the "v-numbers" could be pre-registered by the library as
>>
>>      #define NUMBER_0 (0),
>>      #define NUMBER_1 (1),
>>      #define NUMBER_2 (2),
>>      // ...
>
> Ditto about the ending commas.

As above, only difference is prefix allowing a type-dispatch as opposed
to a value-dispatch.

>> Tests for specific "v-identifiers" can be done via extraction from the
>> above and using macro definitions such as
>>
>>      #define COMPARE_CIRCLE_CIRCLE ()
>>      #define COMPARE_SQUARE_SQUARE ()
>>      // ...
>
> Sure.
>
>> In many cases this sort of comparison is not needed, as one would
>> directly dispatch off of the "v-identifier" itself.
>>
>
> I appreciate your technique of matching a v-identifier/v-number by
> returning a registered identifier/number and then dispatching off of the
> identifier itself. It appears to make parsing a v-sequence much easier,
> but I need to consider this further.

One of the big things about it (IMO) is to make the registrations
reusable.  If something is defined the same, it can be defined more than
once.  All C/C++ keywords can also be registered by the library.

>> In any case, one could "parse" a "v-sequence" into "v-types" without
>> many of the limitations of the current design.  The following example
>> uses Chaos, but I do not think there is anything in the design that
>> fundamentally cannot be made to work on (e.g.) VC++.
>
> This is a bit unfair. You are asking me to understand Chaos code as
> opposed to Boost PP and VMD. But I think I get the idea from your
> original suggestion above and will study it further.

It is just a demonstration of a possibility.  I did it quickly so had to
reuse some existing stuff, but essentially it is just a while loop.  It
terminates when its (constrained) input is empty.  Its "operator" (which
iterates the data) tests the front the the "v-sequence" for a
parenthetic expression.  If it has one, the operator extracts it and
puts it into the result.  If it does not, it tests it (via
token-pasting) for starting with a "v-number".  If it does, the operator
extracts it and puts it into the result.  If it does not, it tests it
(again via token-pasting) for starting with a registered "v-identifier".
  If it does, the operator extracts it and puts it into the result.  If
it does not, it puts the entire remainder of the "v-sequence" into the
result.

>>
>>      #include <chaos/preprocessor.h>
>>
>>      #define NUMBER_1 (1),
>>      #define NUMBER_2 (2),
>>      #define NUMBER_3 (3),
>>      #define NUMBER_4 (4),
>>      #define NUMBER_5 (5),
>>      // ...
>>
>>      // parser implementation...
>>      #define PARSE(vseq) PARSE_S(CHAOS_PP_STATE(), vseq)
>>      #define PARSE_S(s, vseq) \
>>          CHAOS_PP_SPLIT(0, CHAOS_PP_EXPR_S(s)(CHAOS_PP_WHILE_S( \
>>              s, PARSE_P, PARSE_O,, vseq \
>>          ))) \
>>          /**/
>>      #define PARSE_P(s, seq, vseq) \
>>          CHAOS_PP_COMPL(CHAOS_PP_IS_EMPTY_NON_FUNCTION(vseq)) \
>>          /**/
>>      #define PARSE_O(s, seq, vseq) \
>>          seq CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(vseq))( \
>>              PARSE_O_A, PARSE_O_B \
>>          )(vseq) \
>>          /**/
>>      #define PARSE_O_A(vseq) PARSE_O_A_A vseq
>>      #define PARSE_O_A_A(...) (3, (__VA_ARGS__)),
>>      #define PARSE_O_B(vseq) \
>>          CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(NUMBER_ ## vseq))( \
>>              PARSE_O_B_A, \
>>              CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(IDENTIFIER_ ## vseq))( \
>>                  PARSE_O_B_B, PARSE_O_B_C \
>>              ) \
>>          )(vseq) \
>>          /**/
>>      #define PARSE_O_B_A(vseq) \
>>          (1, CHAOS_PP_REM_CTOR( \
>>              CHAOS_PP_SPLIT(0, NUMBER_ ## vseq) \
>>          )), \
>>          CHAOS_PP_SPLIT(1, NUMBER_ ## vseq) \
>>          /**/
>>      #define PARSE_O_B_B(vseq) \
>>          (2, CHAOS_PP_REM_CTOR( \
>>              CHAOS_PP_SPLIT(0, IDENTIFIER_ ## vseq) \
>>          )), \
>>          CHAOS_PP_SPLIT(1, IDENTIFIER_ ## vseq) \
>>          /**/
>>      #define PARSE_O_B_C(vseq) (0, vseq),
>>
>>      // registrations...
>>      #define IDENTIFIER_CIRCLE (CIRCLE),
>>      #define IDENTIFIER_SQUARE (SQUARE),
>>      #define IDENTIFIER_TRIANGLE (TRIANGLE),
>>      #define IDENTIFIER_RECTANGLE (RECTANGLE),
>>
>>      PARSE(
>>          CIRCLE
>>          RECTANGLE
>>          (a, b, c)
>>          TRIANGLE
>>          3
>>          (xyz)
>>          UNREGISTERED
>>          SQUARE
>>      )
>>
>> (Apologies for any mistakes in the above.  I wrote it up quickly.)
>>
>> This particular example results in key-value pairs of the form:
>> (0, <unrecognized>)
>> (1, <v-number>)
>> (2, <registered-identifier>)
>> (3, <tuple>)
>>
>> (2, CIRCLE) (2, RECTANGLE) (3, (a, b, c)) (2, TRIANGLE) (1, 3) (3,
>> (xyz)) (0, UNREGISTERED SQUARE)
>>
>> There are a number of things that could be done instead of producing a
>> binary sequence like this (which Boost.Preprocessor cannot handle).  One
>> could fold the results via a user-supplied macro, for example.
>>
>> Aside from the massive amount of clutter introduced by workarounds, many
>> of the pieces of the above are already available either in
>> Boost.Preprocessor or in other parts of the VMD library.  For example,
>> BOOST_VMD_IS_EMPTY is essentially CHAOS_PP_IS_EMPTY_NON_FUNCTION.

FYI

The CHAOS_PP_IS_VARIADIC macro is equivalent to your
BOOST_VMD_IS_BEGIN_TUPLE macro.  The CHAOS_PP_REM macro is just a
parentheses remover (e.g. #define REM(...) __VA_ARGS__).  The
CHAOS_PP_REM_CTOR macro is just a "constructed" parentheses remover
(e.g. #define REM_CTOR(...) REM __VA_ARGS__).  The CHAOS_PP_SPLIT macro
is basically just a head/tail macro of a comma separated list.

>> -- Implementation --
>>
>> I have not looked closely at the implementation, but I do know what a
>> herculean effort it is to get this type of stuff working with VC++ in
>> particular.
>
> Thanks ! VC++ is horrible to make it work "correctly". But you already
> know that.

Yes.

>> -- Documentation --
>>
>> There are few inaccuracies in the documentation related to motivations
>> for various Boost.Preprocessor things, but those could be fixed.
>
> Please feel free to point them out and I will update the documentation
> accordingly.

They are minor.  They have to do with the supposition that
Boost.Preprocessor is designed to not produce erroneous results given
erroneous inputs--which it isn't designed to do at all--and the reason
why the low-level detection macros (such as BOOST_PP_IS_NULLARY and
BOOST_PP_IS_EMPTY) is not because of their constrained input
requirements, but rather because using them frequently leads to the need
for hacks and workarounds in the code surrounding their use.  They are
"non-public" because they cannot fully encapsulate preprocessor workarounds.

>> The documentation for BOOST_VMD_IS_EMPTY is decent except that
>> essentially any input that ends with a function-like macro name will
>> cause weird results or errors.  So much so that the documentation should
>> just disallow input that ends with a function-like macro name across the
>> board.
>
> I don't think that input that ends with a function-like macro name, when
> that macro takes 0 or 1 parameter, causes incorrect results on a C++
> standard conforming compiler. Can you explain why it would cause an
> incorrect result or an error ?

Contrived but an easy demonstration:

#define MACRO(x) CAT(+, x C)
// valid input to MACRO is + or =

MACRO(+) // ++ C
MACRO() // error

BOOST_VMD_IS_EMPTY(MACRO) // error

You can get erroneous results too if the macro BOOST_VMD_BEGIN_TUPLE
used in the BOOST_VMD_DETAIL_IS_EMPTY_PROCESS macro somehow causes
another scan to be applied to its input.

>> Other than that, the documentation needs some clean-up and organization
>> (particularly of definitions), but does a reasonable job of documenting
>> the library.  I do feel that a lot of documentation complexity comes
>> from the way that v-keys/v-identifiers (etc) are handled.
>
> I agree that if I use your methodology of registering v-identifiers and
> v-numbers then the documentation, as well as the code, for parsing
> v-sequences might be much simplified and I may be automatically able to
> parse any v-sequence into its v-types. I will have to take a look at
> this a bit more.

I think it is possible.  The main difficulty will be VC++ because you
will have to bend over backwards to get those macro-generated commas to
form an argument separator in a variety of situations (if I recall
correctly).

>> I had my skull opened up and had "preprocessor metaprogramming" tattooed
>> directly onto my brain.
>
> Who coined that term anyway <g> ?

:) I don't know.

Regards,
Paul Mensonides


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Edward Diener-3
On 9/13/2014 7:05 PM, Paul Mensonides wrote:

> On 9/13/2014 1:51 PM, Edward Diener wrote:
>> On 9/13/2014 9:44 AM, Paul Mensonides wrote:
>
>>> Perhaps Edward can enlighten me about the use-cases for which this is
>>> ideal.  I do see some use-cases, but in most cases it requires dispatch
>>> based on what the particular "v-identifier" it is anyway.
>>
>> The use case is to be able to determine if some preprocessor input
>> matches a v-identifier and then to logically work off of that decision.
>> The idea is to parse some input and, for instance, if the input is X,
>> create output A, and if the input is Y, create output B etc. etc., where
>> X and Y are v-identifiers.
>>
>> In any given situation since the v-keys are being
>> supplied, the end-user knows that if 1 is returned identifier X has been
>> matched and if 2 is returned identifier Y has been matched, and of
>> course if 0 is returned no identifier match has been found. This enables
>> the end-user to dispatch on the result and to know which identifier has
>> been found. Admittedly it does not extract the identifier itself as
>> output from the macro.
>
> What I meant WRT dispatching directly off of the v-identifier something
> along the lines of extracting the identifier and then using it to form
> the suffix of another macro.  I.e. a conditional without going through
> an intermediate integer.

I understood that once you mentioned it.

>  However, there are still cases where something
> like
>
>      #define COMPARE_ABC_ABC ()
>
> is useful

I understand this. You concatenate your result to 'COMPARE_ABC_' and if
it expands to an empty single element tuple you know your result is
'ABC'. Ditto with any identifier 'XXX' for which you have
'#define COMPARE_XXX_XXX ()' macros.

> and direct comparison can be implemented with something like:
>
> #include <chaos/preprocessor.h>
>
> #define IDENTIFIER_CIRCLE (CIRCLE),
> #define IDENTIFIER_SQUARE (SQUARE),
> #define IDENTIFIER_TRIANGLE (TRIANGLE),
> #define IDENTIFIER_RECTANGLE (RECTANGLE),
>
> #define COMPARE_CIRCLE_CIRCLE ()
> #define COMPARE_RECTANGLE_RECTANGLE ()
> #define COMPARE_TRIANGLE_TRIANGLE ()
> #define COMPARE_RECTANGLE_RECTANGLE ()
>
> #define IS_IDENTIFIER(...) \
>      CHAOS_PP_QUICK_OVERLOAD(IS_IDENTIFIER_, __VA_ARGS__)(__VA_ARGS__) \
>      /**/
> #define IS_IDENTIFIER_1(vseq) \
>      CHAOS_PP_BITAND \
>          (CHAOS_PP_COMPL(CHAOS_PP_IS_VARIADIC(vseq))) \
>          (IS_IDENTIFIER_1A(vseq)) \
>      /**/
> #define IS_IDENTIFIER_1A(vseq) \
>      CHAOS_PP_BITAND \
>          (CHAOS_PP_IS_VARIADIC(IDENTIFIER_ ## vseq)) \
>          (CHAOS_PP_IS_EMPTY_NON_FUNCTION( \
>              CHAOS_PP_SPLIT(1, IDENTIFIER_ ## vseq) \
>          )) \
>      /**/
> #define IS_IDENTIFIER_2(vseq, tuple) \
>      IS_IDENTIFIER_3(CHAOS_PP_STATE(), vseq, tuple) \
>      /**/
> #define IS_IDENTIFIER_3(s, vseq, tuple) \
>      CHAOS_PP_BITAND \
>          (IS_IDENTIFIER_1(vseq)) \
>          (CHAOS_PP_VARIADIC_ELEM(0, \
>              CHAOS_PP_EXPR_S(s)(CHAOS_PP_TUPLE_FOLD_LEFT_S( \
>                  s, IS_IDENTIFIER_3A, tuple, 0, \
>                  CHAOS_PP_REM_CTOR( \
>                      CHAOS_PP_SPLIT(0, IDENTIFIER_ ## vseq) \
>                  ) \
>              )) \
>          )) \
>      /**/
> #define IS_IDENTIFIER_3A(s, e, b, id) \
>      CHAOS_PP_BITOR \
>          (b) \
>          (CHAOS_PP_IS_VARIADIC(COMPARE_ ## e ## _ ## id)), id \
>      /**/
>
> IS_IDENTIFIER(RECTANGLE, (CIRCLE, SQUARE, RECTANGLE))
>
> Note this isn't finding the particular index, but it could be made to do
> so.
>
>>> One could do better for this low-level part with registrations such as
>>> (without a BOOST_VMD_ prefix for brevity)
>>>
>>>      #define IDENTIFIER_CIRCLE (CIRCLE),
>>>      #define IDENTIFIER_SQUARE (SQUARE),
>>>      #define IDENTIFIER_TRIANGLE (TRIANGLE),
>>>      #define IDENTIFIER_RECTANGLE (RECTANGLE),
>>
>> Is the ending comma there for a reason ?
>
> Yes, though I think you already figured it out.  The parentheses are
> there so that you can detect that your prefix token-pasting did
> something, the "v-identifier" is there so you can extract it, and the
> comma is there to separate the "v-identifier" from the rest of a
> "v-sequence".

When I saw what the comma did that light in my brain went on <g>.

>
>>> These definitions don't allow one to directly test for specific
>>> "v-identifiers", but they do allow one to extract registered
>>> "v-identifiers" from a "v-sequence".
>>>
>>> If all such registrations are required to be the same, there is no need
>>> for separate per-library prefixes.  Macros can be defined multiple times
>>> if they are defined the same way.
>>>
>>> Similarly, the "v-numbers" could be pre-registered by the library as
>>>
>>>      #define NUMBER_0 (0),
>>>      #define NUMBER_1 (1),
>>>      #define NUMBER_2 (2),
>>>      // ...
>>
>> Ditto about the ending commas.
>
> As above, only difference is prefix allowing a type-dispatch as opposed
> to a value-dispatch.
>
>>> Tests for specific "v-identifiers" can be done via extraction from the
>>> above and using macro definitions such as
>>>
>>>      #define COMPARE_CIRCLE_CIRCLE ()
>>>      #define COMPARE_SQUARE_SQUARE ()
>>>      // ...
>>
>> Sure.
>>
>>> In many cases this sort of comparison is not needed, as one would
>>> directly dispatch off of the "v-identifier" itself.
>>>
>>
>> I appreciate your technique of matching a v-identifier/v-number by
>> returning a registered identifier/number and then dispatching off of the
>> identifier itself. It appears to make parsing a v-sequence much easier,
>> but I need to consider this further.
>
> One of the big things about it (IMO) is to make the registrations
> reusable.  If something is defined the same, it can be defined more than
> once.  All C/C++ keywords can also be registered by the library.

Understood. I agree this is a nice feature.

My own key-value v-identifier implementation was to try to make things
as simple as possible for the end-user. I now realize that if I keep it,
possibly along with your own methodology, as a primary or alternative
implementation of finding a v-identifier, it would have been better to
have the key-value macros expand to a single comma (',') rather than
expand to emptiness. Then the whole complication of parsing a single
input sequence for an identifier will be much easier. I really
appreciate your alerting me to the 'comma' technique. Substituting a
comma into an input sequence is easy to detect and will tell me if I
have matched an identifier or a number wthout having to recursively
parse the rest of the v-sequence or have to know what type of input
comes next. That is a wonderful improvement to VMD's overly complicated
v-sequence parsing macros, and I thank you very much for pointing it out.

Do realize that your own v-identifier technique requires dispatching off
of the v-identifier itself using concatenation, a technique some  macro
programmers may not know as well as dispatching off of a Boost PP number
using BOOST_PP_EQUAL. I agree that dispatching off of the identifier
itself is more elegant and a bit faster as far as the preprocessor is
concerned.

>
>>> In any case, one could "parse" a "v-sequence" into "v-types" without
>>> many of the limitations of the current design.  The following example
>>> uses Chaos, but I do not think there is anything in the design that
>>> fundamentally cannot be made to work on (e.g.) VC++.
>>
>> This is a bit unfair. You are asking me to understand Chaos code as
>> opposed to Boost PP and VMD. But I think I get the idea from your
>> original suggestion above and will study it further.
>
> It is just a demonstration of a possibility.  I did it quickly so had to
> reuse some existing stuff, but essentially it is just a while loop.  It
> terminates when its (constrained) input is empty.  Its "operator" (which
> iterates the data) tests the front the the "v-sequence" for a
> parenthetic expression.  If it has one, the operator extracts it and
> puts it into the result.  If it does not, it tests it (via
> token-pasting) for starting with a "v-number".  If it does, the operator
> extracts it and puts it into the result.  If it does not, it tests it
> (again via token-pasting) for starting with a registered "v-identifier".
>   If it does, the operator extracts it and puts it into the result.  If
> it does not, it puts the entire remainder of the "v-sequence" into the
> result.
>

Thanks for the explanation. I can follow it.

>>>
>>>      #include <chaos/preprocessor.h>
>>>
>>>      #define NUMBER_1 (1),
>>>      #define NUMBER_2 (2),
>>>      #define NUMBER_3 (3),
>>>      #define NUMBER_4 (4),
>>>      #define NUMBER_5 (5),
>>>      // ...
>>>
>>>      // parser implementation...
>>>      #define PARSE(vseq) PARSE_S(CHAOS_PP_STATE(), vseq)
>>>      #define PARSE_S(s, vseq) \
>>>          CHAOS_PP_SPLIT(0, CHAOS_PP_EXPR_S(s)(CHAOS_PP_WHILE_S( \
>>>              s, PARSE_P, PARSE_O,, vseq \
>>>          ))) \
>>>          /**/
>>>      #define PARSE_P(s, seq, vseq) \
>>>          CHAOS_PP_COMPL(CHAOS_PP_IS_EMPTY_NON_FUNCTION(vseq)) \
>>>          /**/
>>>      #define PARSE_O(s, seq, vseq) \
>>>          seq CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(vseq))( \
>>>              PARSE_O_A, PARSE_O_B \
>>>          )(vseq) \
>>>          /**/
>>>      #define PARSE_O_A(vseq) PARSE_O_A_A vseq
>>>      #define PARSE_O_A_A(...) (3, (__VA_ARGS__)),
>>>      #define PARSE_O_B(vseq) \
>>>          CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(NUMBER_ ## vseq))( \
>>>              PARSE_O_B_A, \
>>>              CHAOS_PP_IIF(CHAOS_PP_IS_VARIADIC(IDENTIFIER_ ## vseq))( \
>>>                  PARSE_O_B_B, PARSE_O_B_C \
>>>              ) \
>>>          )(vseq) \
>>>          /**/
>>>      #define PARSE_O_B_A(vseq) \
>>>          (1, CHAOS_PP_REM_CTOR( \
>>>              CHAOS_PP_SPLIT(0, NUMBER_ ## vseq) \
>>>          )), \
>>>          CHAOS_PP_SPLIT(1, NUMBER_ ## vseq) \
>>>          /**/
>>>      #define PARSE_O_B_B(vseq) \
>>>          (2, CHAOS_PP_REM_CTOR( \
>>>              CHAOS_PP_SPLIT(0, IDENTIFIER_ ## vseq) \
>>>          )), \
>>>          CHAOS_PP_SPLIT(1, IDENTIFIER_ ## vseq) \
>>>          /**/
>>>      #define PARSE_O_B_C(vseq) (0, vseq),
>>>
>>>      // registrations...
>>>      #define IDENTIFIER_CIRCLE (CIRCLE),
>>>      #define IDENTIFIER_SQUARE (SQUARE),
>>>      #define IDENTIFIER_TRIANGLE (TRIANGLE),
>>>      #define IDENTIFIER_RECTANGLE (RECTANGLE),
>>>
>>>      PARSE(
>>>          CIRCLE
>>>          RECTANGLE
>>>          (a, b, c)
>>>          TRIANGLE
>>>          3
>>>          (xyz)
>>>          UNREGISTERED
>>>          SQUARE
>>>      )
>>>
>>> (Apologies for any mistakes in the above.  I wrote it up quickly.)
>>>
>>> This particular example results in key-value pairs of the form:
>>> (0, <unrecognized>)
>>> (1, <v-number>)
>>> (2, <registered-identifier>)
>>> (3, <tuple>)
>>>
>>> (2, CIRCLE) (2, RECTANGLE) (3, (a, b, c)) (2, TRIANGLE) (1, 3) (3,
>>> (xyz)) (0, UNREGISTERED SQUARE)
>>>
>>> There are a number of things that could be done instead of producing a
>>> binary sequence like this (which Boost.Preprocessor cannot handle).  One
>>> could fold the results via a user-supplied macro, for example.
>>>
>>> Aside from the massive amount of clutter introduced by workarounds, many
>>> of the pieces of the above are already available either in
>>> Boost.Preprocessor or in other parts of the VMD library.  For example,
>>> BOOST_VMD_IS_EMPTY is essentially CHAOS_PP_IS_EMPTY_NON_FUNCTION.
>
> FYI
>
> The CHAOS_PP_IS_VARIADIC macro is equivalent to your
> BOOST_VMD_IS_BEGIN_TUPLE macro.  The CHAOS_PP_REM macro is just a
> parentheses remover (e.g. #define REM(...) __VA_ARGS__).  The
> CHAOS_PP_REM_CTOR macro is just a "constructed" parentheses remover
> (e.g. #define REM_CTOR(...) REM __VA_ARGS__).  The CHAOS_PP_SPLIT macro
> is basically just a head/tail macro of a comma separated list.
>
>>> -- Implementation --
>>>
>>> I have not looked closely at the implementation, but I do know what a
>>> herculean effort it is to get this type of stuff working with VC++ in
>>> particular.
>>
>> Thanks ! VC++ is horrible to make it work "correctly". But you already
>> know that.
>
> Yes.
>
>>> -- Documentation --
>>>
>>> There are few inaccuracies in the documentation related to motivations
>>> for various Boost.Preprocessor things, but those could be fixed.
>>
>> Please feel free to point them out and I will update the documentation
>> accordingly.
>
> They are minor.  They have to do with the supposition that
> Boost.Preprocessor is designed to not produce erroneous results given
> erroneous inputs--which it isn't designed to do at all--

I don't know where in the doc I stated the above. I was trying to
emphasize that Boost PP emphasized safety over the sort of functionality
which VMD provides, which depends more on end-users following certain
constraints on VMD input. Obviously depending on detecting emptiness as
a core functionality of VMD, where the test for emptiness, largely taken
from your own Internet code, can never be perfect, is an example of the
sort of riskier use of macros which VMD represents.

A number of people and a few reviewers also wanted to know why VMD, if
accepted, should not just be an addition to Boost PP and I was
expressing in my remarks to them a difference I perceived in philosophy
between Boost PP and VMD.

> and the reason
> why the low-level detection macros (such as BOOST_PP_IS_NULLARY and
> BOOST_PP_IS_EMPTY) is not because of their constrained input
> requirements, but rather because using them frequently leads to the need
> for hacks and workarounds in the code surrounding their use.  They are
> "non-public" because they cannot fully encapsulate preprocessor
> workarounds.

OK. My interpretation of why you did not document them publicly was
wrong. I really thought it was along the lines of "Because these macros
could be dangerous or erroneous given certain input I don't want to
document them for you." But if you are concerned with other issues
related to their use I understand.

>
>>> The documentation for BOOST_VMD_IS_EMPTY is decent except that
>>> essentially any input that ends with a function-like macro name will
>>> cause weird results or errors.  So much so that the documentation should
>>> just disallow input that ends with a function-like macro name across the
>>> board.
>>
>> I don't think that input that ends with a function-like macro name, when
>> that macro takes 0 or 1 parameter, causes incorrect results on a C++
>> standard conforming compiler. Can you explain why it would cause an
>> incorrect result or an error ?
>
> Contrived but an easy demonstration:
>
> #define MACRO(x) CAT(+, x C)
> // valid input to MACRO is + or =
>
> MACRO(+) // ++ C
> MACRO() // error
>
> BOOST_VMD_IS_EMPTY(MACRO) // error

You think of things I haven't even considered <g>. Anyway I will also
document this sort of situation in my discussion of emptiness in VMD.

>
> You can get erroneous results too if the macro BOOST_VMD_BEGIN_TUPLE
> used in the BOOST_VMD_DETAIL_IS_EMPTY_PROCESS macro somehow causes
> another scan to be applied to its input.
>

I really did already understand that ( I was bitten by it once when I
was programming VMD ) but I do not see how that could happen unless I
specifically change the BOOST_VMD_DETAIL_IS_EMPTY_PROCESS and mess it
up. But thanks for alerting me about it.

>>> Other than that, the documentation needs some clean-up and organization
>>> (particularly of definitions), but does a reasonable job of documenting
>>> the library.  I do feel that a lot of documentation complexity comes
>>> from the way that v-keys/v-identifiers (etc) are handled.
>>
>> I agree that if I use your methodology of registering v-identifiers and
>> v-numbers then the documentation, as well as the code, for parsing
>> v-sequences might be much simplified and I may be automatically able to
>> parse any v-sequence into its v-types. I will have to take a look at
>> this a bit more.
>
> I think it is possible.  The main difficulty will be VC++ because you
> will have to bend over backwards to get those macro-generated commas to
> form an argument separator in a variety of situations (if I recall
> correctly).

I am on the alert. Thanks ! I do like the comma technique enormously if
I can get it to work with the usually horrid VC++isms. All that
recursive processing I was doing with v-sequences, along with the
complicated input that was needed to parse v-identifiers and v-numbers
in the v-sequences, was hard work but I was too stupid to realize your
much better way.

>
>>> I had my skull opened up and had "preprocessor metaprogramming" tattooed
>>> directly onto my brain.
>>
>> Who coined that term anyway <g> ?
>
> :) I don't know.

Likely story <g>.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Paul Fultz II
In reply to this post by Paul Mensonides
> >> One could do better for this low-level part with registrations such as
> >> (without a BOOST_VMD_ prefix for brevity)
> >>
> >>      #define IDENTIFIER_CIRCLE (CIRCLE),
> >>      #define IDENTIFIER_SQUARE (SQUARE),
> >>      #define IDENTIFIER_TRIANGLE (TRIANGLE),
> >>      #define IDENTIFIER_RECTANGLE (RECTANGLE),
> >
> > Is the ending comma there for a reason ?
>
> Yes, though I think you already figured it out.  The parentheses are
> there so that you can detect that your prefix token-pasting did
> something, the "v-identifier" is there so you can extract it, and the
> comma is there to separate the "v-identifier" from the rest of a
> "v-sequence".

There is no need for the commas though. Its easy enough to insert them in.
Since we take sequence of tokens first:

    CIRCLE SQUARE

And then we concat the identifiers, which gives us this:

    (CIRCLE) SQUARE

Then we could use this:

    #define APPEND_COMMA_I(...) (__VA_ARGS__),
    #define APPEND_COMMA(...) APPEND_COMMA_I __VA_ARGS__

    APPEND_COMMA((CIRCLE) SQUARE) // Expands to (CIRCLE), SQUARE

It does require a little bit more for the implementation, however, this helps
to simplify the "interface", I think.

> I think it is possible.  The main difficulty will be VC++ because you
> will have to bend over backwards to get those macro-generated commas to
> form an argument separator in a variety of situations (if I recall
> correctly).

I actually use this `MSVC_INVOKE` macro to help in those situations:

    #define MSVC_INVOKE BOOST_PP_CAT(MSVC_INVOKE_, BOOST_PP_AUTO_REC(DETAIL_MSVC_INVOKE_P, 16))

    #define DETAIL_MSVC_INVOKE_P(n) BOOST_PP_IS_NULLARY( MSVC_INVOKE_ ## n((),) )

    #define MSVC_INVOKE_1(macro, args)  MSVC_INVOKE_I_1(macro, args)
    #define MSVC_INVOKE_2(macro, args)  MSVC_INVOKE_I_2(macro, args)
    ...
    #define MSVC_INVOKE_16(macro, args) MSVC_INVOKE_I_16(macro, args)

    #define MSVC_INVOKE_I_1(macro, args)  MSVC_INVOKE_X_1(macro args)
    #define MSVC_INVOKE_I_2(macro, args)  MSVC_INVOKE_X_2(macro args)
    ...
    #define MSVC_INVOKE_I_16(macro, args) MSVC_INVOKE_X_16(macro args)

    #define MSVC_INVOKE_X_1(x) x
    #define MSVC_INVOKE_X_2(x) x
    ...
    #define MSVC_INVOKE_X_16(x) x

Of course, I made it re-entrant which could slow down the preprocessor for
MSVC, however, it helps simplify when I need to workaround MSVC. So if I need
to call a macro with vardiac args, I can just call it like this:

    #define HEAD(x, ...) x
    MSVC_INVOKE(HEAD, (1,2,3,4)) // Expands to 1

Unfortunately, there are still times where this fails as well, but most of the
time this will work.

Finally, when I have done this DSL like parser(like in LINQ for example), I
would convert the string of tokens to a sequence and then process them
afterwards using the algorithms for a sequence. So I would essentially
transform this:

    #define STRING_from (from)
    #define STRING_where (where)
    #define STRING_select (select)

    STRING_TO_SEQ(from(x, numbers) where(x < 3) select(x * x))

That then expands to:

    (from)((x, numbers))(where)((x < 3))(select)((x * x))

Just in case you are curious, here is the code I used to implement
`STRING_TO_SEQ`, which works on MSVC as well:

    //
    // IS_PAREN
    //
    #define IS_PAREN(x) IS_PAREN_CHECK(IS_PAREN_PROBE x)
    #ifndef _MSC_VER
    #define IS_PAREN_CHECK(...) IS_PAREN_CHECK_N(__VA_ARGS__,0)
    #else
    #define IS_PAREN_CHECK(...) MSVC_INVOKE(IS_PAREN_CHECK_N, (__VA_ARGS__,0))
    #endif
    #define IS_PAREN_PROBE(...) ~, 1,
    #define IS_PAREN_CHECK_N(x, n, ...) n

    //
    // IS_EMPTY
    //
    #define IS_EMPTY(x) BOOST_PP_CAT(DETAIL_IS_EMPTY_, IS_PAREN(x))(x)
    #define DETAIL_IS_EMPTY_0(x) BOOST_PP_IS_EMPTY(x)
    #define DETAIL_IS_EMPTY_1(x) 0

    //
    // HEAD retrieves the first element of a sequence.
    // Example:
    //
    //     HEAD((1)(2)(3)) // Expands to (1)
    //
    #define HEAD(x) PICK_HEAD(MARK x)
    #define MARK(...) (__VA_ARGS__),
    #define PICK_HEAD(...) PICK_HEAD_I(__VA_ARGS__,)
    #ifndef _MSC_VER
    #define PICK_HEAD_I(x, ...) x
    #else
    // MSVC workarounds
    #define PICK_HEAD_II(x, ...) x
    #define PICK_HEAD_I(...) MSVC_INVOKE(PICK_HEAD_II, (__VA_ARGS__))
    #endif

    //
    // TAIL
    //
    #define EAT(...)
    #define TAIL(x) EAT x

    //
    // STRING_TOKEN
    //
    #define STRING_TOKEN(x) BOOST_PP_IIF(IS_PAREN(x), STRING_TOKEN_PAREN, STRING_TOKEN_KEYWORD)(x)
    #define STRING_TOKEN_KEYWORD(x) STRING_TOKEN_KEYWORD_CHECK(BOOST_PP_CAT(STRING_, x), x)
    #define STRING_TOKEN_KEYWORD_CHECK(tokened, raw) BOOST_PP_IIF(IS_PAREN(tokened), tokened, (raw))
    #define STRING_TOKEN_PAREN(x) (HEAD(x)) TAIL(x)

    //
    // STRING_TO_SEQ
    //
    #define STRING_TO_SEQ(x) STRING_TO_SEQ_WHILE_M \
    ( \
        BOOST_PP_WHILE(STRING_TO_SEQ_WHILE_P, STRING_TO_SEQ_WHILE_O, (,x)) \
    )

    #define STRING_TO_SEQ_WHILE_P(r, state) STRING_TO_SEQ_P state
    #define STRING_TO_SEQ_WHILE_O(r, state) STRING_TO_SEQ_O state
    #define STRING_TO_SEQ_WHILE_M(state) STRING_TO_SEQ_M state

    #define STRING_TO_SEQ_P(prev, tail) BOOST_PP_NOT(IS_EMPTY(tail))
    #define STRING_TO_SEQ_O(prev, tail)  \
       STRING_TO_SEQ_REPLACE(prev, STRING_TOKEN(tail))

    #define STRING_TO_SEQ_REPLACE(prev, tail) \
        (prev HEAD(tail), TAIL(tail))

    #define STRING_TO_SEQ_M(prev, tail) prev


Paul Fultz II
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Edward Diener-3
On 9/14/2014 10:58 AM, pfultz2 wrote:

>>>> One could do better for this low-level part with registrations such as
>>>> (without a BOOST_VMD_ prefix for brevity)
>>>>
>>>>       #define IDENTIFIER_CIRCLE (CIRCLE),
>>>>       #define IDENTIFIER_SQUARE (SQUARE),
>>>>       #define IDENTIFIER_TRIANGLE (TRIANGLE),
>>>>       #define IDENTIFIER_RECTANGLE (RECTANGLE),
>>>
>>> Is the ending comma there for a reason ?
>>
>> Yes, though I think you already figured it out.  The parentheses are
>> there so that you can detect that your prefix token-pasting did
>> something, the "v-identifier" is there so you can extract it, and the
>> comma is there to separate the "v-identifier" from the rest of a
>> "v-sequence".
>
> There is no need for the commas though. Its easy enough to insert them in.
> Since we take sequence of tokens first:
>
>      CIRCLE SQUARE
>
> And then we concat the identifiers, which gives us this:
>
>      (CIRCLE) SQUARE
>
> Then we could use this:
>
>      #define APPEND_COMMA_I(...) (__VA_ARGS__),
>      #define APPEND_COMMA(...) APPEND_COMMA_I __VA_ARGS__
>
>      APPEND_COMMA((CIRCLE) SQUARE) // Expands to (CIRCLE), SQUARE
>
> It does require a little bit more for the implementation, however, this
> helps
> to simplify the "interface", I think.

The point is, however, that you don't know that the concatenation will
produce a tuple when you call your APPEND_COMMA. If it does not and just
produces an identifier of the form IDENTIFIER_XXX then your
APPEND_COMMA(IDENTIFIER_XXX SQUARE) will end up as 'APPEND_COMMA_I
IDENTIFIER_XXX SQUARE' which you do not want.

Actually I realized that the comma is not be needed at all in Paul's
IDENTIFIER macros above. There is already code in VMD that can split a
sequence beginning with a tuple into the beginning tuple and the rest of
the sequence ( BOOST_VMD_TUPLE ). So the logic of parsing for an
identifier at the beginning of a sequence is to start by looking for a
beginning tuple in the sequence, if not found test for an identifier
through concatenation, and if the subsequent sequence after
concatenation now starts with a beginning tuple, the code can separate
the beginning tuple from the remainder of the sequence and extract the
v-identifier.

>
>> I think it is possible.  The main difficulty will be VC++ because you
>> will have to bend over backwards to get those macro-generated commas to
>> form an argument separator in a variety of situations (if I recall
>> correctly).
>
> I actually use this `MSVC_INVOKE` macro to help in those situations:
>
>      #define MSVC_INVOKE BOOST_PP_CAT(MSVC_INVOKE_,
> BOOST_PP_AUTO_REC(DETAIL_MSVC_INVOKE_P, 16))
>
>      #define DETAIL_MSVC_INVOKE_P(n) BOOST_PP_IS_NULLARY( MSVC_INVOKE_ ##
> n((),) )
>
>      #define MSVC_INVOKE_1(macro, args)  MSVC_INVOKE_I_1(macro, args)
>      #define MSVC_INVOKE_2(macro, args)  MSVC_INVOKE_I_2(macro, args)
>      ...
>      #define MSVC_INVOKE_16(macro, args) MSVC_INVOKE_I_16(macro, args)
>
>      #define MSVC_INVOKE_I_1(macro, args)  MSVC_INVOKE_X_1(macro args)
>      #define MSVC_INVOKE_I_2(macro, args)  MSVC_INVOKE_X_2(macro args)
>      ...
>      #define MSVC_INVOKE_I_16(macro, args) MSVC_INVOKE_X_16(macro args)
>
>      #define MSVC_INVOKE_X_1(x) x
>      #define MSVC_INVOKE_X_2(x) x
>      ...
>      #define MSVC_INVOKE_X_16(x) x

Sorry I can't follow this or udnerstand what you are doing. I don't know
what BOOST_PP_AUTO_REC does even when I look at its code <g>.

>
> Of course, I made it re-entrant which could slow down the preprocessor for
> MSVC, however, it helps simplify when I need to workaround MSVC. So if I
> need
> to call a macro with vardiac args, I can just call it like this:
>
>      #define HEAD(x, ...) x
>      MSVC_INVOKE(HEAD, (1,2,3,4)) // Expands to 1

Are you passing '1,2,3,4' to HEAD ?

>
> Unfortunately, there are still times where this fails as well, but most of
> the
> time this will work.
>
> Finally, when I have done this DSL like parser(like in LINQ for example), I
> would convert the string of tokens to a sequence and then process them
> afterwards using the algorithms for a sequence. So I would essentially
> transform this:
>
>      #define STRING_from (from)
>      #define STRING_where (where)
>      #define STRING_select (select)
>
>      STRING_TO_SEQ(from(x, numbers) where(x < 3) select(x * x))
>
> That then expands to:
>
>      (from)((x, numbers))(where)((x < 3))(select)((x * x))

You might find this expansion easier with VMD. With Paul's suggestion
and my realization that the final comma is not needed, I will be
reprogramming the VMD v-sequence macros so that you should be able to
take a v-sequence, like your 'string' of 'from(x, numbers) where(x < 3)
select(x * x)' and break it down into its constituent v-types without
the limitations current VMD incurs. In fact I intend to add a VMD macro,
something like BOOT_VMD_SEQUENCE(vsequence) which will return all the
v-types in some way ( probably a tuple, seq, or just variadic data, I
have yet to decide ).


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Paul Fultz II
> The point is, however, that you don't know that the concatenation will
> produce a tuple when you call your APPEND_COMMA. If it does not and just
> produces an identifier of the form IDENTIFIER_XXX then your
> APPEND_COMMA(IDENTIFIER_XXX SQUARE) will end up as 'APPEND_COMMA_I
> IDENTIFIER_XXX SQUARE' which you do not want.

Yes, of course, you will need to check that you have an identifier first.

> Actually I realized that the comma is not be needed at all in Paul's
> IDENTIFIER macros above. There is already code in VMD that can split a
> sequence beginning with a tuple into the beginning tuple and the rest of
> the sequence ( BOOST_VMD_TUPLE ). So the logic of parsing for an
> identifier at the beginning of a sequence is to start by looking for a
> beginning tuple in the sequence, if not found test for an identifier
> through concatenation, and if the subsequent sequence after
> concatenation now starts with a beginning tuple, the code can separate
> the beginning tuple from the remainder of the sequence and extract the
> v-identifier.

Yes, there is similiar logic in the `STRING_TOKEN` macro I showed, except my
code doesn't handle custom namespaces(ie `STRING_`) nor multiple namespaces.

> Sorry I can't follow this or udnerstand what you are doing. I don't know
> what BOOST_PP_AUTO_REC does even when I look at its code <g>.

Well the macro is auto reentrant. So it has a macro for each recursion level
up to 16(ie `MSVC_INVOKE_1`, `MSVC_INVOKE_2`, etc). The `BOOST_PP_AUTO_REC`
then will search for a macro that can be called(it hasn't been painted blue).
It does this by calling the `DETAIL_MSVC_INVOKE_P` predicate. Then
`DETAIL_MSVC_INVOKE_P` wil pass in nullary parenthesis into `MSVC_INVOKE`, and
if that result isn't nullary then it means that recursion level got painted
blue. So the `BOOST_PP_AUTO_REC` will do a binary search and try to find the
first predicate that is true. It will return that recursion level, which it
will concat with `MSVC_INVOKE_` so the correct macro will be called. Does that
make sense? Perhaps Paul can explain it better than me.

> You might find this expansion easier with VMD. With Paul's suggestion
> and my realization that the final comma is not needed, I will be
> reprogramming the VMD v-sequence macros so that you should be able to
> take a v-sequence, like your 'string' of 'from(x, numbers) where(x < 3)
> select(x * x)' and break it down into its constituent v-types without
> the limitations current VMD incurs. In fact I intend to add a VMD macro,
> something like BOOT_VMD_SEQUENCE(vsequence) which will return all the
> v-types in some way ( probably a tuple, seq, or just variadic data, I
> have yet to decide ).

Of course for my case, I don't need to know what category or type it is(it may
be useful for other people). So having it as a sequence like I've shown is
very useful. Also, I was thinking it might be useful to have some kind of
`BELONGS_TO` to query which namespace a token belong to.

Paul Fultz II

Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Edward Diener-3
On 9/14/2014 5:10 PM, pfultz2 wrote:

>> The point is, however, that you don't know that the concatenation will
>> produce a tuple when you call your APPEND_COMMA. If it does not and just
>> produces an identifier of the form IDENTIFIER_XXX then your
>> APPEND_COMMA(IDENTIFIER_XXX SQUARE) will end up as 'APPEND_COMMA_I
>> IDENTIFIER_XXX SQUARE' which you do not want.
>
> Yes, of course, you will need to check that you have an identifier first.
>
>> Actually I realized that the comma is not be needed at all in Paul's
>> IDENTIFIER macros above. There is already code in VMD that can split a
>> sequence beginning with a tuple into the beginning tuple and the rest of
>> the sequence ( BOOST_VMD_TUPLE ). So the logic of parsing for an
>> identifier at the beginning of a sequence is to start by looking for a
>> beginning tuple in the sequence, if not found test for an identifier
>> through concatenation, and if the subsequent sequence after
>> concatenation now starts with a beginning tuple, the code can separate
>> the beginning tuple from the remainder of the sequence and extract the
>> v-identifier.
>
> Yes, there is similiar logic in the `STRING_TOKEN` macro I showed, except my
> code doesn't handle custom namespaces(ie `STRING_`) nor multiple namespaces.

I am not sure what you mean by namespaces ? Do you mean the type of the
token parsed ?

>
>> Sorry I can't follow this or udnerstand what you are doing. I don't know
>> what BOOST_PP_AUTO_REC does even when I look at its code <g>.
>
> Well the macro is auto reentrant. So it has a macro for each recursion level
> up to 16(ie `MSVC_INVOKE_1`, `MSVC_INVOKE_2`, etc). The `BOOST_PP_AUTO_REC`
> then will search for a macro that can be called(it hasn't been painted
> blue).
> It does this by calling the `DETAIL_MSVC_INVOKE_P` predicate. Then
> `DETAIL_MSVC_INVOKE_P` wil pass in nullary parenthesis into `MSVC_INVOKE`,
> and
> if that result isn't nullary then it means that recursion level got painted
> blue. So the `BOOST_PP_AUTO_REC` will do a binary search and try to find the
> first predicate that is true. It will return that recursion level, which it
> will concat with `MSVC_INVOKE_` so the correct macro will be called. Does
> that
> make sense?

No, but that's OK.

> Perhaps Paul can explain it better than me.

The BOOST_PP_AUTO_REC macro is not documented so maybe you can
understand my being puzzled by it and what it is used for. Other
undocumented macros in Boost PP are generally easier for me to follow.

>
>> You might find this expansion easier with VMD. With Paul's suggestion
>> and my realization that the final comma is not needed, I will be
>> reprogramming the VMD v-sequence macros so that you should be able to
>> take a v-sequence, like your 'string' of 'from(x, numbers) where(x < 3)
>> select(x * x)' and break it down into its constituent v-types without
>> the limitations current VMD incurs. In fact I intend to add a VMD macro,
>> something like BOOT_VMD_SEQUENCE(vsequence) which will return all the
>> v-types in some way ( probably a tuple, seq, or just variadic data, I
>> have yet to decide ).
>
> Of course for my case, I don't need to know what category or type it is(it
> may
> be useful for other people). So having it as a sequence like I've shown is
> very useful. Also, I was thinking it might be useful to have some kind of
> `BELONGS_TO` to query which namespace a token belong to.

The BOOST_VMD_SEQUENCE macro I plan will return for each v-type found
not only its data by is v-type also. I am leaning toward returning the
data as a Boost PP seq where every element of he seq is a tuple of
(type,value) pairs. Each type will be a number identifying a distinct
v-type (number,identifier,array,list,seq,tuple). Then you could iterate
through that returned seq, probably with BOOST_PP_SEQ_FOR_EACH, and do
what you want with the type/data to create your final construct. This
will allow a v-sequence, what you call a 'string' in your example, to be
parsed for its individual pieces of VMD v-type data.

You could actually do something similar with VMD now but you would have
to try each v-type separately to see if it begins the v-sequence. But
after Paul Mensonides provided a better solution to identifying
v-identifiers and v-numbers I can improve the parsing of VMD v-sequences
greatly.

In fact with VMD a v-sequence becomes another preprocessor "data type"
similar to the high level Boost PP data types of array, list, seq, and
tuple. I should really work to produce whatever operations with a
v-sequence I can that would mimic some of those Boost PP data types. Its
challenging to think about but it will be fun work except for the usual
VC++isms.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Paul Mensonides
On 9/14/2014 3:12 PM, Edward Diener wrote:
> On 9/14/2014 5:10 PM, pfultz2 wrote:

> I am not sure what you mean by namespaces ? Do you mean the type of the
> token parsed ?

I believe he means macro prefixes such as BOOST_PP_, BOOST_VMD_, or
BOOST_VMD_NAMESPACE_.

>> Perhaps Paul can explain it better than me.
>
> The BOOST_PP_AUTO_REC macro is not documented so maybe you can
> understand my being puzzled by it and what it is used for. Other
> undocumented macros in Boost PP are generally easier for me to follow.

The BOOST_PP_AUTO_REC macro is used to find the first macro of a set of
macros which has not been painted (i.e. will expand).  It requires a set
of macros which are a power of 2 which are used in order.  For example:

#include <boost/preprocessor.hpp>
#include <boost/preprocessor/detail/is_nullary.hpp>

#define CAT_1(a, b) PRIMITIVE_CAT_1(a, b)
#define CAT_2(a, b) PRIMITIVE_CAT_2(a, b)
#define CAT_3(a, b) PRIMITIVE_CAT_3(a, b)
#define CAT_4(a, b) PRIMITIVE_CAT_4(a, b)

#define PRIMITIVE_CAT_1(a, b) a ## b
#define PRIMITIVE_CAT_2(a, b) a ## b
#define PRIMITIVE_CAT_3(a, b) a ## b
#define PRIMITIVE_CAT_4(a, b) a ## b

#define CAT \
     BOOST_PP_CAT( \
         CAT_, \
         BOOST_PP_AUTO_REC(PRIMITIVE_CAT_P, 4) \
     )
     /**/
#define PRIMITIVE_CAT \
     BOOST_PP_CAT( \
         PRIMITIVE_CAT_, \
         BOOST_PP_AUTO_REC(PRIMITIVE_CAT_P, 4) \
     ) \
     /**/
#define PRIMITIVE_CAT_P(n) \
     BOOST_PP_IS_NULLARY(PRIMITIVE_CAT_ ## n(,)()) \
     /**/

Here the CAT and PRIMITIVE_CAT macros are defined as object-like macros
which expand to the next available CAT_n and PRIMITIVE_CAT_n macros
which are found via the the BOOST_PP_AUTO_REC macro--which does a binary
search from 1 to 4 in this case finding the first index for which the
passed predicate PRIMITIVE_CAT_P expands to 1.

Essentially, it makes it look like CAT can be invoked recursively, but
CAT itself is not really even a function-like macro.  E.g.

#define A1 CAT(B, 1) // here CAT expands to CAT_2
#define B1 CAT(C, 1) // here CAT expands to CAT_3
#define C1 CAT(D, 1) // here CAT expands to CAT_4

CAT(A, 1) // here CAT expands to CAT_1

> You could actually do something similar with VMD now but you would have
> to try each v-type separately to see if it begins the v-sequence. But
> after Paul Mensonides provided a better solution to identifying
> v-identifiers and v-numbers I can improve the parsing of VMD v-sequences
> greatly.
>
> In fact with VMD a v-sequence becomes another preprocessor "data type"
> similar to the high level Boost PP data types of array, list, seq, and
> tuple. I should really work to produce whatever operations with a
> v-sequence I can that would mimic some of those Boost PP data types. Its
> challenging to think about but it will be fun work except for the usual
> VC++isms.

The alternate registration mechanism (excluding the comma in the
registration itself) that you guys realized is good and is obvious is
retrospect.  That would make the registration much cleaner.

Regards,
Paul Mensonides


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Edward Diener-3
On 9/14/2014 6:49 PM, Paul Mensonides wrote:

> On 9/14/2014 3:12 PM, Edward Diener wrote:
>> On 9/14/2014 5:10 PM, pfultz2 wrote:
>
>> I am not sure what you mean by namespaces ? Do you mean the type of the
>> token parsed ?
>
> I believe he means macro prefixes such as BOOST_PP_, BOOST_VMD_, or
> BOOST_VMD_NAMESPACE_.
>
>>> Perhaps Paul can explain it better than me.
>>
>> The BOOST_PP_AUTO_REC macro is not documented so maybe you can
>> understand my being puzzled by it and what it is used for. Other
>> undocumented macros in Boost PP are generally easier for me to follow.
>
> The BOOST_PP_AUTO_REC macro is used to find the first macro of a set of
> macros which has not been painted (i.e. will expand).  It requires a set
> of macros which are a power of 2 which are used in order.

Here is my understanding of what BOOST_PP_AUTO_REC does from your
example. It takes a function-like macro name and a parameter which is a
Boost PP number which is a power of 2, from a minimum of 2 to a maximum
of 256. It then calls the function-like macro name passing a number
starting with 1 through the number passed to it. If the function-like
macro returns 1 it stops and returns the current number else it
continues with the next number. I won't ask what BOOST_PP_AUTO_REC
returns if its function-like macro never returns 1 before it finishes
cycling through its numbers <g>.

I assume the "painted" part is because it is usually used with
BOOST_PP_NULLARY as in your example below.

> For example:
>
> #include <boost/preprocessor.hpp>
> #include <boost/preprocessor/detail/is_nullary.hpp>
>
> #define CAT_1(a, b) PRIMITIVE_CAT_1(a, b)
> #define CAT_2(a, b) PRIMITIVE_CAT_2(a, b)
> #define CAT_3(a, b) PRIMITIVE_CAT_3(a, b)
> #define CAT_4(a, b) PRIMITIVE_CAT_4(a, b)
>
> #define PRIMITIVE_CAT_1(a, b) a ## b
> #define PRIMITIVE_CAT_2(a, b) a ## b
> #define PRIMITIVE_CAT_3(a, b) a ## b
> #define PRIMITIVE_CAT_4(a, b) a ## b
>
> #define CAT \
>      BOOST_PP_CAT( \
>          CAT_, \
>          BOOST_PP_AUTO_REC(PRIMITIVE_CAT_P, 4) \
>      )
>      /**/
> #define PRIMITIVE_CAT \
>      BOOST_PP_CAT( \
>          PRIMITIVE_CAT_, \
>          BOOST_PP_AUTO_REC(PRIMITIVE_CAT_P, 4) \
>      ) \
>      /**/
> #define PRIMITIVE_CAT_P(n) \
>      BOOST_PP_IS_NULLARY(PRIMITIVE_CAT_ ## n(,)()) \
>      /**/
>
> Here the CAT and PRIMITIVE_CAT macros are defined as object-like macros
> which expand to the next available CAT_n and PRIMITIVE_CAT_n macros
> which are found via the the BOOST_PP_AUTO_REC macro--which does a binary
> search from 1 to 4 in this case finding the first index for which the
> passed predicate PRIMITIVE_CAT_P expands to 1.
>
> Essentially, it makes it look like CAT can be invoked recursively, but
> CAT itself is not really even a function-like macro.  E.g.
>
> #define A1 CAT(B, 1) // here CAT expands to CAT_2
> #define B1 CAT(C, 1) // here CAT expands to CAT_3
> #define C1 CAT(D, 1) // here CAT expands to CAT_4
>
> CAT(A, 1) // here CAT expands to CAT_1

Thanks for the explanation.

>
>> You could actually do something similar with VMD now but you would have
>> to try each v-type separately to see if it begins the v-sequence. But
>> after Paul Mensonides provided a better solution to identifying
>> v-identifiers and v-numbers I can improve the parsing of VMD v-sequences
>> greatly.
>>
>> In fact with VMD a v-sequence becomes another preprocessor "data type"
>> similar to the high level Boost PP data types of array, list, seq, and
>> tuple. I should really work to produce whatever operations with a
>> v-sequence I can that would mimic some of those Boost PP data types. Its
>> challenging to think about but it will be fun work except for the usual
>> VC++isms.
>
> The alternate registration mechanism (excluding the comma in the
> registration itself) that you guys realized is good and is obvious is
> retrospect.  That would make the registration much cleaner.

I agree. Your proposal is obvious in retrospect but it is like the
legendary story of Christopher Columbus and the egg.

I am now very glad I can make a v-sequence much easier with better
functionality. Why I didn't think of your solution to being with ... <g>.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Paul Mensonides
On 9/14/2014 5:12 PM, Edward Diener wrote:
> On 9/14/2014 6:49 PM, Paul Mensonides wrote:

>> The BOOST_PP_AUTO_REC macro is used to find the first macro of a set of
>> macros which has not been painted (i.e. will expand).  It requires a set
>> of macros which are a power of 2 which are used in order.
>
> Here is my understanding of what BOOST_PP_AUTO_REC does from your
> example. It takes a function-like macro name and a parameter which is a
> Boost PP number which is a power of 2, from a minimum of 2 to a maximum
> of 256. It then calls the function-like macro name passing a number
> starting with 1 through the number passed to it. If the function-like
> macro returns 1 it stops and returns the current number else it
> continues with the next number. I won't ask what BOOST_PP_AUTO_REC
> returns if its function-like macro never returns 1 before it finishes
> cycling through its numbers <g>.

Yes, except that it does a binary rather than linear search, so it can
search a fairly large set with minimal invocations of the predicate.

>> The alternate registration mechanism (excluding the comma in the
>> registration itself) that you guys realized is good and is obvious is
>> retrospect.  That would make the registration much cleaner.
>
> I agree. Your proposal is obvious in retrospect but it is like the
> legendary story of Christopher Columbus and the egg.
>
> I am now very glad I can make a v-sequence much easier with better
> functionality. Why I didn't think of your solution to being with ... <g>.

Sorry, I meant excluding the comma in the registration was obvious to me
in retrospect, but I didn't think of it at the time.

I think that removing the existing length (etc.) limitations on
v-sequences will make things much more usable--and make the
documentation much much simpler.

Regards,
Paul Mensonides


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Edward Diener-3
On 9/14/2014 10:34 PM, Paul Mensonides wrote:

> On 9/14/2014 5:12 PM, Edward Diener wrote:
>> On 9/14/2014 6:49 PM, Paul Mensonides wrote:
>
>>> The BOOST_PP_AUTO_REC macro is used to find the first macro of a set of
>>> macros which has not been painted (i.e. will expand).  It requires a set
>>> of macros which are a power of 2 which are used in order.
>>
>> Here is my understanding of what BOOST_PP_AUTO_REC does from your
>> example. It takes a function-like macro name and a parameter which is a
>> Boost PP number which is a power of 2, from a minimum of 2 to a maximum
>> of 256. It then calls the function-like macro name passing a number
>> starting with 1 through the number passed to it. If the function-like
>> macro returns 1 it stops and returns the current number else it
>> continues with the next number. I won't ask what BOOST_PP_AUTO_REC
>> returns if its function-like macro never returns 1 before it finishes
>> cycling through its numbers <g>.
>
> Yes, except that it does a binary rather than linear search, so it can
> search a fairly large set with minimal invocations of the predicate.

Still better. Good to know.

>
>>> The alternate registration mechanism (excluding the comma in the
>>> registration itself) that you guys realized is good and is obvious is
>>> retrospect.  That would make the registration much cleaner.
>>
>> I agree. Your proposal is obvious in retrospect but it is like the
>> legendary story of Christopher Columbus and the egg.
>>
>> I am now very glad I can make a v-sequence much easier with better
>> functionality. Why I didn't think of your solution to being with ... <g>.
>
> Sorry, I meant excluding the comma in the registration was obvious to me
> in retrospect, but I didn't think of it at the time.
>
> I think that removing the existing length (etc.) limitations on
> v-sequences will make things much more usable--and make the
> documentation much much simpler.

Absolutely. It will also make the implementation of v-sequences much
simpler. The whole business of trying to identify identifiers and
numbers in a v-sequence the way I was doing it, even though it worked,
imposed limitations which I am now glad to do without. With your
suggestion I can throw all that onerous code away. It was a burden to do
it and I am glad to be able to get rid of it for a much better and
easier implementation.




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Paul Fultz II
In reply to this post by Edward Diener-3
> I am not sure what you mean by namespaces ? Do you mean the type of the
> token parsed ?

I mean the prefix for the tokens.

> > Perhaps Paul can explain it better than me.
>
> The BOOST_PP_AUTO_REC macro is not documented so maybe you can
> understand my being puzzled by it and what it is used for. Other
> undocumented macros in Boost PP are generally easier for me to follow.

Oops! Well it is documented in Chaos. I guess its more of an implementation
detail in Boost.PP. Of course, its not at all obvious what it does from
looking at the code(even though it is quite nicely formatted).

> The BOOST_VMD_SEQUENCE macro I plan will return for each v-type found
> not only its data by is v-type also. I am leaning toward returning the
> data as a Boost PP seq where every element of he seq is a tuple of
> (type,value) pairs. Each type will be a number identifying a distinct
> v-type (number,identifier,array,list,seq,tuple).

There may be use cases for this. However, it would be nice just to map it
directly to a sequence without the v-type. So you could have a mpping to a
v-sequence which has the v-types(perhaps the tuple (type,value) pairs), but in
addition have a mapping to pp sequence which will just have the value. I can
see how there are uses for both of these.

Paul Fultz II
Reply | Threaded
Open this post in threaded view
|

Re: [vmd] Library Review

Edward Diener-3
On 9/15/2014 12:17 AM, pfultz2 wrote:
>> I am not sure what you mean by namespaces ? Do you mean the type of the
>> token parsed ?
>
> I mean the prefix for the tokens.

OK. I will choose something easy, starting with BOOST_VMD_

>
>>> Perhaps Paul can explain it better than me.
>>
>> The BOOST_PP_AUTO_REC macro is not documented so maybe you can
>> understand my being puzzled by it and what it is used for. Other
>> undocumented macros in Boost PP are generally easier for me to follow.
>
> Oops! Well it is documented in Chaos. I guess its more of an implementation
> detail in Boost.PP. Of course, its not at all obvious what it does from
> looking at the code(even though it is quite nicely formatted).

Paul Mensonides explained it in his reply.

>
>> The BOOST_VMD_SEQUENCE macro I plan will return for each v-type found
>> not only its data by is v-type also. I am leaning toward returning the
>> data as a Boost PP seq where every element of he seq is a tuple of
>> (type,value) pairs. Each type will be a number identifying a distinct
>> v-type (number,identifier,array,list,seq,tuple).
>
> There may be use cases for this. However, it would be nice just to map it
> directly to a sequence without the v-type. So you could have a mpping to a
> v-sequence which has the v-types(perhaps the tuple (type,value) pairs), but
> in
> addition have a mapping to pp sequence which will just have the value. I can
> see how there are uses for both of these.

Sure. Easy enough.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost