Spirit typeof tests

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Spirit typeof tests

Arkadiy Vertleyb
The Spirit typeof tests seem to just include the registration files.  Is
more comprehensive testing being planned?

Regards,
Arkadiy



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Spirit typeof tests

Tobias Schwinger
Arkadiy Vertleyb wrote:
> The Spirit typeof tests seem to just include the registration files.  
> Is more comprehensive testing being planned?

<cite -- email correspondence on testing typeof registration>

Joel de Guzman wrote:

> Tobias Schwinger wrote:
>
>> Joel de Guzman wrote:
>>
>>> Tobias Schwinger wrote:
>>>
>>>> Is there a way to tweak the existing test suite to apply Typeof to
>>>> the types involved?
>>>
>>> Hmmm...I'm not sure if that's a good idea --but I'm not sure.
>>> I'm afraid of entanglement. Wouldn't it be better to have
>>> separate tests?
>>
>> I just added simple separate tests that just include Spirit plus typeof
>
> cool!
>
>> headers to ensure none of the forward declarations (those in the
>> typeof.hpp files which are not tied to the component header) have
>> been broken. These tests also catch typos in the registration.
>> If we want to find forgotten registrations, however, the tests must
>> cover all there is in Spirit -- but we can also wait for someone to
>> complain, I figure (and -on second thought- I think I like it).
>
> I think this is the right way to go. I'm not quite fond of the
> carpet bombing approach to testing anyway. If we add typeof tests
> to all the tests, then that would amount to 2X testing time (more
> considering the compile time of Boost.Typeof is relatively not
> insignificant). A more focused set of tests is my preference. In
> fact, when we have Spirit-2, I'd re-focus those tests we already
> have there. It's already taking up too much testing resources as
> it was.

</cite>

The infrastructure beyond registration (to actually utilizy typeof support for building grammars)

    boost/spirit/utility/rule_parser.hpp
    libs/spirit/example/techniques/no_rules_with_typeof/*

is still in experimental state; there is no documentation except a preliminary manual in the header, there are no tests except the examples and right now even these few require a *very* decent compiler to work (GCC 3.4 emulated/native Typeof and VC8 with some issues left work. VC7.1 seems to randomly ICE inside Typeof in some -sometimes surprisingly simple- cases, so if you're looking for a new challenge, please go ahead ;-) ).


Regards,

Tobias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Spirit typeof tests

Arkadiy Vertleyb
"Tobias Schwinger" <[hidden email]> wrote

> there are no tests except the examples and right now even these few
require a *very*
> decent compiler to work (GCC 3.4 emulated/native Typeof and VC8 with some
issues left
> work. VC7.1 seems to randomly ICE inside Typeof in some -sometimes
surprisingly simple-
> cases

I would like to take a look at them (not that I am expecting to be able to
fix ICEs, though).  Are these ICEs related to types used or to any
particular context the typeof is used from?

Regards,
Arkadiy



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Spirit typeof tests

Tobias Schwinger
Arkadiy Vertleyb wrote:
>
> I would like to take a look at them (not that I am expecting to be able to
> fix ICEs, though).
>
> Are these ICEs related to types used or to any
> particular context the typeof is used from?
>

The opaque_rule_parser example (only uses non-templated typeof) works.

The rule_parser_1_1 example makes VC7.1 ICE in the first, very simple rule (it uses non-templated typeof).

The rule_parser_1_2 example, which uses some really tricky stuff (and both templated and non-templated typeof), works fine.

The rule_parser_2_1 example works too (it requires native mode to compile because LIMIT_SIZE == 100 is beyond the limit).

The rule_parser_2_2 fails in the third rule (maybe because of the self-reference or maybe just for the same obscure reasons rule_parser_1_1 does).

For the examples that ICE it does not matter whether BOOST_TYPEOF_EMULATION is defined or not.


Regards,

Tobias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Spirit typeof tests

Peder Holt
On 2/22/06, Tobias Schwinger <[hidden email]> wrote:

> Arkadiy Vertleyb wrote:
> >
> > I would like to take a look at them (not that I am expecting to be able to
> > fix ICEs, though).
> >
> > Are these ICEs related to types used or to any
> > particular context the typeof is used from?
> >
>
> The opaque_rule_parser example (only uses non-templated typeof) works.
>
> The rule_parser_1_1 example makes VC7.1 ICE in the first, very simple rule (it uses non-templated typeof).

This is not a problem in BOOST_TYPEOF.
The following code replicates the ICE under VC7.1:

#include <boost/spirit/core.hpp>
#include <boost/spirit/utility/confix.hpp>

using namespace boost::spirit;
template<typename T>
void test(const T&) {}

int main()
{
  test(
      confix_p("//",*anychar_p,eol_p) |
      confix_p("/*",*anychar_p,"*/") |
      space_p
  );
  return 0;
}

Regards,
Peder

>
> The rule_parser_1_2 example, which uses some really tricky stuff (and both templated and non-templated typeof), works fine.
>
> The rule_parser_2_1 example works too (it requires native mode to compile because LIMIT_SIZE == 100 is beyond the limit).
>
> The rule_parser_2_2 fails in the third rule (maybe because of the self-reference or maybe just for the same obscure reasons rule_parser_1_1 does).
>
> For the examples that ICE it does not matter whether BOOST_TYPEOF_EMULATION is defined or not.
>
>
> Regards,
>
> Tobias
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

[bug][vc7.1] confix_p and literals

Tobias Schwinger
Peder Holt wrote:
>
> This is not a problem in BOOST_TYPEOF.

Well, I never said it was -- it's a VC7.1 problem of course ;-).

It's a confix_p with VC7.1 -- that's why I cross post to spirit.devel.
If it's unfixable it would probably be a good idea to have a portability note in the docuementation.

> The following code replicates the ICE under VC7.1:
>

Minimized further:

> #include <boost/spirit/core.hpp>
> #include <boost/spirit/utility/confix.hpp>
>
> using namespace boost::spirit;
> template<typename T>
> void test(const T&) {}
>
> int main()
> {
>   test(
>       confix_p("//",*anychar_p,eol_p)
<snip>
>   );
>   return 0;
> }

Using

    confix_p(str_p("//"),*anychar_p,eol_p)

melts the ICE.


Regards,

Tobias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [bug][vc7.1] confix_p and literals

Joel de Guzman-2
Tobias Schwinger wrote:
> Peder Holt wrote:
>
>>This is not a problem in BOOST_TYPEOF.
>
>
> Well, I never said it was -- it's a VC7.1 problem of course ;-).
>
> It's a confix_p with VC7.1 -- that's why I cross post to spirit.devel.
> If it's unfixable it would probably be a good idea to have a portability note in the docuementation.

Yes we are aware of this problem. With VC7.1, I tend to avoid the
confix stuff. The problem is somewhere in the complexity of the
meta facilities of Spirit. Hopefully, this will go away with
Spirit-2.

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

VC7.1 ICE

Tobias Schwinger
Joel de Guzman wrote:
> Tobias Schwinger wrote:
>>It's confix_p with VC7.1 -- that's why I cross post to spirit.devel.
>>If it's unfixable it would probably be a good idea to have a portability note in the docuementation.
>
> Yes we are aware of this problem. With VC7.1, I tend to avoid the
> confix stuff. The problem is somewhere in the complexity of the
> meta facilities of Spirit. Hopefully, this will go away with
> Spirit-2.
>

Maybe having an vague idea what causes the problem can help. Does anyone know more about it?

Here's my attempt of reconstructing another instance that occurs when

    int_p[& do_int]

is used in the wrong context (a case without confix stuff and hopefully less "meta depth").
 
    // define BOOM e [1;3] to make VC1 ICE

    #include <cstddef>
    #include <iostream>
    #include <boost/spirit/core.hpp>

    void do_int(int) { } // using a functor instead works

    template<std::size_t> struct take_size_t { typedef void type; };

    using namespace boost::spirit;

    #if ! defined BOOM
    template<typename T> struct wrap
    {
      // OK
      static const std::size_t value = sizeof(int_p[& do_int]);
      // wrapping the expression (see BOOM = { 2,3 }) also works

      typedef typename take_size_t<value>::type type;
    };
    #elif BOOM == 1
    template<typename T> struct wrap
    {
      // overloaded function: illegal sizeof operand
      // note: the function is *not* overloaded
      typedef take_size_t< sizeof( int_p[& do_int] ) >::type type;
    };
    #elif BOOM == 2
    template<typename U>
    U const & const_ref_bind(U const & u) { return u; }

    template<typename T> struct wrap
    {
      // ICE
      typedef take_size_t<
        sizeof(const_ref_bind(int_p[& do_int])) >::type
      type;
    };
    #else // BOOM == 3
    template<typename T, typename U>
    U const & depend_on_type(U const & u) { return u; }

    template<typename T> struct wrap
    {
      // ICE
      typedef typename take_size_t<
        sizeof(depend_on_type<T>(int_p[& do_int])) >::type
      type;
    };
    #endif


Thanks,

Tobias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: VC7.1 ICE

Peder Holt
In confix_parser_gen:
Adding another level of indirection solves the problem:

template<typename StartT, typename ExprT, typename EndT>
confix_parser<
            typename as_parser<StartT>::type, <- VC7.1 ICEs on these
            typename as_parser<ExprT>::type,
            typename as_parser<EndT>::type,
            typename as_parser<ExprT>::type::parser_category_t,
            NestedT,
            LexemeT
        >
    operator()(
        StartT const &start_, ExprT const &expr_, EndT const &end_) const

Filtering the confix_parser into type-generating template,
result_type, solves this ICE.

    template<typename StartT, typename ExprT, typename EndT>
    struct result_type {
        typedef confix_parser<
            typename as_parser<StartT>::type,
            typename as_parser<ExprT>::type,
            typename as_parser<EndT>::type,
            typename as_parser<ExprT>::type::parser_category_t,
            NestedT,
            LexemeT
        > type;
    };

    template<typename StartT, typename ExprT, typename EndT>
    typename result_type<StartT,ExprT,EndT>::type
    operator()(
        StartT const &start_, ExprT const &expr_, EndT const &end_) const

Now rule_parser_1_1 passes without problems.
Attached is a patch with this fix.

Regards,
Peder

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

confix.hpp.patch (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: VC7.1 ICE

Peder Holt
On 2/26/06, Tobias Schwinger <[hidden email]> wrote:

> Peder Holt wrote:
> > In confix_parser_gen:
> > Adding another level of indirection solves the problem:
> > [...]
> >
> > Now rule_parser_1_1 passes without problems.
>
> Great!
>
>
> > Attached is a patch with this fix.
> >
>
> I applied the workaround to all functions in the CVS version of confix.hpp.
> Unfortunately the same trick won't help to get rid of the ICE in rule_parser_2_2 when applied to the subscript operator of spirit::parser (would've been too easy)...

I have looked at this, and this is a problem between VC7.1 and the way
the typeof emulation works, as you showed in the simplified test-case.
The problem lies in the use of the operator[] with function pointers.

The only workaround for this problem that I have found, apart from
rewriting the user code, is to separate the sizeof operation and the
typedef.
(like in the #if ! defined BOOM clause)
Problem is, this has to be done inside a struct, and can not be made
to emulate the BOOST_TYPEOF syntax.
The proposed solution is then to add a new macro called something like:
BOOST_TYPEOF_NESTED_TYPEDEF(name,expr) which looks roughly like this:
struct name {
   //Twophase typeof using expr
   typedef resulting_type_calculated type;
};

To access the type, you have to do a typedef typename name::type
my_type; outside BOOST_TYPEOF_NESTED_TYPEDEF.

Introducing this construct has two positive effects:
1. rule_parser_2_2 (and similar tests) pass.
2. In emulation mode, there is no longer any limit to the complexity
of the types passed to BOOST_TYPEOF_NESTED_TYPEDEF.
BOOST_TYPEOF_LIMIT_SIZE is no longer a limiting factor.

The downside is the added complexity of the library. It also doesn't
reflect any proposed C++ features.

Is there any interest in adding this feature to the typeof library?

Regards,
Peder

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: VC7.1 ICE

Tobias Schwinger
In reply to this post by Peder Holt
Peder Holt wrote:

> Tobias Schwinger wrote:
>>Unfortunately the same trick won't help to get rid of the ICE in
>>rule_parser_2_2 when applied to the subscript operator of
>>spirit::parser (would've been too easy)...
>
>
> I have looked at this, and this is a problem between VC7.1 and the way
> the typeof emulation works, as you showed in the simplified test-case.
> The problem lies in the use of the operator[] with function pointers.
>
> The only workaround for this problem that I have found, apart from
> rewriting the user code, is to separate the sizeof operation and the
> typedef.
> (like in the #if ! defined BOOM clause)
> Problem is, this has to be done inside a struct, and can not be made
> to emulate the BOOST_TYPEOF syntax.
> The proposed solution is then to add a new macro called something like:
> BOOST_TYPEOF_NESTED_TYPEDEF(name,expr) which looks roughly like this:
> struct name {
>    //Twophase typeof using expr
>    typedef resulting_type_calculated type;
> };
>
> To access the type, you have to do a typedef typename name::type
> my_type; outside BOOST_TYPEOF_NESTED_TYPEDEF.
>
> Introducing this construct has two positive effects:
> 1. rule_parser_2_2 (and similar tests) pass.
> 2. In emulation mode, there is no longer any limit to the complexity
> of the types passed to BOOST_TYPEOF_NESTED_TYPEDEF.
> BOOST_TYPEOF_LIMIT_SIZE is no longer a limiting factor.

Wow! Sounds like a pretty cool workaround to me.

Will the proposed interface allow me to inject static variables into the context the expression is evaluated in?

Clarification:

Using a SPIRIT_RULE_PARSER with the expression

   p >> *(d >> p)

, p and d being templated parameters, the macro generates

   // ...
   struct __parser
   {
       // Injected names
       // (these are member variables in the "runtime version" of the
       // expression which is assigned to a member variable of type
       // __parser::type)
       static P0 & p;        
       static P1 & d;

       typedef typeof(  p >> *(d >> p)  ) type;
   };
   // ...

Wouldn't I have to declare p and d inside the nested struct in this case?

> The downside is the added complexity of the library.

> It also doesn't reflect any proposed C++ features.

So what? It's closer to the semantics of native typeof because the limits aren't so tight and because it's stable.

It would be interesting to know whether it also eliminates the ADL failure and the weird "deduction from base fails for non-overloaded functions" bug in VC8 [ see http://tinyurl.com/7j9me ] ...


> Is there any interest in adding this feature to the typeof library?

YES!

It also seems very well-suited for planned Spirit-2

   http://spirit.sourceforge.net/dl_docs/redesigning_spirit.html

as we'll easily hit the current limits trying to handle the results of the proposed kind of parser-transformation, I figure.
In fact, I was tempted to make "subexpression registration" [ discussed with Arakdiy here: http://tinyurl.com/asnxq ], which can significantly reduce the required TYPEOF_LIMIT_SIZE, an official feature request for Typeof after reading that document.


Thanks again -you da man- ;-),

Tobias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: VC7.1 ICE

Arkadiy Vertleyb
In reply to this post by Peder Holt
"Peder Holt" <[hidden email]> wrote

> 2. In emulation mode, there is no longer any limit to the complexity
> of the types passed to BOOST_TYPEOF_NESTED_TYPEDEF.
> BOOST_TYPEOF_LIMIT_SIZE is no longer a limiting factor.

Interesting.

I think I can see how you avoid the limit related to the number of template
parameters (which is a major improvement by itself).

However, I don't think you can get over the preprocessor limit, can you?

Also, BOOST_TYPEOF_LIMIT_SIZE is there to avoid unnecessary evaluations of
the expression, and again, I don't see how you are planning to not depend on
it...  Assumming, of course, that I figured out things correctly, and you
are planning to do something like this:

struct name
{
    enum {
        value1 = sizeof(encode(expr).item1),
        value2 = sizeof(encode(expr).item2),
        ...
        valueN = sizeof(encode(expr).itemN),
    };
    typedef<decode<some_iterator<name> >::type type;
};

encode still needs to be called BOOST_TYPEOF_LIMIT_SIZE times, doesn't it?
Am I missing something?

Regards,
Arkadiy



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: VC7.1 ICE

Peder Holt
On 3/1/06, Arkadiy Vertleyb <[hidden email]> wrote:

> "Peder Holt" <[hidden email]> wrote
>
> > 2. In emulation mode, there is no longer any limit to the complexity
> > of the types passed to BOOST_TYPEOF_NESTED_TYPEDEF.
> > BOOST_TYPEOF_LIMIT_SIZE is no longer a limiting factor.
>
> Interesting.
>
> I think I can see how you avoid the limit related to the number of template
> parameters (which is a major improvement by itself).
>
> However, I don't think you can get over the preprocessor limit, can you?
>
> Also, BOOST_TYPEOF_LIMIT_SIZE is there to avoid unnecessary evaluations of
> the expression, and again, I don't see how you are planning to not depend on
> it...  Assumming, of course, that I figured out things correctly, and you
> are planning to do something like this:
>
> struct name
> {
>     enum {
>         value1 = sizeof(encode(expr).item1),
>         value2 = sizeof(encode(expr).item2),
>         ...
>         valueN = sizeof(encode(expr).itemN),
>     };
>     typedef<decode<some_iterator<name> >::type type;
> };
>
> encode still needs to be called BOOST_TYPEOF_LIMIT_SIZE times, doesn't it?
> Am I missing something?

Yep.
What I do is basically the following:
struct name {
   template<int Iteration>
   struct encode_part {
      static int Offset=Iteration*BOOST_TYPEOF_NESTED_SIZE (currently 50)
      typedef offset_vector<nested_vector0<>,boost::mpl::size_t<Offset>
> start_vector;
      template<typename T>
      static sizer<encode_type<start,vector,T> > encode(const T& arg);
      static int value0=sizeof(encode(expr).item0);
      typedef boost::mpl::size_t<value0> item0;
      ...
   };
   template<typename Pos>
   struct decode_iter {
      static const int pos=Pos::value;
      static const int where=pos%BOOST_TYPEOF_NESTED_SIZE;
      static const int iteration=pos/BOOST_TYPEOF_NESTED_SIZE;
      typedef typename
v_iter<encode_part<iteration>,mpl::size_t<where> >::type type;
      typedef decode_iter<Pos::next> next;
   };
   typedef typename decode_type<decode_iter<mpl::size_t<0> > >::type type;
};

What this means, is that a new encode_vector of size
BOOST_TYPEOF_NESTED_SIZE will be calculated if the expresion is
greater than BOOST_TYPEOF_NESTED_SIZE.

E.g. if the expression needs 151 integers to be encoded, 4 separate
encoding vectors will be used to represent the expression.

push_back for offset_vector decrements the offset, until the offset is
0, then it calls push_back on the nested_vector.

push_back on a nested_vector of size BOOST_TYPEOF_NESTED_SIZE returns
the input vector.

Regards,
Peder.

>
> Regards,
> Arkadiy
>
>
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost