migrate from classic to qi

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

migrate from classic to qi

Steffen.Roeber
Hi,
I want to migrate from spirit classic to qi: I'm parsing a Vector .dbc file)
QByteArray data;
//classic
using namespace boost::spirit::classic;
auto result = parse(data.constData(), data.constData() + data.size(), str_p("BO_ "), space_p);
//qi
using namespace boost::spirit::qi;
auto result = phrase_parse(data.constData(), data.constData() + data.size(), lit("BO_ "), space);

This demonstrates a very simple case. If I measure the time of parsing about 10000 lines, then the classic case is about 2 times faster than qi. Is there anything I do wrong?
I'm using VS2013x64 and boost 1.55.

Regards


Hella Aglaia Mobile Vision GmbH
Steffen Roeber
Firmware & Tools
Treskowstr. 14, D-13089 Berlin
Amtsgericht Berlin-Charlottenburg HRB 66976 B
Geschäftsführer: Kay Talmi

Fon:  +49 30 200 04 29– 412
Fax:  +49 30 200 04 29– 109
Mail: [hidden email]
URL:
www.aglaia-gmbh.de

URL:
www.mobilevision.de

Dieses Dokument ist vertraulich zu behandeln. Die Weitergabe sowie Vervielfältigung, Verwertung und Mitteilung seines Inhalts ist nur mit unserer ausdrücklichen Genehmigung gestattet. Alle Rechte vorbehalten, insbesondere für den Fall der Schutzrechtsanmeldung.
This document has to be treated confidentially. Its contents are not to be passed on, duplicated, exploited or disclosed without our express permission. All rights reserved, especially the right to apply for protective rights.

------------------------------------------------------------------------------

_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: migrate from classic to qi

Joel de Guzman
On 8/10/15 2:58 PM, [hidden email] wrote:

> Hi,
> I want to migrate from spirit classic to qi: I'm parsing a Vector .dbc file)
> QByteArray data;
> //classic
> using namespace boost::spirit::classic;
> auto result = parse(data.constData(), data.constData() + data.size(), str_p("BO_ "),
> space_p);
> //qi
> using namespace boost::spirit::qi;
> auto result = phrase_parse(data.constData(), data.constData() + data.size(), lit("BO_ "),
> space);
>
> This demonstrates a very simple case. If I measure the time of parsing about 10000 lines,
> then the classic case is about 2 times faster than qi. Is there anything I do wrong?
> I'm using VS2013x64 and boost 1.55.

I can't be sure unbless I dig deeper and investigate... but here's a guess:
Qi compiles the parser inside phrase_parse from the lit("BO_ ") expression
template (a Boost proto expression). That may account for the overhead.

1) If you pass in a grammar or a rule (typical) Qi does not have to
compile that inside the phrase_parse --they are already parsers.
2) If you have a more complex parser (typical) with loops, recursion, etc.
the (one-time) overhead will be negligible.

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: migrate from classic to qi

Steffen.Roeber
In reply to this post by Steffen.Roeber
Ok. passing the rule directly to parse() is not so clever, but why is classic parser so much faster in this case?



Von:        [hidden email]
An:        [hidden email],
Datum:        10.08.2015 22:08
Betreff:        Spirit-general Digest, Vol 111, Issue 3




Send Spirit-general mailing list submissions to
                [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
               
https://lists.sourceforge.net/lists/listinfo/spirit-general
or, via email, send a message with subject or body 'help' to
                [hidden email]

You can reach the person managing the list at
                [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Spirit-general digest..."


Today's Topics:

  1. Re: lazy parser, the rules have inherited attributes, c++11
     (Lee Clagett)
  2. Re: lazy parser, the rules have inherited attributes, c++11 (MM)


----------------------------------------------------------------------

Message: 1
Date: Mon, 10 Aug 2015 15:33:50 -0400
From: Lee Clagett <[hidden email]>
Subject: Re: [Spirit-general] lazy parser, the rules have inherited
                attributes, c++11
To: Spirit General Mailing List <[hidden email]>
Message-ID:
                <CALD9dK3bU5Hx_+Fj=[hidden email]>
Content-Type: text/plain; charset="utf-8"

On Mon, Aug 10, 2015 at 2:36 PM, MM <[hidden email]> wrote:

> On 10 August 2015 at 02:11, TONGARI J <[hidden email]> wrote:
>
>> 2015-08-10 3:12 GMT+08:00 MM <[hidden email]>:
>>
>>>
>>>
>>> On 25 July 2015 at 05:40, Joel de Guzman <[hidden email]> wrote:
>>>
>>>> On 7/25/15 8:30 AM, MM wrote:
>>>> >
>>>> >
>>>> > On 25 July 2015 at 00:44, Joel de Guzman <[hidden email] <mailto:
>>>> [hidden email]>> wrote:
>>>> >
>>>> >     On 7/25/15 5:38 AM, MM wrote:
>>>> >     > Where I choose the parser at parse time, by having a base
>>>> pointer that set at parse time
>>>> >     > to point to the right rule to use, the call
>>>> >     >   lazy(*_a)
>>>> >     >
>>>> >     > from the nabialek trick example works when the rule pointed to
>>>> by _a does not have any
>>>> >     > inherited attributes.
>>>> >     >
>>>> >     > How does lazy(*_a) change then?
>>>> >     >
>>>> >     > I have tried the 2nd part of the post here
>>>> >     >
>>>>
http://stackoverflow.com/questions/17717590/boost-spirit-qi-how-to-return-attributes-with-nabialek-trick
>>>> >     >
>>>> >     > without success.
>>>> >     >
>>>> >     > What does lazy() expect?   An actor?
>>>> >
>>>> >     lazy expects a function object. it has similar arguments as
>>>> actions, so
>>>> >     you can get the _val and all the stuff just like actions. unlike
>>>> actions
>>>> >     though, it is executed prior to parse.
>>>> >
>>>> >     Regards,
>>>> >     --
>>>> >     Joel de Guzman
>>>> >
>>>> > Thanks,
>>>> >
>>>> > but if in the above
>>>> >    ... >>  lazy(*_a)  ;
>>>> > _a pointer to a rule with no inherited attributes, it works.
>>>> >
>>>> > but if _a points to a rule with 1 attribute (for e.g _r1), how do you
>>>> call that?
>>>> >
>>>> >    ... >>  lazy(*_a) (_r1)
>>>> >
>>>> > or
>>>> >
>>>> >    ... >> lazy( (*_a) (_r1) )
>>>> >
>>>> > or use a phoenix function ?
>>>>
>>>> I think it should be the latter. Keep in mind though that _r1 in this
>>>> case
>>>> is the inherited attribute of the rule at the call site (not the *_a).
>>>> If
>>>> the inherited attribute is an int, for example, you can write:
>>>>
>>>>     lazy( (*_a)(1234) )
>>>>
>>>>
>>> I haven't made progress  here, so I am restarting from scratch:
>>> The following fails to compile with g++ 5.1.1 boost 1.57. My grammar is
>>> wrong somewhere but I don't know where. The compile error refers to the
>>> "reference" header files so I guess it's the &r1 bit that doesn't work...
>>> Once the following is sorted, I will have a followup question.
>>>
>>> #include <boost/spirit/include/support_utree.hpp>
>>> #include <boost/spirit/include/qi.hpp>
>>> #include <boost/spirit/include/phoenix_core.hpp>
>>> #include <boost/spirit/include/phoenix_operator.hpp>
>>> #include <boost/fusion/include/adapt_struct.hpp>
>>> #include <boost/assert.hpp>
>>> #include <iostream>
>>> #include <string>
>>> #include <cstdlib>
>>>
>>> template <typename P, typename T>
>>> void test_phrase_parser_attr(
>>>     char const* input, P const& p, T& attr, bool full_match = true)
>>> {
>>>     using boost::spirit::qi::phrase_parse;
>>>     using boost::spirit::qi::ascii::blank;
>>>
>>>     char const* f(input);
>>>     char const* l(f + strlen(f));
>>>     if (phrase_parse(f, l, p, blank, attr) && (!full_match || (f == l)))
>>>         std::cout << "ok" << std::endl;
>>>     else
>>>         std::cout << "fail" << std::endl;
>>> }
>>>
>>> int main()
>>> {
>>>   namespace qis = boost::spirit::qi;
>>>   using qis::lazy; using qis::_val; using qis::lit;
>>>   using qis::_a;
>>>   using boost::phoenix::val;
>>>
>>>   using Rule =
>>>     qis::rule<std::string::const_iterator, std::string(),
>>> qis::blank_type>;
>>>
>>>   Rule r1 = lit("X1") [ _val = "X1" ];
>>>   Rule r2 = lit("X2") [ _val = "X2" ];
>>>   Rule* rptr;
>>>
>>>   qis::rule<std::string::const_iterator, std::string(),
>>>     qis::locals<Rule*>,  qis::blank_type> dispatcher;
>>>   dispatcher  %=
>>>     ( qis::lit("Test1") [ _a = &r1 ] | qis::lit("Test2") [ _a = &r2 ]) >>
>>>     lazy(*_a);
>>>
>>>   std::string attr;
>>>   test_phrase_parser_attr("Test1 X1", dispatcher, attr );
>>>   return 0;
>>> }
>>>
>>
>> I didn't compile your code. By skimming over it, I think you need
>> addressof instead of & for rules at least.
>>
>>
>> ------------------------------------------------------------------------------
>>
>
> Sorry, what is this addressof... I couldn't find it in the spirit QI
> index. Btw, this is not x3.
>
> Rds,
>
> MM
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Spirit-general mailing list
> [hidden email]
>
https://lists.sourceforge.net/lists/listinfo/spirit-general
>
>
>From cppreference[1]:

Obtains the actual address of the object or function arg, even in presence
of overloaded operator&
Boost [2] also has a version if you are using a c++11 only compiler.

[1]
http://en.cppreference.com/w/cpp/memory/addressof
[2]
http://www.boost.org/doc/libs/1_58_0/libs/core/doc/html/core/addressof.html

Lee
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 2
Date: Mon, 10 Aug 2015 21:08:23 +0100
From: MM <[hidden email]>
Subject: Re: [Spirit-general] lazy parser, the rules have inherited
                attributes, c++11
To: Spirit General Mailing List <[hidden email]>
Message-ID:
                <CADsB1iDxHfzJecFurm9QxJzBfCMhW=[hidden email]>
Content-Type: text/plain; charset="utf-8"

On 10 August 2015 at 20:33, Lee Clagett <[hidden email]> wrote:

>
>
> On Mon, Aug 10, 2015 at 2:36 PM, MM <[hidden email]> wrote:
>
>> On 10 August 2015 at 02:11, TONGARI J <[hidden email]> wrote:
>>
>>> 2015-08-10 3:12 GMT+08:00 MM <[hidden email]>:
>>>
>>>>
>>>>
>>>> On 25 July 2015 at 05:40, Joel de Guzman <[hidden email]> wrote:
>>>>
>>>>> On 7/25/15 8:30 AM, MM wrote:
>>>>> >
>>>>> >
>>>>> > On 25 July 2015 at 00:44, Joel de Guzman <[hidden email] <mailto:
>>>>> [hidden email]>> wrote:
>>>>> >
>>>>> >     On 7/25/15 5:38 AM, MM wrote:
>>>>> >     > Where I choose the parser at parse time, by having a base
>>>>> pointer that set at parse time
>>>>> >     > to point to the right rule to use, the call
>>>>> >     >   lazy(*_a)
>>>>> >     >
>>>>> >     > from the nabialek trick example works when the rule pointed to
>>>>> by _a does not have any
>>>>> >     > inherited attributes.
>>>>> >     >
>>>>> >     > How does lazy(*_a) change then?
>>>>> >     >
>>>>> >     > I have tried the 2nd part of the post here
>>>>> >     >
>>>>>
http://stackoverflow.com/questions/17717590/boost-spirit-qi-how-to-return-attributes-with-nabialek-trick
>>>>> >     >
>>>>> >     > without success.
>>>>> >     >
>>>>> >     > What does lazy() expect?   An actor?
>>>>> >
>>>>> >     lazy expects a function object. it has similar arguments as
>>>>> actions, so
>>>>> >     you can get the _val and all the stuff just like actions. unlike
>>>>> actions
>>>>> >     though, it is executed prior to parse.
>>>>> >
>>>>> >     Regards,
>>>>> >     --
>>>>> >     Joel de Guzman
>>>>> >
>>>>> > Thanks,
>>>>> >
>>>>> > but if in the above
>>>>> >    ... >>  lazy(*_a)  ;
>>>>> > _a pointer to a rule with no inherited attributes, it works.
>>>>> >
>>>>> > but if _a points to a rule with 1 attribute (for e.g _r1), how do
>>>>> you call that?
>>>>> >
>>>>> >    ... >>  lazy(*_a) (_r1)
>>>>> >
>>>>> > or
>>>>> >
>>>>> >    ... >> lazy( (*_a) (_r1) )
>>>>> >
>>>>> > or use a phoenix function ?
>>>>>
>>>>> I think it should be the latter. Keep in mind though that _r1 in this
>>>>> case
>>>>> is the inherited attribute of the rule at the call site (not the *_a).
>>>>> If
>>>>> the inherited attribute is an int, for example, you can write:
>>>>>
>>>>>     lazy( (*_a)(1234) )
>>>>>
>>>>>
>>>> I haven't made progress  here, so I am restarting from scratch:
>>>> The following fails to compile with g++ 5.1.1 boost 1.57. My grammar is
>>>> wrong somewhere but I don't know where. The compile error refers to the
>>>> "reference" header files so I guess it's the &r1 bit that doesn't work...
>>>> Once the following is sorted, I will have a followup question.
>>>>
>>>> #include <boost/spirit/include/support_utree.hpp>
>>>> #include <boost/spirit/include/qi.hpp>
>>>> #include <boost/spirit/include/phoenix_core.hpp>
>>>> #include <boost/spirit/include/phoenix_operator.hpp>
>>>> #include <boost/fusion/include/adapt_struct.hpp>
>>>> #include <boost/assert.hpp>
>>>> #include <iostream>
>>>> #include <string>
>>>> #include <cstdlib>
>>>>
>>>> template <typename P, typename T>
>>>> void test_phrase_parser_attr(
>>>>     char const* input, P const& p, T& attr, bool full_match = true)
>>>> {
>>>>     using boost::spirit::qi::phrase_parse;
>>>>     using boost::spirit::qi::ascii::blank;
>>>>
>>>>     char const* f(input);
>>>>     char const* l(f + strlen(f));
>>>>     if (phrase_parse(f, l, p, blank, attr) && (!full_match || (f == l)))
>>>>         std::cout << "ok" << std::endl;
>>>>     else
>>>>         std::cout << "fail" << std::endl;
>>>> }
>>>>
>>>> int main()
>>>> {
>>>>   namespace qis = boost::spirit::qi;
>>>>   using qis::lazy; using qis::_val; using qis::lit;
>>>>   using qis::_a;
>>>>   using boost::phoenix::val;
>>>>
>>>>   using Rule =
>>>>     qis::rule<std::string::const_iterator, std::string(),
>>>> qis::blank_type>;
>>>>
>>>>   Rule r1 = lit("X1") [ _val = "X1" ];
>>>>   Rule r2 = lit("X2") [ _val = "X2" ];
>>>>   Rule* rptr;
>>>>
>>>>   qis::rule<std::string::const_iterator, std::string(),
>>>>     qis::locals<Rule*>,  qis::blank_type> dispatcher;
>>>>   dispatcher  %=
>>>>     ( qis::lit("Test1") [ _a = &r1 ] | qis::lit("Test2") [ _a = &r2 ])
>>>> >>
>>>>     lazy(*_a);
>>>>
>>>>   std::string attr;
>>>>   test_phrase_parser_attr("Test1 X1", dispatcher, attr );
>>>>   return 0;
>>>> }
>>>>
>>>
>>> I didn't compile your code. By skimming over it, I think you need
>>> addressof instead of & for rules at least.
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>
>> Sorry, what is this addressof... I couldn't find it in the spirit QI
>> index. Btw, this is not x3.
>>
>> Rds,
>>
>> MM
>>
>>
>> ------------------------------------------------------------------------------
>>
>>
> From cppreference[1]:
>
> Obtains the actual address of the object or function arg, even in
> presence of overloaded operator&
> Boost [2] also has a version if you are using a c++11 only compiler.
>
> [1]
http://en.cppreference.com/w/cpp/memory/addressof
> [2]
>
http://www.boost.org/doc/libs/1_58_0/libs/core/doc/html/core/addressof.html
>
> Lee
>

Fantastic, that was the problem. Below is the code that compiles and runs
(I corrected the rule's iterator and the blank type as well):

#include <boost/spirit/include/support_utree.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/assert.hpp>
#include <iostream>
#include <string>
#include <cstdlib>

template <typename P, typename T>
void test_phrase_parser_attr(
   char const* input, P const& p, T& attr, bool full_match = true)
{
   using boost::spirit::qi::phrase_parse;
   using boost::spirit::qi::standard::blank;

   char const* f(input);
   char const* l(f + strlen(f));
   if (phrase_parse(f, l, p, blank, attr) && (!full_match || (f == l)))
       std::cout << "ok" << std::endl;
   else
       std::cout << "fail" << std::endl;
}

int main()
{
 namespace qis = boost::spirit::qi;
 using qis::lazy; using qis::_val; using qis::lit;
 using qis::_a;
 using boost::phoenix::val;

 using Rule =
   qis::rule<const char*, std::string(), qis::blank_type>;

 Rule r1 = lit("X1") [ _val = "X1" ];
 Rule r2 = lit("X2") [ _val = "X2" ];
 Rule* rptr;

 qis::rule<const char*, std::string(), qis::locals<Rule*>,
qis::blank_type>
   dispatcher;
 dispatcher  %=
   ( qis::lit("Test1") [ _a = std::addressof(r1) ] | qis::lit("Test2") [
_a = std::addressof(r2) ]) >>
   lazy(*_a);

 std::string attr;
 test_phrase_parser_attr("Test2 X2", dispatcher, attr );
 std::cout<< "value="<< attr <<std::endl;

 return 0;
}

Now, how can I make r1 and r2 take an inherited attribute of type int, and
lazily pass the int ?
Let's say I add a local variable _b of type int, which I populate with
either 1 or 2 depending on whether I parsed Test1 or Test2, and pass that 1
or 2 to whatever rule is pointed to by _a ?

Thanks

MM
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

------------------------------------------------------------------------------


------------------------------

_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general


End of Spirit-general Digest, Vol 111, Issue 3
**********************************************


------------------------------------------------------------------------------

_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: migrate from classic to qi

Joel de Guzman
On 8/11/15 1:39 PM, [hidden email] wrote:
> Ok. passing the rule directly to parse() is not so clever, but why is classic parser so
> much faster in this case?

I can't conclude that it is faster in this case without further tests.
I can assure you though that Qi is a lot faster than classic in the
normal case, especially with ASTs involved.

<BTW, Please avoid top posting>

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general