Design/structure X3 parser more like Qi parser

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
On 12/20/2016 03:02 AM, Sandro Pirkwieser wrote:
Hi Sandro.
 > Hi Larry,
 >
 >> This doesn't use BOOST_SPIRIT_DEFINE, and I don't think it can because
 >> that would generate one or more parse_rule's template functions within
 >> the make_csv_parser function.  Is that right?
 >> If so, then wouldn't that make this method slower to compile than if
 >> BOOST_SPIRIT_DEFINE could be used?
 >
 > yes, to my knowledge using BOOST_SPIRIT_DEFINE inside a function,
e.g. make_csv_parser, is not possible.
 > But I don't know whether this influences compile time.

I've not measured the compile time; however, I assume it
would take more time because of the template metaprogramming
required at these places:

   * parse_rhs_main:

https://github.com/boostorg/spirit/blob/master/include/boost/spirit/home/x3/nonterminal/detail/rule.hpp#L184

     where a new context is created.

   * default parse_rule:

https://github.com/boostorg/spirit/blob/master/include/boost/spirit/home/x3/nonterminal/rule.hpp#L37 


     where the rule_definition is looked up.

However, I realize that's rather circumstantial evidence and
maybe an actual benchmark is needed to show *if* there's a
significant compile-time cost.

 >
 >> In contrast, the following:
 >>
https://github.com/cppljevans/spirit/blob/ExagonLinkingError/workbench/x3/rule_defns/parse_rule_crtp.cpp
 >> shows a *prototype* of how to emulate the qi method in a revised x3
 >> *and* use something like BOOST_SPIRIT_DEFINE.
[snip]
 >> Hence, Sandro, could you provide some use-case for this feature?
 >

 > The main topic (aside the more recent performance
 > analysis) was the question to "better" (at least in my
 > view) encapsulate an X3 parser than with namespaces,
 > i.e. more in the style of a Qi parser. There a larger,
 > more complex parser can be built by incorporating several
 > smaller ones as class members, yet retaining a nice
 > encapsulation. Something that does not seem possible with
 > X3.

 > An alternative, using parsers defined in functions (as for
 > make_csv_parser), is not always an option though, as such
 > parsers cannot be recursive (rule A is defined via rule B
 > and vice versa).

I think this is the problem described here:

http://boost.2283326.n4.nabble.com/x3-devel-why-2-methods-linking-rule-to-RHS-tp4688021.html

which is another reason, besides the assumed compile time
penalty, for using BOOST_SPIRIT_DEFINE.

 > Your prototype allows using BOOST_SPIRIT_DEFINE inside a
 > struct, hence would definitely come closer to the Qi like
 > encapsulation.
 > I haven't tried it yet, but is it also possible to use
 > parameters, like the field separator fort the CSV parser?
 > As it can't be given at the time of construction, as for
 > make_csv_parser(), one could probably use x3::with<>.

I've updated the code to use an Ops template parameter to
the calc1_gram::gram_deriv:

https://github.com/cppljevans/spirit/blob/ExagonLinkingError/workbench/x3/rule_defns/parse_rule_crtp.cpp#L160

The Ops parameter shows how to change the character use for
the arithmetic operators somewhat like the main_x3's
make_csv_parser's fieldSep argument.

That method is not as "clean" as passing a runtime parameter
thru the argument list, but it's close.

I failed to find documentation for the x3::with; hence, I
couldn't try that.  I even tried grepping the source code;
however, all I found was names beginning with with_ but no
x3::with.  That was with boost_1_61_0.  Do you know where I
could find examples or documentation of x3::with?

HTH,

Larry



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

TONGARI J
2016-12-20 20:34 GMT+08:00 Larry Evans <[hidden email]>:
I've updated the code to use an Ops template parameter to
the calc1_gram::gram_deriv:

https://github.com/cppljevans/spirit/blob/ExagonLinkingError/workbench/x3/rule_defns/parse_rule_crtp.cpp#L160

The Ops parameter shows how to change the character use for
the arithmetic operators somewhat like the main_x3's
make_csv_parser's fieldSep argument.

That method is not as "clean" as passing a runtime parameter
thru the argument list, but it's close.

I failed to find documentation for the x3::with; hence, I
couldn't try that.  I even tried grepping the source code;
however, all I found was names beginning with with_ but no
x3::with.  That was with boost_1_61_0.  Do you know where I
could find examples or documentation of x3::with?


Basically, it allows you to inject some data into the context with a tag.

I like your class-based grammar encapsulation, seems you took a step in the right direction.
It'd be great if you could integrate it into real X3 codebase not just some workbench ;)

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

Sandro Pirkwieser
In reply to this post by cppljevans
Hi,

> I think this is the problem described here:
> http://boost.2283326.n4.nabble.com/x3-devel-why-2-methods-linking-rule-to-RHS-tp4688021.html
> which is another reason, besides the assumed compile time penalty, for using BOOST_SPIRIT_DEFINE.

yes, it covers this topic, too (btw interesting post). In order to write such recursive parsers, in general one has to use BOOST_SPIRIT_DEFINE, yet by doing so the parser has to be defined in a namespace (at least with the current version of BOOST_SPIRIT_DEFINE).

> I've updated the code to use an Ops template parameter to the calc1_gram::gram_deriv:
> https://github.com/cppljevans/spirit/blob/ExagonLinkingError/workbench/x3/rule_defns/parse_rule_crtp.cpp#L160
> The Ops parameter shows how to change the character use for the arithmetic operators somewhat like the main_x3's make_csv_parser's fieldSep argument.
> That method is not as "clean" as passing a runtime parameter thru the argument list, but it's close.

Yeah, good idea. Seth did something for the CSV parser, too.
 
> I failed to find documentation for the x3::with; hence, I couldn't try that.  I even tried grepping the source code; however, all I found was names beginning with with_ but no x3::with.
> That was with boost_1_61_0.  Do you know where I could find examples or documentation of x3::with?

Currently it's not contained in the documentation. I can offer you another example, my CSV parser in the X3 variant (main_x3.cpp), to be found as attachement in this previous mail: https://sourceforge.net/p/spirit/mailman/message/35546745/

Best regards,
Sandro

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

sehe
In reply to this post by cppljevans
On 19-12-16 12:25, Larry Evans wrote:
This use of BOOST_SPIRIT_DEFINE avoids the extra compile time
needed by make_csv_parser method above; 
I don't know how you established that there is such a compile time penalty
hence, Seth, I'm wondering
what the advantage of *not* using the BOOST_SPIRIT_DEFINE method
for associating a rule with it's rule_definition?

That's not what the code does at all. It just creates a parser. You can store it in an auto variable just like any other rule and use it. I have little to no reason to believe the performance of the code or compilation time would differ a lot from the equivalent using BOOST_SPIRIT_DEFINE, but I welcome the effort to produce more comparative benchmarks.

In case you missed it, my gist included most of that. Only the very last of 9 revisions dropped the BOOST_SPIRIT_DEFINE and the baseline chart is from this revision:

commit 01941be830f99106f6526e2961ed79127dc92a10
Author: sehe [hidden email]
Date:   Tue Dec 13 00:57:35 2016 +0100

    Benchmarkified with sanity_checksum

I'm sorry if I should have made it clearer the gist includes the revision history

On 19-12-16 12:25, Larry Evans wrote:
As the comments at the top indicate, I question (as Seth did in an
earlier post) whether there's any advantage to this method.

My goal was to check for easy performance wins. Which I didn't find.

Along the way I refuted the notion that X3 grammar definitions would be REQUIRED to be at namespace scope (except for the case where rules are recursive). Again, the X3 take I gave does not pretend to replace qi::rule, it merely shows how you can package the whole thing in a constructor function, much like `qi::rule`'s constructor, and it's quite elegant when you runtime-parameterize it with the delimiter.


TBH I think the performance bug is the only real issue here.


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

sehe
In reply to this post by cppljevans
On 20-12-16 13:34, Larry Evans wrote:
I failed to find documentation for the x3::with; hence, I
couldn't try that.  I even tried grepping the source code;
however, all I found was names beginning with with_ but no
x3::with.  That was with boost_1_61_0.  Do you know where I
could find examples or documentation of x3::with?

I link three examples in my answer here: https://stackoverflow.com/questions/35208162/boost-spirit-x3-local-variables-and-getting-the-synthesized-attribute/35254118#35254118

One comment there suggested that I had "discovered" `x3::with` during a session of live-coding, and lo and behold, it's here, at the 1:06:00 mark. You can still see me stumble across it and figure out how it works :/


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
In reply to this post by sehe
On 12/20/2016 05:11 PM, Seth wrote:
 > On 19-12-16 12:25, Larry Evans wrote:
 >> This use of BOOST_SPIRIT_DEFINE avoids the extra compile time
 >> needed by make_csv_parser method above;
 > I don't know how you established that there is such a
 > compile time penalty

Sorry for not listing my reasons.  They are listed in my
other post:

   https://sourceforge.net/p/spirit/mailman/message/35561750/

starting with the line:

   I assume it would take more time because of the template
   metaprogramming required at these places:

 >> hence, Seth, I'm wondering what the advantage of *not*
 >> using the BOOST_SPIRIT_DEFINE method for associating a
 >> rule with it's rule_definition?

 > That's not what the code does at all. It just creates a
 > parser.

Sure, but that parser, the parse_rule parser, produced by
BOOST_SPIRIT_DEFINE, is only used in the
rule<ID,ATTR>::parse function:

 
https://github.com/boostorg/spirit/blob/master/include/boost/spirit/home/x3/nonterminal/rule.hpp#L116

Hence, it's only use is to allow the rule::parser to parse
it's right-hand-side (RHS).

Maybe I should have said:

    A BOOST_SPIRIT_DEFINE call in *combination* with the
    rule::parse call to parse_rule "associates a rule with
    it's RHS.".

Is that a fair description?

 > You can store it in an auto variable just like any other
 > rule and use it. I have little to no reason to believe the
 > performance of the code or compilation time would differ a
 > lot from the equivalent using BOOST_SPIRIT_DEFINE, but I
 > welcome the effort to produce more comparative benchmarks.

OK.  Working on it.  I'll post a followup when I've got some
sort of benchmark.

[snip]
 >
 > TBH I think the performance bug is the only real issue here.
 >

I assume you meant the runtime performance bug you first and
then Sandro confirmed, and was probably, according to your
profile runs, somewhere in the `parse_into_container` code.

I agree that this is the most important and most immediate
problem; however, having 2 methods to associate the rule
with it's RHS results in more complicated metaprogramming
and harder to understand code.  It took me a long while to
undersand how the parse_rule actually associated the rule
with it's RHS, and even after I thought I understood it, I
still made a mistake as shown by my post to spirit.devel:

http://boost.2283326.n4.nabble.com/x3-devel-design-add-is-default-parse-rule-true-flag-to-rule-definition-parse-tt4690489.html#none

If this added complexity doesn't buy you anything, then it
should be removed.  As mentioned earlier, I'm working on a
benchmark to see if using 2 methods (Let's label them as the
in-context and BOOST_SPIRIT_DEFINE methods) instead of just
the BOOST_SPIRIT_DEFINE buys anything.

As a benchmark test case, I was thinking of this grammar:

   rul<0> = char_('a');
   rul<1> = char_('1') >> rul<0>;
   rul<2> = char_('2') >> rul<1>;
   rul<3> = char_('3') >> rul<2>;
   .
   .
   .
   rul<N> = char_('N') >> rul<N-1>;

Of course, by <I> I mean something like id<I> where id is:

   template<unsigned I>struct id{};

Since the grammar is non-recursive, the in-context method
could handle it.

Of course, in the case of the in-context method, this would
be:

   def<I> = rul<I> = char_('I') >> def<I-1>;

and, in the case of the BOOST_SPIRIT_DEFINE method, this
would be:

   BOOST_SPIRIT_DEFINE
   ( (rul<0> = char_('a') )
   , (rul<1> = char_('1') >> rul<0> )
   , (rul<2> = char_('2') >> rul<1> )
   , (rul<3> = char_('3') >> rul<2> )
   .
   .
   .
   , (rul<N> = char_('N') >> rul<N-1> )
   )

where BOOST_SPIRIT_DEFINE is the one defined here:

 
https://github.com/cppljevans/spirit/blob/ExagonLinkingError/workbench/x3/rule_defns/parse_rule_crtp.cpp#L41 


Since that code does no attribute processing, to be fair to
the in-context method, the code here:

https://github.com/cppljevans/spirit/blob/ExagonLinkingError/workbench/x3/rule_defns/rule_defns.cpp#L247

which also does no attribute processing, will be used, with
any modifications needed to accommodate the benchmark.
AFAICT, that code does the same template metaprogramming
that real spirit does to allow the rule::parse to call the
parser for it's RHS. (NOTE, if you run the code, you'll see
that the calc_gram_tree test calls parse_rhs_main; whereas
the calc_gram_recur test does not.  That's because the
[rule]_def's in calc_gram_recur are not rule_definition's;
whereas, in calc_gram_tree, the [rule]_def's are
rule_definitions.).

Varying the size of the grammar (by varying N), should
highlight any affect that has on the meta-programming cost.

Does such a benchmark test seem fair to you?

-regards,
Larry



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

sehe
On 21-12-16 13:48, Larry Evans wrote:
> having 2 methods to associate the rule
> with it's RHS results in more complicated metaprogramming
> and harder to understand code

Oh yeah. I'm absolutely not interested in that. I have no use for it.

I'm trying to focus on the perf issue which Sandro reported and I
confirmed after making a proper benchmark and trying to find a culprit.


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
In reply to this post by Sandro Pirkwieser
On 12/20/2016 03:02 AM, Sandro Pirkwieser wrote:
 > Hi Larry,
 >
 >> This doesn't use BOOST_SPIRIT_DEFINE, and I don't think it can because
 >> that would generate one or more parse_rule's template functions within
 >> the make_csv_parser function.  Is that right?
 >> If so, then wouldn't that make this method slower to compile than if
 >> BOOST_SPIRIT_DEFINE could be used?
 >
 > yes, to my knowledge using BOOST_SPIRIT_DEFINE inside a function,
e.g. make_csv_parser, is not possible.
 > But I don't know whether this influences compile time.

According to:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/rule_defns/bench.tmp

there's a significant compile-time penalty without using
BOOST_SPIRIT_DEFINE.

The benchmark code:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/rule_defns/rule_defns_bench.cpp

was run with various methods and results were tabulated in
the bench.tmp file.  The method descriptions are found in
comments to:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/rule_defns/RULE2RHS_METHODS.hpp

The method:

   RULE2RHS_CTX_LIST

is the one which runs without using BOOST_SPIRIT_DEFINE.
As shown in bench.tmp, this performs worse than any other
method, by every measure, when number of rules=12.  In
particular, the elapsed time is significantly worse:

 
https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/rule_defns/bench.tmp#L17 


Note, the code was not run with spirit, but with an
"emulation" of spirit found in the rule_defns_benchmark.cpp.
That emulation just doesn't handle attributes, but, AFAICT,
faithfully mimics what spirit does.   If someone sees where
that's not so; please let me know.

HTH.

-regards,
Larry








------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
On 01/04/2017 12:39 PM, Larry Evans wrote:

> On 12/20/2016 03:02 AM, Sandro Pirkwieser wrote:
>  > Hi Larry,
>  >
>  >> This doesn't use BOOST_SPIRIT_DEFINE, and I don't think it can because
>  >> that would generate one or more parse_rule's template functions within
>  >> the make_csv_parser function.  Is that right?
>  >> If so, then wouldn't that make this method slower to compile than if
>  >> BOOST_SPIRIT_DEFINE could be used?
>  >
>  > yes, to my knowledge using BOOST_SPIRIT_DEFINE inside a function,
> e.g. make_csv_parser, is not possible.
>  > But I don't know whether this influences compile time.
>
> According to:
>
> https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/rule_defns/bench.tmp
>
> there's a significant compile-time penalty without using
> BOOST_SPIRIT_DEFINE.
>
The bench.tmp now includes a 5th method and rules starting at 8.



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
In reply to this post by TONGARI J
On 12/20/2016 07:10 AM, TONGARI J wrote:
> 2016-12-20 20:34 GMT+08:00 Larry Evans <[hidden email]
> <mailto:[hidden email]>>:
>
>     I've updated the code to use an Ops template parameter to
>     the calc1_gram::gram_deriv:
>
>     https://github.com/cppljevans/spirit/blob/ExagonLinkingError/workbench/x3/rule_defns/parse_rule_crtp.cpp#L160
>     <https://github.com/cppljevans/spirit/blob/ExagonLinkingError/workbench/x3/rule_defns/parse_rule_crtp.cpp#L160>
[snip]
>

> I like your class-based grammar encapsulation, seems you took a step in
> the right direction.
Thanks.
> It'd be great if you could integrate it into real X3 codebase not just
> some workbench ;)
>
>
It's in a fork here:

https://github.com/cppljevans/spirit/blob/get_rhs/include/boost/spirit/home/x3/nonterminal/rule.hpp#L237

However, I don't think it allows separate compilation of the
definition, which is, IIUC, purpose of BOOST_SPIRIT_INSTANTIATE:

https://github.com/cppljevans/spirit/blob/get_rhs/include/boost/spirit/home/x3/nonterminal/rule.hpp#L230

Oh well, I'll see if I can think of a workaround :(

HTH.

-regards,
Larry




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
In reply to this post by Sandro Pirkwieser
On 12/20/2016 03:02 AM, Sandro Pirkwieser wrote:
[snip]
 >> Hence, Sandro, could you provide some use-case for this feature?
 >

 > The main topic (aside the more recent performance
 > analysis) was the question to "better" (at least in my
 > view) encapsulate an X3 parser than with namespaces,
 > i.e. more in the style of a Qi parser. There a larger,
 > more complex parser can be built by incorporating several
 > smaller ones as class members, yet retaining a nice
 > encapsulation. Something that does not seem possible with
 > X3.

 > An alternative, using parsers defined in functions (as for
 > make_csv_parser), is not always an option though, as such
 > parsers cannot be recursive (rule A is defined via rule B
 > and vice versa).

 > Your prototype allows using BOOST_SPIRIT_DEFINE inside a
 > struct, hence would definitely come closer to the Qi like
 > encapsulation.

Hi, Sandro.

The code:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/csv_parser/x3_crtp.hpp

implements the csv_parser pretty much as encapsulated as the
qi versiion at:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/csv_parser/qi.hpp

Unfortunately, the runtime suffers the same slowdown as the
x3 version.  The timings here:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/csv_parser/means.out

where produced by:

   make -k USE_MACRO_INCLUDE=1 means

Those timings, and looking at the code, indicates the
slowdown is due to the attribute transforms done by:

https://github.com/cppljevans/spirit/blob/get_rhs/include/boost/spirit/home/x3/nonterminal/detail/rule.hpp#L310

That's because, as shown here:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/csv_parser/means.out#L5

and here:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/csv_parser/means.out#L7

the time is *less* than the time for the qi method here:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/csv_parser/means.out#L1

OTOH, the lines where XRULE0 occurs (meaning the attribute
transform is done within the rule_definition::parse
function), the times are more than the qi method.

 > I haven't tried it yet, but is it also possible to use
 > parameters, like the field separator fort the CSV parser?
 > As it can't be given at the time of construction, as for
 > make_csv_parser(), one could probably use x3::with<>.

This is done here:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/csv_parser/x3_crtp.hpp#L28

 > Best regards,
 > Sandro
 >
HTH.

-regards,
Larry



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
In reply to this post by sehe
On 12/13/2016 03:31 PM, Seth wrote:
[snip]
> Well. I did start off with the bad news because I think it's a
> performance /bug/.
> I agree that the performance doesn't entice a move to X3 for this
> particular grammar :)
>
> I haven't understood it fully but the profiler results seem to point at
> `parse_into_container` and `std::....::basic_string::aux_replace`
> (quoting from memory)

Hi Seth,

Could you please let us know how you profiled the code?
I'd like to see if I could do s similar profile on a revision
of spirit and this benchmark code to try and narrow down the
bug location.

-regards,
Larry
[snip]


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

sehe
On 26-01-17 03:41, Larry Evans wrote:
Could you please let us know how you profiled the code?

I thought sharing the full code with history would have been quite sufficient, from December 13th:

    git clone https://gist.github.com/7dc223f00d8cc46d8be566756b7fdf03.git

That was reiterated to you personally in the post of December 21st:

In case you missed it, my gist included most of that. Only the very last of 9 revisions dropped the BOOST_SPIRIT_DEFINE and the baseline chart is from this revision:

commit 01941be830f99106f6526e2961ed79127dc92a10
Author: sehe [hidden email]
Date:   Tue Dec 13 00:57:35 2016 +0100

    Benchmarkified with sanity_checksum

I'm sorry if I should have made it clearer the gist includes the revision history

The only "outside" dependency is Nonius: https://github.com/rmartinho/nonius


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
On 01/27/2017 09:57 AM, Seth wrote:
> On 26-01-17 03:41, Larry Evans wrote:
>> Could you please let us know how you profiled the code?
>
> I thought sharing the full code with history would have been quite
> sufficient, from December 13th:
>
>>     git clone https://gist.github.com/7dc223f00d8cc46d8be566756b7fdf03.git
>>
Sorry Seth, I've downloaded that code, and looked at the history; yet,
I don't see any indication of how it was profiled.  IOW, I don't
see any CMake file or make file or Jamfile which have recipe's with
any profiler commands such as those mentioned here:

http://gernotklingler.com/blog/gprof-valgrind-gperftools-evaluation-tools-application-level-cpu-profiling-linux/

I did run nonius with a revision of the code, as shown here:

https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/csv_parser/Makefile

but that just shows which code runs faster than the other,
it doesn't indicate where the code hot spots are; hence,
that's why I was interested in which profiler you used.
I've tried 2, but they don't seem much help.

-regards,
Larry




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
On 01/27/2017 12:24 PM, Larry Evans wrote:

> On 01/27/2017 09:57 AM, Seth wrote:
>> On 26-01-17 03:41, Larry Evans wrote:
>>> Could you please let us know how you profiled the code?
>>
>> I thought sharing the full code with history would have been quite
>> sufficient, from December 13th:
>>
>>>     git clone https://gist.github.com/7dc223f00d8cc46d8be566756b7fdf03.git
>>>
> Sorry Seth, I've downloaded that code, and looked at the history; yet,
> I don't see any indication of how it was profiled.  IOW, I don't
> see any CMake file or make file or Jamfile which have recipe's with
> any profiler commands such as those mentioned here:
>
> http://gernotklingler.com/blog/gprof-valgrind-gperftools-evaluation-tools-application-level-cpu-profiling-linux/
>
> I did run nonius with a revision of the code, as shown here:
>
> https://github.com/cppljevans/spirit/blob/get_rhs/workbench/x3/csv_parser/Makefile
>
[snip]
Never mind.  I've got gprof and valgind --tool=callgrind to profile the
code.

-Larry




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

sehe
On 29-01-17 12:36, Larry Evans wrote:
Sorry Seth, I've downloaded that code, and looked at the history; yet,
> I don't see any indication of how it was profiled. 
Ah. I start to see you don't mean benchmarking, but actual profiling. I don't think I did.

I've tried 2, but they don't seem much help.
And that's why.
[snip]
Never mind.  I've got gprof and valgind --tool=callgrind to profile the 
code.

If I use anything, it's callgrind or perf.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Design/structure X3 parser more like Qi parser

cppljevans
In reply to this post by sehe
On 12/13/2016 11:56 AM, Seth wrote:
[snip]
 >
 > Hope this helps someone spot an improvement/cause of the poor
performance.
 >
 > Cheers,
 > Seth
Seth, and anyone else who might be interested in helping
find the cause of the poor performance, I've uploaded:

   summary.chrono-csv-*.out

to:

https://github.com/cppljevans/spirit/tree/develop/workbench/x3/csv_parser

which shows that at -O0, x3 performs better, but as the
optimization increases, qi begins to perform better.

Since the crossover of performance occurs between -O0 and
-O1 for the gcc compiler, I also uploaded profile's for
those to optimization's for gcc to:

https://github.com/cppljevans/spirit/blob/develop/workbench/x3/csv_parser/gprof-csv-x3_ns_rdef-spirit-GET0-XRULE0-OptO0-gcc6_20151018.flat_graph
https://github.com/cppljevans/spirit/blob/develop/workbench/x3/csv_parser/gprof-csv-x3_ns_rdef-spirit-GET0-XRULE0-OptO1-gcc6_20151018.flat_graph

At the moment, based on the contents of those files, I'm
unable to discern any likely cause of the poor performance.

I'm hoping that someone reading this, with more experience
with and understanding of gprof, could infer some likely
cause of the poor performance.  If so, maybe they could
suggest some change to improve the performance which I would
gratefully implement and report back here with the results.

-regards,
Larry




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
12