Hello, How to parse lists?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Hello, How to parse lists?

Jens Kallup
Hello,

How to parse "parameter ccc , ccc, ggg, ddd" ? dynamical - not static!
I get success: "parameter ccc, vv vvv vvv",
and this is wrong.

TIA for help
Jens

This rule, I use, is wrong:

   symbol_def_parameter =
     (symbol_parameter >> symbol_alpha >> *(qi::char_(',') % symbol_alpha))
     ;

              symbol_alpha =
                   qi::char_("a-zA-Z_") >>
                  *qi::char_("a-zA-Z0-9_")
                  ;

symbol_parameter = no_case["parameter"];


------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Hello, How to parse lists?

Mikael Asplund
Hi Jens,

First, the list parser ("%") doesn't need the kleen star ("*") before it, since it parsers one or more entries into a list.
Second, the list parsers is "WHAT to parse, delimited by WHAT" ("entry % delimiter"), so change:

symbol_def_parameter =
     (symbol_parameter >> symbol_alpha >> *(qi::char_(',') % symbol_alpha))
     ;

...into this (one or more entries in list):

symbol_def_parameter =
     (symbol_parameter >> (symbol_alpha  % qi::char_(',')))
     ;

... or, if you want ZERO or more entries in your list (notice the minus character that means optional):

symbol_def_parameter =
     (symbol_parameter >> -(symbol_alpha  % qi::char_(',')))
     ;

Regards,
  Mikael

-----Original Message-----
From: Jens Kallup [mailto:[hidden email]]
Sent: Saturday, October 29, 2016 13:11
To: Spirit General Mailing List <[hidden email]>
Subject: [Spirit-general] Hello, How to parse lists?

Hello,

How to parse "parameter ccc , ccc, ggg, ddd" ? dynamical - not static!
I get success: "parameter ccc, vv vvv vvv", and this is wrong.

TIA for help
Jens

This rule, I use, is wrong:

   symbol_def_parameter =
     (symbol_parameter >> symbol_alpha >> *(qi::char_(',') % symbol_alpha))
     ;

              symbol_alpha =
                   qi::char_("a-zA-Z_") >>
                  *qi::char_("a-zA-Z0-9_")
                  ;

symbol_parameter = no_case["parameter"];


------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general

------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Hello, How to parse lists?

Jens Kallup
Hi Mikael,

I try out your hint, but it does not work!
What version of spirit did you refer?

I download 1.62.0 from boost.org for few
week, ago.

Have the libary changes some thing new?

Jens

------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Hello, How to parse lists?

Jens Kallup
In reply to this post by Mikael Asplund
The problem:

1. input("parameter fff");  // true
2. input("parameter fff , ccc");  // true
3. input("parameter fff, ccc ddd ddd dd dd");  // true

Fall 3 should be wrong! :-(

Jens

------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Hello, How to parse lists?

Exagon
Could you give us a Ebnf or another grammar of what you are trying to achieve?
Greetings Exagon
Reply | Threaded
Open this post in threaded view
|

Re: Hello, How to parse lists?

Mikael Asplund
In reply to this post by Jens Kallup
I'm not sure about that. Never used qi, and this is probably something with skippers or so. Anyone else?

Full code below:

  symbol_def_parameter =
           (symbol_parameter >  (symbol_alpha % qi::char_(','))
          ;

  symbol_alpha =
       qi::char_("a-zA-Z_") >>
      *qi::char_("a-zA-Z0-9_")
      ;

   symbol_parameter = no_case["parameter"];

Regards,
  Mikael

-----Original Message-----
From: Jens Kallup [mailto:[hidden email]]
Sent: Saturday, October 29, 2016 22:56
To: Spirit General Mailing List <[hidden email]>
Subject: Re: [Spirit-general] Hello, How to parse lists?

The problem:

1. input("parameter fff");  // true
2. input("parameter fff , ccc");  // true 3. input("parameter fff, ccc ddd ddd dd dd");  // true

Fall 3 should be wrong! :-(

Jens

------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general

------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Hello, How to parse lists?

Jens Kallup
Here is the Syntax:

parameter =  alpha.input("parameter") > def_param
def_param =  alpha.input(alpha) |
             (alpha.input(alpha) > ',' > def_param)
alpha = [a-zA-Z_][a-zA-Z0-9_]
------------------------------------------
source example:
parameter ccc, ccc, ccd

Problem:
parameter ccxc ffff fff

Here is the source code:

#define BOOST_SPIRIT_DEBUG
#define BOOST_SPIRIT_ACTIONS_ALLOW_ATTR_COMPAT

#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>

#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/spirit/include/phoenix_fusion.hpp>
#include <boost/spirit/include/phoenix_stl.hpp>
#include <boost/spirit/include/phoenix_object.hpp>

#include <string>
#include <iostream>

//#define USE_QT
#ifdef  USE_QT
#include <QMessageBox>
#endif

using namespace std;
using namespace boost::spirit;

namespace client
{
      namespace fusion = boost::fusion;
      namespace phoenix = boost::phoenix;

      namespace qi = boost::spirit::qi;
      namespace ascii = boost::spirit::ascii;

      template <typename Iterator>
      struct dbase_skipper : public qi::grammar<Iterator>
      {
          dbase_skipper() : dbase_skipper::base_type(my_skip, "dBase")
          {
              using qi::ascii::char_;
              using qi::ascii::space;
              using qi::eol;
              using qi::eoi;

              using qi::on_error;
              using qi::fail;

              using phoenix::val;

              my_skip = (char_("[ \t\n\r]"))                            |
              ("**" >> *(char_ - eol) >> (eol | eoi | char_("[\n\r]"))) |
              ("&&" >> *(char_ - eol) >> (eol | eoi | char_("[\n\r]"))) |
              ("//" >> *(char_ - eol) >> (eol | eoi | char_("[\n\r]"))) |
              ("/*" >> *(char_ - "*/") >> "*/")
              ;

              on_error<fail>
               (
                   my_skip
                 , std::cout
                       << val("Error! Expecting comment")
                       << std::endl
               );

              BOOST_SPIRIT_DEBUG_NODE((my_skip));
          }
          qi::rule<Iterator> my_skip;
      };

      template <typename Iterator, typename Skipper =
dbase_skipper<Iterator>>
      struct dbase_grammar : public qi::grammar<Iterator, Skipper>
      {
          qi::rule<Iterator, Skipper> start, run_app;
          dbase_grammar()  : dbase_grammar::base_type(start)
          {
              using boost::spirit::ascii::no_case;

              using qi::lit;
              using qi::char_;
              using qi::lexeme;

              using qi::on_error;
              using qi::fail;

              using phoenix::construct;
              using phoenix::val;

              start   = run_app.alias();
              run_app = - symsbols;

              expression =
                      term.alias()
                      >> *(
                        ('+' >> term )
                      | ('-' >> term ))
                      ;
              term =
                      factor.alias()
                      >> *(
                        ('*' >> factor )
                      | ('/' >> factor ))
                      ;

              factor =
                      ( symbol_digit
                      ///| symbol_alpha
                      )
                      >> *(
                      ('('   >> expression >> ')')
                      | ('-' >> factor     )
                      | ('+' >> factor     ))
                      ;

              symsbols =
                    (symbol_def_parameter)
                      |
                  (((symbol_class > symbol_alpha >
                     symbol_of    > symbol_alpha > symbol_endclass >>
run_app) |
                    (symbol_class > symbol_alpha >
                     symbol_of    > symbol_alpha > symbol_endclass)
                    )              |
                  ((symbol_alpha  > qi::char_('=') > expression >>
run_app) |
                   (symbol_alpha  > qi::char_('=') > expression         ))
                  )
                  ;

              symbol_class     = no_case["class"];
              symbol_of        = no_case["of"];
              symbol_endclass  = no_case["endclass"];
              symbol_parameter = no_case["parameter"];

             symbol_def_parameter =
                   (symbol_parameter > (symbol_alpha % qi::char_(',')))
                   ;

              symbol_space =
                  +(qi::char_(" \t\n\r") | eol | eoi)
                  ;

              symbol_alpha =
                   qi::char_("a-zA-Z_") >>
                  *qi::char_("a-zA-Z0-9_")
                  ;

              symbol_digit =
                  +(qi::digit)
                  ;

              on_error<fail>
               (
                   start
                 , std::cout
                       << val("Error! Expecting ")
                       << _4 // what failed?
                       << val(" here: \"")
                       << construct<std::string>(_3, _2)   // iterators
to error-pos, end
                       << val("\"")
                       << std::endl
               );

              BOOST_SPIRIT_DEBUG_NODE(start);
              BOOST_SPIRIT_DEBUG_NODE(symsbols);
              BOOST_SPIRIT_DEBUG_NODE(symbol_of);
              BOOST_SPIRIT_DEBUG_NODE(symbol_endclass);
              BOOST_SPIRIT_DEBUG_NODE(symbol_class);
              BOOST_SPIRIT_DEBUG_NODE(symbol_space);
              BOOST_SPIRIT_DEBUG_NODE(symbol_alpha);
              BOOST_SPIRIT_DEBUG_NODE(symbol_digit);
              BOOST_SPIRIT_DEBUG_NODE(symbol_ident);

              BOOST_SPIRIT_DEBUG_NODE(symbol_parameter);
              BOOST_SPIRIT_DEBUG_NODE(symbol_def_parameter);


              BOOST_SPIRIT_DEBUG_NODE(expression);
              BOOST_SPIRIT_DEBUG_NODE(term);
              BOOST_SPIRIT_DEBUG_NODE(factor);
          }

          qi::rule<Iterator, std::string()>
          symbol_alpha,
          symbol_ident;

          qi::rule<Iterator, Skipper>
          symsbols,
          symbol_digit,
          symbol_space,
          symbol_class,
          symbol_endclass,
          symbol_of,
          symbol_parameter,
          symbol_def_parameter,
          symbol_def_parameter2;

          qi::rule<Iterator, Skipper> expression, term, factor;
      };
}

bool my_parser(std::string const str)
{
      typedef std::string::const_iterator iterator_t;

      typedef client::dbase_grammar <iterator_t> grammar;
      typedef client::dbase_skipper <iterator_t> skipper;

      grammar pg;
      skipper skp;

      iterator_t iter = str.begin();
      iterator_t end  = str.end();

      bool r = phrase_parse(iter, end, pg, skp);
      if (r == true) {
          #ifdef USE_QT
          QMessageBox::information(0,"Parser", "Parsing SUCCESS.");
          #else
          std::cout << "SUCCESS" << std::endl;
          #endif
          return true;
      }

      if (iter != end) {
          std::cout << "Remaining: '" << std::string(iter, end) <<
std::endl;
          #ifdef USE_QT
          QMessageBox::information(0,"Parser", "Parsing ERROR.");
          #else
          std::cout << "ERROR" << std::endl;
          #endif
          return false;
      }   return false;
}

bool parseText(std::string str, int mode)
{
      return my_parser(str);
}


------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Hello, How to parse lists?

sehe
On 30-10-16 00:29, Jens Kallup wrote:
parameter =  alpha.input("parameter") > def_param
def_param =  alpha.input(alpha) |
             (alpha.input(alpha) > ',' > def_param)
alpha = [a-zA-Z_][a-zA-Z0-9_]

Finally something we can work with. (remember I've asked for SSCCE or goals for weeks).

Here's the simplest translation of the above:

        ident = qi::char_("a-zA-Z_") >> qi::char_("a-zA-Z0-9_");
        parameter = qi::lexeme["parameter"] >> (ident % ',');

First off, I think `alpha` should be `[a-zA-Z_][a-zA-Z0-9_]...` since otherwise fff wouldn't match (too many characters)

        ident = qi::char_("a-zA-Z_") >> *qi::char_("a-zA-Z0-9_");

Now it parses the test cases from your earlier mail:

<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">===== input 'parameter fff'
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ok: 1
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ident: fff
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">===== input 'parameter fff , ccc'
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ok: 2
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ident: fff
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ident: ccc
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">===== input 'parameter fff, ccc ddd ddd dd dd'
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ok: 2
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ident: fff
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ident: ccc
Remaining unparsed: 'ddd ddd dd dd'...
Now you wish for the last one to "fail"? I presume that might be because end of input has not been reached. You can either check that the end iterator is reached, or add `>> qi::eoi` to ensure that EOI (end-of-input) was reached:

        parameter = qi::lexeme["parameter"] >> (ident % ',') >> qi::eoi;

Then the output becomes:

<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">===== input 'parameter fff'
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ok: 1
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ident: fff
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">===== input 'parameter fff , ccc'
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ok: 2
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ident: fff
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Ident: ccc
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">===== input 'parameter fff, ccc ddd ddd dd dd'
<pre class="f0" onclick="window.jumpToError(this)" onmouseover="window.highlightError(this, true)" onmouseout="window.highlightError(this, false)" style="display: inline-block; margin: 0px; padding: 0px;">Fail
Remaining unparsed: 'parameter fff, ccc ddd ddd dd dd'...



------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Hello, How to parse lists?

Jens Kallup
Hello Seth,

thank You!!!
This is exactly what i want:

symbol_parameter = lexeme[no_case["parameter"]];
symbol_def_parameter =
   (symbol_parameter >> (symbol_alpha % ',') >> (qi::eoi | qi::eol));

Thank You very much, You made my day!
cheers Jens

On 10/31/2016 09:15 AM, Seth wrote:

> Now you wish for the last one to "fail"? I presume that might be
> because end of input has not been reached. You can either check that
> the end iterator is reached, or add `>> qi::eoi` to ensure that EOI
> (end-of-input) was reached:
>
>         parameter = qi::lexeme["parameter"] >> (ident % ',') >> qi::eoi;
>
> Then the output becomes
> <http://coliru.stacked-crooked.com/a/95c7f4a181ff0713>:
>
>     ===== input 'parameter fff'
>
>     Ok: 1
>
>     Ident: fff
>
>     ===== input 'parameter fff , ccc'
>
>     Ok: 2
>
>     Ident: fff
>
>     Ident: ccc
>
>     ===== input 'parameter fff, ccc ddd ddd dd dd'
>
>     Fail
>
>     Remaining unparsed: 'parameter fff, ccc ddd ddd dd dd'...
>


------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general