[Qi] Integer parsing

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[Qi] Integer parsing

fpelliccioni
Hi all,

I have a type like this...

    typedef
        boost::variant<
            short
            , int
            , long
            , long long
        >
        number;                //TODO: unsigned and char
       
... and this is the Qi Rule for parsing it.

     qi::rule<Iterator, number(), ascii::space_type> integer_number_rule = short_ | int_ | long_ | long_long;
       
These are the data to parse:
       
    std::string int_str_1 = "31555";
    std::string int_str_2 = "-30666";
    std::string int_str_3 = "63999";
       
When I parse int_str_1 and int_str_2, it is OK, but when I try to parse int_str_3 I got a parsing error.

On my platform (Win32 - VC9.0) 63999 is an unsigned short, but I think that Iy should be parsed as int (signed).
What do you think? Am I wrong?

Thanks,
Fernando.



------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: [Qi] Integer parsing

Hartmut Kaiser
> I have a type like this...
>
>     typedef
>         boost::variant<
>             short
>             , int
>             , long
>             , long long
>         >
>         number;                //TODO: unsigned and char
>
> ... and this is the Qi Rule for parsing it.
>
>      qi::rule<Iterator, number(), ascii::space_type> integer_number_rule =
> short_ | int_ | long_ | long_long;
>
> These are the data to parse:
>
>     std::string int_str_1 = "31555";
>     std::string int_str_2 = "-30666";
>     std::string int_str_3 = "63999";
>
> When I parse int_str_1 and int_str_2, it is OK, but when I try to parse
> int_str_3 I got a parsing error.
>
> On my platform (Win32 - VC9.0) 63999 is an unsigned short, but I think
> that Iy should be parsed as int (signed).
> What do you think? Am I wrong?

That's a bug in Spirit which has been fixed in the next release.

Regards Hartmut

---------------
Meet me at BoostCon
www.boostcon.com




------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: [Qi] Integer parsing

fpelliccioni
On Tue, Jul 13, 2010 at 5:43 PM, Hartmut Kaiser <[hidden email]> wrote:
> I have a type like this...
>
>     typedef
>         boost::variant<
>             short
>             , int
>             , long
>             , long long
>         >
>         number;                //TODO: unsigned and char
>
> ... and this is the Qi Rule for parsing it.
>
>      qi::rule<Iterator, number(), ascii::space_type> integer_number_rule =
> short_ | int_ | long_ | long_long;
>
> These are the data to parse:
>
>     std::string int_str_1 = "31555";
>     std::string int_str_2 = "-30666";
>     std::string int_str_3 = "63999";
>
> When I parse int_str_1 and int_str_2, it is OK, but when I try to parse
> int_str_3 I got a parsing error.
>
> On my platform (Win32 - VC9.0) 63999 is an unsigned short, but I think
> that Iy should be parsed as int (signed).
> What do you think? Am I wrong?

That's a bug in Spirit which has been fixed in the next release.

Regards Hartmut



Hartmut, thanks for all your quick responses to my questions, you're very kind.

I have another question ...

Anyone know what is the memory footprint of boost::variant ?

I'm thinking it might be better to use ...

struct number
{
    long long val_;
};

... instead of ...

     typedef
         boost::variant<
             short
             , int
             , long
             , long long
         >
         number;

With the first I can parse char, short, int, long and long long... The drawback is for store a value less than "long long" I will be wasting memory.
It all depends on what the is the
memory footprint of boost::variant.

Regards,
Fernando.



------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: [Qi] Integer parsing

Cory Nelson
On Tue, Jul 13, 2010 at 2:56 PM, Fernando Pelliccioni
<[hidden email]> wrote:

> I have another question ...
>
> Anyone know what is the memory footprint of boost::variant ?
>
> I'm thinking it might be better to use ...
>
> struct number
> {
>     long long val_;
> };
>
> ... instead of ...
>
>      typedef
>          boost::variant<
>              short
>              , int
>              , long
>              , long long
>          >
>          number;
>
> With the first I can parse char, short, int, long and long long... The
> drawback is for store a value less than "long long" I will be wasting
> memory.
> It all depends on what the is the memory footprint of boost::variant.

Variant is a union of all its types plus an int to specify which type
is the current type.  A single long long will take up less memory, and
a single long_long parser will also be faster than what you've got.

--
Cory Nelson
http://int64.org

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: [Qi] Integer parsing

Hartmut Kaiser
In reply to this post by fpelliccioni
> I have another question ...
>
> Anyone know what is the memory footprint of boost::variant ?
>
> I'm thinking it might be better to use ...
> struct number
> {
>     long long val_;
> };
> ... instead of ...
>
>      typedef
>          boost::variant<
>              short
>              , int
>              , long
>              , long long
>          >
>          number;
>
> With the first I can parse char, short, int, long and long long... The
> drawback is for store a value less than "long long" I will be wasting
> memory.
> It all depends on what the is the memory footprint of boost::variant.

Well, a simple program:

#include <iostream>
#include <boost/variant.hpp>

struct number
{
    long long val_;
};

typedef boost::variant<
    short, int, long, long long> variant_number;

int main()
{
    std::cout << "plain struct: " << sizeof(number) << std::endl;
    std::cout << "variant: " << sizeof(variant_number) << std::endl;
   
    return 0;
}

Gives me for gcc 4.4.3 (32bit):

plain struct: 8
variant: 16

while VC2010 (64bit) gives:

plain struct: 8
variant: 16

which seems to be fairly consistent.

Regards Hartmut

---------------
Meet me at BoostCon
www.boostcon.com




------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: [Qi] Integer parsing

fpelliccioni
Yes! Thanks!

I saw the S-Expressions article of Joel, but was to late !!!
I think that I will use an approach like utree!

Thanks,
Fernando.

On Tue, Jul 13, 2010 at 8:14 PM, Hartmut Kaiser <[hidden email]> wrote:
> I have another question ...
>
> Anyone know what is the memory footprint of boost::variant ?
>
> I'm thinking it might be better to use ...
> struct number
> {
>     long long val_;
> };
> ... instead of ...
>
>      typedef
>          boost::variant<
>              short
>              , int
>              , long
>              , long long
>          >
>          number;
>
> With the first I can parse char, short, int, long and long long... The
> drawback is for store a value less than "long long" I will be wasting
> memory.
> It all depends on what the is the memory footprint of boost::variant.

Well, a simple program:

#include <iostream>
#include <boost/variant.hpp>

struct number
{
   long long val_;
};

typedef boost::variant<
   short, int, long, long long> variant_number;

int main()
{
   std::cout << "plain struct: " << sizeof(number) << std::endl;
   std::cout << "variant: " << sizeof(variant_number) << std::endl;

   return 0;
}

Gives me for gcc 4.4.3 (32bit):

plain struct: 8
variant: 16

while VC2010 (64bit) gives:

plain struct: 8
variant: 16

which seems to be fairly consistent.

Regards Hartmut

---------------
Meet me at BoostCon
www.boostcon.com




------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general


------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: [Qi] Integer parsing

Joel de Guzman-2
On 7/14/10 7:39 AM, Fernando Pelliccioni wrote:
> Yes! Thanks!
>
> I saw the S-Expressions article of Joel, but was to late !!!
> I think that I will use an approach like utree!

I still owe you guys a follow-up article on this, but just FYI,
I'm positioning the utree as a generic AST for Spirit. Hartmut
and I agreed that it will be an official part of Spirit support.

Regards,
--
Joel de Guzman
http://www.boostpro.com
http://spirit.sf.net




------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: [Qi] Integer parsing

cppljevans
In reply to this post by fpelliccioni
On 07/13/10 18:39, Fernando Pelliccioni wrote:
> Yes! Thanks!
>
> I saw the S-Expressions article of Joel, but was to late !!!
> I think that I will use an approach like utree!
>
> Thanks,
> Fernando.
I don't see how that will help.  the utree in the S-Expressions article
still has a tag associated with it.  Even if the tag is 1 bit, it will
still cause as much space as the largest variant component to be
consumed to achieve proper alignment for that largest variant component.
The utree simply uses that extra space as part of, I think, the
fast_string component of the union. IOW, take the largest component of
the union, other than the fast_string, double that, and that's the size
of the union required.  In your case, the union would be:

         union
         {
              short s;
              int i;
              long l;
              long long ll;
         };

Now to store the tag, you need something that values from:
   0 (the tag for s)
to:
   3 (the tag for ll)
Say you choose car as the tag type (as utree does, IIRC).
Then your variant would be:
       struct integral_variant
       {
         union
         {
              short s;
              int i;
              long l;
              long long ll;
         } u;
         char tag;
       };

The size of alignment of u has to be the largest
alignment of all the union members which I think is
the alignment of ll.  Now in order for integral_variant
to have it's u aligned correctly, it's alignment must
be the same as ll's.  Now, if two integral_variant's
are put into a vector<integral_variant>, then the space
in which the tag is placed must be padded to the size
of ll's alignment so that both members fo the
vector<integral_variant> will be at the proper alignment.
Since:

   boost::alignment_of<long long>::value

is 8, that means tag has to be padded to a size of 8 and
since:

   sizeof(long long)

is also 8, sizeof(integral_variant) has to be 16 (8 for
the u and 8 for the padded tag.

HTH.

-Larry




------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: [Qi] Integer parsing

fpelliccioni
On Wed, Jul 14, 2010 at 7:06 AM, Larry Evans <[hidden email]> wrote:
I don't see how that will help.  the utree in the S-Expressions article
still has a tag associated with it.  Even if the tag is 1 bit, it will
still cause as much space as the largest variant component to be
consumed to achieve proper alignment for that largest variant component.
The utree simply uses that extra space as part of, I think, the
fast_string component of the union. IOW, take the largest component of
the union, other than the fast_string, double that, and that's the size
of the union required.  In your case, the union would be:

        union
        {
             short s;
             int i;
             long l;
             long long ll;
        };

Now to store the tag, you need something that values from:
  0 (the tag for s)
to:
  3 (the tag for ll)
Say you choose car as the tag type (as utree does, IIRC).
Then your variant would be:
      struct integral_variant
      {
        union
        {
             short s;
             int i;
             long l;
             long long ll;
        } u;
        char tag;
      };

The size of alignment of u has to be the largest
alignment of all the union members which I think is
the alignment of ll.  Now in order for integral_variant
to have it's u aligned correctly, it's alignment must
be the same as ll's.  Now, if two integral_variant's
are put into a vector<integral_variant>, then the space
in which the tag is placed must be padded to the size
of ll's alignment so that both members fo the
vector<integral_variant> will be at the proper alignment.
Since:

  boost::alignment_of<long long>::value

is 8, that means tag has to be padded to a size of 8 and
since:

  sizeof(long long)

is also 8, sizeof(integral_variant) has to be 16 (8 for
the u and 8 for the padded tag.

HTH.

-Larry

Yes, that is true, but I forgot to clarify that I have another rules like ..

qi::rule<Iterator, value(), ascii::space_type> value_rule %= object_rule | array_rule | string_rule | number_rule | boolean_rule | null_rule;

The value type is defined as...


typedef
    boost::variant<
        boost::recursive_wrapper<object>
        , boost::recursive_wrapper<array>
        , std::string
        , number
        , boolean
        , null
    >
    value;
   
In this case, I think that it is wise to use a solution as utree.

Regards,
Fernando.


   


------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: [Qi] Integer parsing

gillesb68
In reply to this post by Hartmut Kaiser
QI rules enclosing a “long long” (see the question “link”) causes build problems with Visual Studio 2015. The same rule compiles gracefully with a shorter integer of type “long”. Could you give any idea about what should go wrong?
Gilles