X3: Supported character encodings / string literals

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

X3: Supported character encodings / string literals

teajay-2
Hello,

On the way to implementing no_case, I stumbled across some imperfections
in the character encoding handling and string literals.

1. The lit parser is in the default x3 namespace, and it even pulls in
the standard encoding namespace. All the as_parser templates in the
extension namespace use the standard encoding. This leads to not making
it possible to use anything other encoding out of the box. It this meant
to be so ? I'm not sure, since the char_ parser, supports the different
encodings properly.

2. There is no automatic transformation from std::string to the lit
parser. Is this meant to be left out ?

3. Is the iso8859_1 encoding meant to be fully supported ? I'm asking,
as it seams that this encoding hasn't received as much attention in X3
as the others yet.

4. The character sets as in qi : char_("a-z"), are not supported
anymore. Is this correct ?

For now I'll remove the test cases, which pose problems.

Regards,

Thomas Bernard


------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: X3: Supported character encodings / string literals

Joel de Guzman
On 10/6/14, 6:30 AM, Thomas Bernard wrote:

> Hello,
>
> On the way to implementing no_case, I stumbled across some imperfections
> in the character encoding handling and string literals.
>
> 1. The lit parser is in the default x3 namespace, and it even pulls in
> the standard encoding namespace. All the as_parser templates in the
> extension namespace use the standard encoding. This leads to not making
> it possible to use anything other encoding out of the box. It this meant
> to be so ? I'm not sure, since the char_ parser, supports the different
> encodings properly.

That's the same in Qi. By default, you have the standard encoding,
but that should not prevent you from using some other encodings.
Do you have a particular problem where you can't use other encodings?

> 2. There is no automatic transformation from std::string to the lit
> parser. Is this meant to be left out ?

Not sure. We should do as we do in Qi in this particular case.

> 3. Is the iso8859_1 encoding meant to be fully supported ? I'm asking,
> as it seams that this encoding hasn't received as much attention in X3
> as the others yet.

Yes, it should be fully supported. What do you mean by not
receiving much attention? Apart from standard encoding being
default, all encodings have the same attention, as far as I
am concerned. Did I miss something?

> 4. The character sets as in qi : char_("a-z"), are not supported
> anymore. Is this correct ?

It should be supported, if it is not yet.

> For now I'll remove the test cases, which pose problems.

Please don't. What problems are you seeing?

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/


------------------------------------------------------------------------------
Slashdot TV.  Videos for Nerds.  Stuff that Matters.
http://pubads.g.doubleclick.net/gampad/clk?id=160591471&iu=/4140/ostg.clktrk
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: X3: Supported character encodings / string literals

teajay-2
On 10/06/2014 12:21 AM, Joel de Guzman wrote:

> On 10/6/14, 6:30 AM, Thomas Bernard wrote:
>> Hello,
>>
>> On the way to implementing no_case, I stumbled across some imperfections
>> in the character encoding handling and string literals.
>>
>> 1. The lit parser is in the default x3 namespace, and it even pulls in
>> the standard encoding namespace. All the as_parser templates in the
>> extension namespace use the standard encoding. This leads to not making
>> it possible to use anything other encoding out of the box. It this meant
>> to be so ? I'm not sure, since the char_ parser, supports the different
>> encodings properly.
>
> That's the same in Qi. By default, you have the standard encoding,
> but that should not prevent you from using some other encodings.
> Do you have a particular problem where you can't use other encodings?
>
I noticed this in the no_case test using the iso8859_1 encoding for lit.
The lit parser isn't separated in the different encoding namespaces as
the char_ parser is. This leads to always using the default standard
encoding. I've also noticed that the as_parser templates, in the
extension namespace all use the standard encoding. I don't know if this
is also a problem.

>> 2. There is no automatic transformation from std::string to the lit
>> parser. Is this meant to be left out ?
>
> Not sure. We should do as we do in Qi in this particular case.
>
I also noticed this with the no_case test cases, we need a
specialization of the as_parser templates to handle this.

>> 3. Is the iso8859_1 encoding meant to be fully supported ? I'm asking,
>> as it seams that this encoding hasn't received as much attention in X3
>> as the others yet.
>
> Yes, it should be fully supported. What do you mean by not
> receiving much attention? Apart from standard encoding being
> default, all encodings have the same attention, as far as I
> am concerned. Did I miss something?
>
This assumption, is based on the fact that I didn't find many test with
this encoding variation. It wasn't present in the char_ tests and isn't
there either for lit etc ...

>> 4. The character sets as in qi : char_("a-z"), are not supported
>> anymore. Is this correct ?
>
> It should be supported, if it is not yet.
>
Ok. Either I missed something, or this isn't there yet. I'll double check.
>> For now I'll remove the test cases, which pose problems.
>
> Please don't. What problems are you seeing?
>

Well the problems I mentioned above. Nothing really bad.
What I did is just comment out the test cases.

I think it would be valuable to get some early feedback on the way I
implemented the no_case directive. It's not complete yet, but I could
send a pull request this evening as a reviewing basis.


> Regards,
>



------------------------------------------------------------------------------
Slashdot TV.  Videos for Nerds.  Stuff that Matters.
http://pubads.g.doubleclick.net/gampad/clk?id=160591471&iu=/4140/ostg.clktrk
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general