tree_to_xml with Unicode

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

tree_to_xml with Unicode

mohan0@tiscali.co.uk
I ran across a problem mentioned in an old thread:

Marco De Paoli <depaolim@gm...> - 2006-12-02 11:15 wrote

>I confirm there is a problem with boost::spirit::impl::string_lit on
>VC2003
>
>if i substitute every occurrence of:
>impl::string_lit<CharT>::get("somestring")
>with:
>L"somestring"
>
>everything works fine

and tracked it down to this code:


            size_t len = strlen(source);
            BOOST_ASSERT(len < sizeof(result)/sizeof(result[0]));

            std::use_facet<ctype_t>(std::locale())
                .widen(source, source + len, result);

(tree_to_xml.ipp, ll.62-66)

I think that the end of string character ('\0') is not being widened
here... ie, you need to use

 source + len + 1
 
not

 source + len
 
. That seems to fix the infinite loops, although there are still some
glitches.

(This is my first post to a sourceforge list... I hope I've followed
the netiquette properly! Apologies if not.)

Best wishes,
Mohan





___________________________________________________________

Tiscali Broadband only £9.99 a month for your first 3 months! http://www.tiscali.co.uk/products/broadband/

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Spirit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-devel
Reply | Threaded
Open this post in threaded view
|

Re: tree_to_xml with Unicode

mohan0@tiscali.co.uk
>That seems to fix the infinite loops, although there are still some
>glitches.

Said glitches all stem from the same function:

        static wchar_t const *to_wchar_t(char const* source)
        {
            typedef std::ctype<wchar_t> ctype_t;
            static wchar_t result[64];
           
            using namespace std;        // some systems have size_t in
ns std
            size_t len = strlen(source);
            BOOST_ASSERT(len < sizeof(result)/sizeof(result[0]));

            std::use_facet<ctype_t>(std::locale())
                .widen(source, source + len, result);

            return result;
        }

The problem is here:

            static wchar_t result[64];

Using a static array to store the return string is not the safest of
operations! Every time this function is called, the strings returned
from all previous calls are invalidated...

To check that the error is as described, convert result to a
dynamically allocated array. (This will of course cause a memory
leak... for the actual fix, I would suggest returning a wstring rather
than a C-style string.)

Best wishes,
Mohan





___________________________________________________________

Tiscali Broadband only £9.99 a month for your first 3 months! http://www.tiscali.co.uk/products/broadband/

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Spirit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-devel
Reply | Threaded
Open this post in threaded view
|

Re: tree_to_xml with Unicode

Joel de Guzman-2
[from spirit.devel]

[hidden email] wrote:

>> That seems to fix the infinite loops, although there are still some
>> glitches.
>
> Said glitches all stem from the same function:
>
>         static wchar_t const *to_wchar_t(char const* source)
>         {
>             typedef std::ctype<wchar_t> ctype_t;
>             static wchar_t result[64];
>            
>             using namespace std;        // some systems have size_t in
> ns std
>             size_t len = strlen(source);
>             BOOST_ASSERT(len < sizeof(result)/sizeof(result[0]));
>
>             std::use_facet<ctype_t>(std::locale())
>                 .widen(source, source + len, result);
>
>             return result;
>         }
>
> The problem is here:
>
>             static wchar_t result[64];
>
> Using a static array to store the return string is not the safest of
> operations! Every time this function is called, the strings returned
> from all previous calls are invalidated...
>
> To check that the error is as described, convert result to a
> dynamically allocated array. (This will of course cause a memory
> leak... for the actual fix, I would suggest returning a wstring rather
> than a C-style string.)

This code is indeed odd looking! Hartmut?

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Spirit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-devel
Reply | Threaded
Open this post in threaded view
|

Re: tree_to_xml with Unicode

Hartmut Kaiser
In reply to this post by mohan0@tiscali.co.uk
[hidden email] wrote:

> Using a static array to store the return string is not the
> safest of operations! Every time this function is called, the
> strings returned from all previous calls are invalidated...
>
> To check that the error is as described, convert result to a
> dynamically allocated array. (This will of course cause a
> memory leak... for the actual fix, I would suggest returning
> a wstring rather than a C-style string.)

It's fixed in Boost CVS::HEAD now.

Thanks!
Regards Hartmut


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Spirit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-devel
Reply | Threaded
Open this post in threaded view
|

Re: tree_to_xml with Unicode

Joel de Guzman-2
Hartmut Kaiser wrote:

> [hidden email] wrote:
>
>> Using a static array to store the return string is not the
>> safest of operations! Every time this function is called, the
>> strings returned from all previous calls are invalidated...
>>
>> To check that the error is as described, convert result to a
>> dynamically allocated array. (This will of course cause a
>> memory leak... for the actual fix, I would suggest returning
>> a wstring rather than a C-style string.)
>
> It's fixed in Boost CVS::HEAD now.

Thanks! Hartmut :)

Regards,
--
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Spirit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-devel