[Nowide] Opinion on slightly expanding the scope: LFS and more generic UTF-8 wide<->narrow conversion

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[Nowide] Opinion on slightly expanding the scope: LFS and more generic UTF-8 wide<->narrow conversion

Boost - Dev mailing list
Hi all,

just wanted to gather opinions on 2 changes I'd like to make to
Boost.Nowide which go slightly above the initial goal:

1. Make detail implementations of widen/narrow public.
     - Mainly
https://github.com/boostorg/nowide/blob/develop/include/boost/nowide/detail/convert.hpp 
which does templated conversion from/to wide strings which might be
useful in generic contexts
     - But also the underlying UTF decoding/encoding templates initially
extracted from Boost.Locale
https://github.com/boostorg/nowide/blob/develop/include/boost/nowide/detail/utf.hpp
     I figured (and experienced) that those might be useful for
consumers of Nowide as when they already use those UTF-8 APIs they might
have other interface code which needs translation from/to wide strings,
so exposing the existing code as a feature might make sense.
     Basically this would put them under boost::nowide::utf namespace
instead of in the detail namespace. I guess I don't need to retain the
old ones or put in forwarding headers, do I?
2. As requested and implemented over 2 PRs (last one still open:
https://github.com/boostorg/nowide/pull/104) Large files are now
supported by boost::nowide::filebuf (technically only used on Windows)
but the implementation is pretty portable as e.g. Tests on Unix do use
boost::nowide::filebuf. This is done by implementing ftell/fseek via
_ftelli64/ftello and configure-time checking availability.
    As C++ is lacking C-Style file manipulation functions portably
working with LFS I think a similar reasoning as above can be applied: It
exists, it works and can be useful more generically so moving it out of
the detail namespace might make sense.

Opinions? Ideas?

Regards, Alex

PS: If anyone with filesystem experience would like to review #104 or at
least glance over it, it would be appreciated.




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Nowide] Opinion on slightly expanding the scope: LFS and more generic UTF-8 wide<->narrow conversion

Boost - Dev mailing list
As a Windows user, I find this library less and less attractive. W10 is now
optionally fully Unicode/utf8, there is no reason to have anything else
than https://github.com/degski/Sax/blob/master/include/sax/utf8conv.hpp or
something like it and do perimeter conversions with these two functions,
specifically for windows. `std::filesystem` (and other bits of the STL)
does the right thing.  The original author worked at Microsoft at the time
and the code was published on the Windows blog, i.e.this is how it is done.

On Wed, 10 Jun 2020 at 02:24, Alexander Grund via Boost <
[hidden email]> wrote:

> Hi all,
>
> just wanted to gather opinions on 2 changes I'd like to make to
> Boost.Nowide which go slightly above the initial goal:
>
> 1. Make detail implementations of widen/narrow public.
>      - Mainly
>
> https://github.com/boostorg/nowide/blob/develop/include/boost/nowide/detail/convert.hpp
> which does templated conversion from/to wide strings which might be
> useful in generic contexts
>      - But also the underlying UTF decoding/encoding templates initially
> extracted from Boost.Locale
>
> https://github.com/boostorg/nowide/blob/develop/include/boost/nowide/detail/utf.hpp
>      I figured (and experienced) that those might be useful for
> consumers of Nowide as when they already use those UTF-8 APIs they might
> have other interface code which needs translation from/to wide strings,
> so exposing the existing code as a feature might make sense.
>      Basically this would put them under boost::nowide::utf namespace
> instead of in the detail namespace. I guess I don't need to retain the
> old ones or put in forwarding headers, do I?
> 2. As requested and implemented over 2 PRs (last one still open:
> https://github.com/boostorg/nowide/pull/104) Large files are now
> supported by boost::nowide::filebuf (technically only used on Windows)
> but the implementation is pretty portable as e.g. Tests on Unix do use
> boost::nowide::filebuf. This is done by implementing ftell/fseek via
> _ftelli64/ftello and configure-time checking availability.
>     As C++ is lacking C-Style file manipulation functions portably
> working with LFS I think a similar reasoning as above can be applied: It
> exists, it works and can be useful more generically so moving it out of
> the detail namespace might make sense.
>
> Opinions? Ideas?
>
> Regards, Alex
>
> PS: If anyone with filesystem experience would like to review #104 or at
> least glance over it, it would be appreciated.
>
>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [Nowide] Opinion on slightly expanding the scope: LFS and more generic UTF-8 wide<->narrow conversion

Boost - Dev mailing list
Am 11.06.20 um 19:15 schrieb degski:
> As a Windows user, I find this library less and less attractive. W10
> is now optionally fully Unicode/utf8, there is no reason to have
> anything else than
> https://github.com/degski/Sax/blob/master/include/sax/utf8conv.hpp or
> something like it and do perimeter conversions with these two
> functions, specifically for windows. `std::filesystem` (and other bits
> of the STL) does the right thing.  The original author worked at
> Microsoft at the time and the code was published on the Windows blog,
> i.e.this is how it is done.

Thanks for the input. On that:
- The "optional" UTF-8 (via Manifest) is only available in recent enough
Windows versions. Our company Windows does not support it (It seems to
be some kind of LTS release)
- The code uses the WinAPI for the conversion with exceptions on errors
while Nowide uses replacements. I guess it's choose your poison
- That conversion code requires C++17, Boost.Nowide provides support for
Boost.Filesystem and std::filesystem without requiring C++17

I do wish that a day comes without needing the wide API at all anymore
but this day is not today.
Until then: If you can require recent enough Windows versions go for the
Manifest. For the others there is Boost.Nowide.

PS: sax looks like a cool collection of utils, well done!




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Nowide] Opinion on slightly expanding the scope: LFS and more generic UTF-8 wide<->narrow conversion

Boost - Dev mailing list
On Fri, 12 Jun 2020 at 02:37, Alexander Grund <[hidden email]>
wrote:

> Am 11.06.20 um 19:15 schrieb degski:
> > As a Windows user, I find this library less and less attractive. W10
> > is now optionally fully Unicode/utf8, there is no reason to have
> > anything else than
> > https://github.com/degski/Sax/blob/master/include/sax/utf8conv.hpp or
> > something like it and do perimeter conversions with these two
> > functions, specifically for windows. `std::filesystem` (and other bits
> > of the STL) does the right thing.  The original author worked at
> > Microsoft at the time and the code was published on the Windows blog,
> > i.e.this is how it is done.
>
> Thanks for the input. On that:
> - The "optional" UTF-8 (via Manifest) is only available in recent enough
> Windows versions. Our company Windows does not support it (It seems to
> be some kind of LTS release)
> - The code uses the WinAPI for the conversion with exceptions on errors
> while Nowide uses replacements. I guess it's choose your poison
>

Yeah, that's it. And you're right, I can (and many not) fully control my
system.

PS: sax looks like a cool collection of utils, well done!
>

Thank you.

degski

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost