What happened to Boost.Nowide?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

What happened to Boost.Nowide?

Boost - Dev mailing list
Hi,

I'm a user of (a fork of) Boost.Nowide and already fixed some issues I
found and was looking into getting them upstream.

I also wanted to know, if it is finally in Boost. Unfortunately this
does not seem to be the case. I found
https://lists.boost.org/Archives/boost//2017/06/236475.php which
accepted it into Boost. This is from mid-2017 and nothing has happened
since.

Does anyone know what the status of Boost.Nowide is? It seems the
filestream parts are now incorporated into Boost.FileSystem. So it seems
only cin/cout/cerr, the args wrapper and the C-functions (fopen, ...)
are missing. Especially the first 2 are very useful in writing
cross-platform code.

Might those be integrated into some other Boost Library?

Thanks,
Alex




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: What happened to Boost.Nowide?

Boost - Dev mailing list
On Wed, Nov 6, 2019 at 2:11 AM Alexander Grund via Boost <
[hidden email]> wrote:

> Does anyone know what the status of Boost.Nowide is? It seems the
> filestream parts are now incorporated into Boost.FileSystem. So it seems
> only cin/cout/cerr, the args wrapper and the C-functions (fopen, ...)
> are missing. Especially the first 2 are very useful in writing
> cross-platform code.
>

I don't know the status of Boost.Nowide. However, it's usefulness has
diminished with the introduction of UTF-8 codepage support in Windows 10 in
May this year. See
https://docs.microsoft.com/en-us/windows/uwp/design/globalizing/use-utf8-code-page
.

--
Yakov Galka
http://stannum.co.il/

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: What happened to Boost.Nowide?

Boost - Dev mailing list

Am 07.11.19 um 03:39 schrieb Yakov Galka:

> On Wed, Nov 6, 2019 at 2:11 AM Alexander Grund via Boost
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Does anyone know what the status of Boost.Nowide is? It seems the
>     filestream parts are now incorporated into Boost.FileSystem. So it
>     seems
>     only cin/cout/cerr, the args wrapper and the C-functions (fopen, ...)
>     are missing. Especially the first 2 are very useful in writing
>     cross-platform code.
>
>
> I don't know the status of Boost.Nowide. However, it's usefulness has
> diminished with the introduction of UTF-8 codepage support in Windows
> 10 in May this year. See
> https://docs.microsoft.com/en-us/windows/uwp/design/globalizing/use-utf8-code-page.
Interesting, thanks! Great to see that someone at MS finally made the
right decision so that Windows is no longer the only OS not supporting UTF8.

However it does require Win10 1903 minimum and a change to the
manifest(s). So maybe Nowide still has some use.

Alex



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: What happened to Boost.Nowide?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Am 07.11.19 um 03:39 schrieb Yakov Galka:
>  However, it's usefulness has diminished with the introduction of
> UTF-8 codepage support in Windows 10 in May this year. See
> https://docs.microsoft.com/en-us/windows/uwp/design/globalizing/use-utf8-code-page.

I just noticed that it is very unfortunate, that this didn't happen 3
years (or so) ago. Now not only `boost::filesystem::path` is using
`wchar` but also the C++17 `std::filesystem::path` does so. So we now
have costly conversions and wasting half the space on windows for no gain :/




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: What happened to Boost.Nowide?

Boost - Dev mailing list
On Mon, Nov 11, 2019 at 4:19 AM Alexander Grund via Boost <
[hidden email]> wrote:

> I just noticed that it is very unfortunate, that this didn't happen 3
> years (or so) ago. Now not only `boost::filesystem::path` is using
> `wchar` but also the C++17 `std::filesystem::path` does so. So we now
> have costly conversions and wasting half the space on windows for no gain
> :/
>

I raised this issue many years ago. In fact boost filesystem v2 was better
in this respect, because it followed the established convention of having a
templated basic_path<char>, thus not committing to a specific char type.
Alas, v2 was deprecated and v3 was lobbied into WG21 for standardization.
It was an unprecedented case of introducing a "char on some platforms,
wchar_t on others" interface into the standard, which is a bad decision
from portability stand point.

While we are at it, I would like to say that boost filesystem should have
never introduced a path class in the first place. filesystem::path is just
a glorified string with no extra invariants. Any string -> path conversion
copies the data, even if it's already in the right encoding, even on
operating systems that don't need any conversions at all. There goes your
"don't pay for what you don't use" principle. Most can agree that C++'s
spirit is to separate containers from algorithms. A proper design would
introduce path manipulation functions that work on any string types, and
let users use std::string or even char[] for storage.

--
Yakov Galka
http://stannum.co.il/

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: What happened to Boost.Nowide?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Mon, Nov 11, 2019 at 1:09 AM Alexander Grund via Boost <
[hidden email]> wrote:

> However it does require Win10 1903 minimum and a change to the
> manifest(s). So maybe Nowide still has some use.
>

Looks like there is a way to set UTF-8 globally for existing applications:
https://stackoverflow.com/q/56419639/277176
Though I didn't try it yet.

--
Yakov Galka
http://stannum.co.il/

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: What happened to Boost.Nowide?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 4/12/2019 05:25, Yakov Galka wrote:
> On Mon, Nov 11, 2019 at 4:19 AM Alexander Grund wrote:
> I raised this issue many years ago. In fact boost filesystem v2 was better
> in this respect, because it followed the established convention of having a
> templated basic_path<char>, thus not committing to a specific char type.
> Alas, v2 was deprecated and v3 was lobbied into WG21 for standardization.
> It was an unprecedented case of introducing a "char on some platforms,
> wchar_t on others" interface into the standard, which is a bad decision
> from portability stand point.

While I agree in principle, the simple fact is that performing string
transcoding on filesystem paths is a Very Bad Idea™, since both Windows
and Linux treat them as opaque byte sequences -- but Windows' native
encoding is UTF-16 and Linux' is (mostly) UTF-8.

So, while unfortunate, v3 made the correct choice.  Paths have to be
kept in their original encoding between original source (command line,
file, or UI) and file API usage, otherwise you can get weird errors when
transcoding produces a different byte sequence that appears identical
when actually rendered, but doesn't match the filesystem.  Transcoding
is only safe when you're going to do something with the string other
than using it in a file API.

> While we are at it, I would like to say that boost filesystem should have
> never introduced a path class in the first place. filesystem::path is just
> a glorified string with no extra invariants. Any string -> path conversion
> copies the data, even if it's already in the right encoding, even on
> operating systems that don't need any conversions at all. There goes your
> "don't pay for what you don't use" principle. Most can agree that C++'s
> spirit is to separate containers from algorithms. A proper design would
> introduce path manipulation functions that work on any string types, and
> let users use std::string or even char[] for storage.

While copying is unfortunate, these things are rarely on a
performance-critical path, and the benefits of having consistent
compose/decompose operations on paths vastly outweighs that, in my
opinion.  Combined with the need to maintain native encoding for paths,
separated algorithms don't seem particularly useful -- just less
convenient to use.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost