[review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
70 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

[review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
Boost formal review of Vinnie Falco and Krystian Stasiowski's library JSON
starts today and will run for 10 days ending on 23 Sept 2020. Both of these
authors have already developed a couple of libraries which are accepted in
Boost(boost beast and Static String)

This library focuses on a common and popular use-case for JSON. It provides
a container to hold parsed and serialised JSON types. It provides more
flexibility and better benchmark performance than its competitors.

JSON highlights the following features in the documentation:

   - Fast compilation
   - Require only C++11
   - Fast streaming parser and serializer
   - Easy and safe API with allocator support
   - Constant-time key lookup for objects
   - Options to allow non-standard JSON
   - Compile without Boost, define BOOST_JSON_STANDALONE
   - Optional header-only, without linking to a library

(a point I would like to add in highlight: it has cool Jason logo 😝)


To quickly understand and get the flavour of the library take a look at
"Quick Look"
<http://master.json.cpp.al/json/usage/quick_look.html>

You can find the source code to be reviewed here:
<https://github.com/CPPAlliance/json/tree/master>

You can find the latest documentation here:
<http://master.json.cpp.al/>

Benchmarks are also given in the document which can be found here:
<http://master.json.cpp.al/json/benchmarks.html>

Some people have also given the early reviews, the thread can be found here:
<https://lists.boost.org/Archives/boost/2020/09/249745.php>



Please provide in your review information you think is valuable to
understand your choice to ACCEPT or REJECT including JSON as a
Boost library. Please be explicit about your decision (ACCEPT or REJECT).

Some other questions you might want to consider answering:

  - What is your evaluation of the design?
  - What is your evaluation of the implementation?
  - What is your evaluation of the documentation?
  - What is your evaluation of the potential usefulness of the library?
  - Did you try to use the library? With which compiler(s)? Did you have
any problems?
  - How much effort did you put into your evaluation? A glance? A quick
reading? In-depth study?
  - Are you knowledgeable about the problem domain?

More information about the Boost Formal Review Process can be found
here: <http://www.boost.org/community/reviews.html>

Thank you for your effort in the Boost community.

--
Thank you,
Pranam Lashkari, https://lpranam.github.io/

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
Because of my direct involvement in the C++ Alliance I feel it would be
wrong for me to provide a review that leads to an accept/reject conclusion.

However, I have some experience of integrating this library into a private
project and I felt it might be valuable to share my experiences.

*  - What is your evaluation of the design?*

My personal opinion is that the design is sane and well-reasoned. Any areas
with which I have previously taken issue have been raised with the authors
and concerns covered. Some effort was made to explore the effect of ideas I
presented and outcomes were measured. My opinion is that the final design
is largely data-driven.

*  - What is your evaluation of the implementation?*

I have found no faults in the implementation during use. There is the
slightly off-putting fact that the default text representation of parsing
integers that are complete powers of 10 are expressed in scientific
notation. Unusual as it seems however, this is strictly conformant with the
JSON standard.

*   - What is your evaluation of the documentation?*

The documentation is clear and succinct, the fact that it takes steps to
elucidate the rationale behind design decisions ought to head off a number
of "Wait! Why?" questions.

*   - What is your evaluation of the potential usefulness of the library?*

The library has already proven useful to me.

For me personally, the ability to map the parser directly to C++ objects
without going through the intermediate json::value data structure would
offer a minor improvement in performance.

I have started exploring the building of such a parse handler which I
intend to offer as something to go into the examples section at some future
date assuming I have the time to finish it. Notwithstanding, the fact that
I can supply a custom area-style memory resource to the parser/value
largely offsets this concern in practice. Essentially by voiding the
building of the DOM I can avoid one memory allocation and some redundant
copies. In practice, neither one memory allocation nor the memory copies
have proven measurably expensive in my uses of the library.

Whether this ultimately belongs in the JSON library or should be a
dependent library is not for me to say.

It is worth noting that the separation of concerns between parser and
handler is helpful in that it makes this work possible without having to
rewrite any parsing code.

*  - Did you try to use the library? With which compiler(s)? Did you have
any problems?*

I have used the library with GCC 9&10, and Clang 9&10. Standards selected
were C++17 and C++20. I chose the boost-dependent (default) option rather
than standalone because I was also using the boost libraries Asio, Beast,
Program Options and System.

*   - How much effort did you put into your evaluation? A glance? A quick
reading? In-depth study?*

I have written an application that uses the library: A cryptocurrency
market-making bot that faced off to the Deribit websocket/json API.

*   - Are you knowledgeable about the problem domain?*

Yes. In a previous market data distribution engine I used Nlohmann JSON
(high level but slow), RapidJSON (low level but fast) and JSMN (super low
level and blindingly fast but no DOM representation, only provides indexes
into data).

Regards,

R

On Mon, 14 Sep 2020 at 09:30, Pranam Lashkari via Boost <
[hidden email]> wrote:

> Boost formal review of Vinnie Falco and Krystian Stasiowski's library JSON
> starts today and will run for 10 days ending on 23 Sept 2020. Both of these
> authors have already developed a couple of libraries which are accepted in
> Boost(boost beast and Static String)
>
> This library focuses on a common and popular use-case for JSON. It provides
> a container to hold parsed and serialised JSON types. It provides more
> flexibility and better benchmark performance than its competitors.
>
> JSON highlights the following features in the documentation:
>
>    - Fast compilation
>    - Require only C++11
>    - Fast streaming parser and serializer
>    - Easy and safe API with allocator support
>    - Constant-time key lookup for objects
>    - Options to allow non-standard JSON
>    - Compile without Boost, define BOOST_JSON_STANDALONE
>    - Optional header-only, without linking to a library
>
> (a point I would like to add in highlight: it has cool Jason logo 😝)
>
>
> To quickly understand and get the flavour of the library take a look at
> "Quick Look"
> <http://master.json.cpp.al/json/usage/quick_look.html>
>
> You can find the source code to be reviewed here:
> <https://github.com/CPPAlliance/json/tree/master>
>
> You can find the latest documentation here:
> <http://master.json.cpp.al/>
>
> Benchmarks are also given in the document which can be found here:
> <http://master.json.cpp.al/json/benchmarks.html>
>
> Some people have also given the early reviews, the thread can be found
> here:
> <https://lists.boost.org/Archives/boost/2020/09/249745.php>
>
>
>
> Please provide in your review information you think is valuable to
> understand your choice to ACCEPT or REJECT including JSON as a
> Boost library. Please be explicit about your decision (ACCEPT or REJECT).
>
> Some other questions you might want to consider answering:
>
>   - What is your evaluation of the design?
>   - What is your evaluation of the implementation?
>   - What is your evaluation of the documentation?
>   - What is your evaluation of the potential usefulness of the library?
>   - Did you try to use the library? With which compiler(s)? Did you have
> any problems?
>   - How much effort did you put into your evaluation? A glance? A quick
> reading? In-depth study?
>   - Are you knowledgeable about the problem domain?
>
> More information about the Boost Formal Review Process can be found
> here: <http://www.boost.org/community/reviews.html>
>
> Thank you for your effort in the Boost community.
>
> --
> Thank you,
> Pranam Lashkari, https://lpranam.github.io/
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>


--
Richard Hodges
[hidden email]
office: +442032898513
home: +376841522
mobile: +376380212

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Mon, Sep 14, 2020 at 12:30 AM Pranam Lashkari via Boost <
[hidden email]> wrote:

> <https://github.com/CPPAlliance/json
> <https://github.com/CPPAlliance/json/tree/master>>
>

Don't forget to STAR the repository, and thanks!

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
On Mon, Sep 14, 2020 at 8:01 PM Vinnie Falco <[hidden email]> wrote:

>
> Don't forget to STAR the repository, and thanks!
>

Yes, please start the amazing library it may help to reach this library to
more people.


--
Thank you,
Pranam Lashkari, https://lpranam.github.io/

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 14.09.20 09:30, Pranam Lashkari via Boost wrote:
> Please provide in your review information you think is valuable to
> understand your choice to ACCEPT or REJECT including JSON as a
> Boost library. Please be explicit about your decision (ACCEPT or REJECT).
>
>    - What is your evaluation of the design?

Most of it seems fine, but I do have some issues:

   - The choice of [u]int64_t seems arbitrary and restrictive.  It means
that Boost.JSON will not use a 128 bit integer even where one is
available, and it means that it cannot compile at all on implementations
that don't provide int64_t.  It's good enough for my purposes, but I
would like to see some discussion about this.  The json spec allows
arbitrarily large integers.

   - On a similar note, the use of double restricts floating point
accuracy even when a higher-precision type is available.  The json spec
allows arbitrarily precise decimal values.

   - boost::json::value_to provides a single, clean way to extract
values from json, but it also renders other parts of the library (e.g.
number_cast) redundant except as an implementation detail.

   - boost::json::value_to provides a single, clean way to extract
values from json, but it's syntactically long.  The same functionality
in a member function of boost::json::value would be nicer to use.

   - The omission of binary serialization formats (CBOR et al) bothers
me.  Not from a theoretical point of view, but because I have actual
code that uses CBOR, and I won't be able to convert this code to
Boost.JSON unless CBOR support is provided.  (Or I could write my own,
but that rather defeats the point of using a library in the first place,
especially if Neils Lohmann's library already provides CBOR support.)


>    - What is your evaluation of the implementation?

I didn't look at it.

>    - What is your evaluation of the documentation?

boost::json::value_to provides a single, clean way to extract values
from json, but it's actually rather hard to find in the documentation.
I was looking for a way to extract a std::string from a
boost::json::value, so I looked at the documentation for
boost::json::value and found as_string.  OK, that returns
boost::json::string, which is not implicitly convertible to std::string
(in C++14).  But it has an implicit conversion to string_view, which is
an alias of boost::string_view, which doesn't appear to be documented
anywhere but which (looking at the source code) has a member function
to_string.  So I ended up with this code:

   boost::json::value v = "Hello world.";
   std::string s
     = static_cast<boost::json::string_view>(v.as_string()).to_string();

...which I think we can all agree is an abomination.  /Then/ I found out
about boost::json::value_to, and replaced my code with this:

   std::string s = boost::json::value_to<std::string>(v);

Which is definitely nicer, though still not as nice as Niels Lohmann's code:

   std::string s = v.get<std::string>();

boost::json::value_to or its member function replacement should
definitely be front and center to the documentation.

>    - What is your evaluation of the potential usefulness of the library?

A good json library is very useful for a large number of programs.  The
questions isn't if Boost should have a json library, but if this is the
json library that Boost should have.

>    - Did you try to use the library? With which compiler(s)? Did you have
> any problems?

I converted a small but non-trivial program from Lohmann's json to the
proposed Boost.JSON, and compiled it with several different
cross-compilers.  I did not encounter any problems compiling or running
the program, although I did have to add Boost.Container to the set of
linked libraries.

>    - How much effort did you put into your evaluation? A glance? A quick
> reading? In-depth study?

A few hours of work, most of which was spent converting code from
Lohmann's json library to the proposed Boost.JSON.

>    - Are you knowledgeable about the problem domain?

Yes.

 > Please be explicit about your decision (ACCEPT or REJECT).

For me, the ultimate question is if I would actually use this library,
and my reluctant answer is "not its current state".  I'm basically
satisfied with Lohmann's json library, which requires less verbosity to
use and which provides CBOR support.  I can see the attraction of
Boost.JSON's superior performance, and the attraction of incremental
parsing and serialization, but for my usage none of this matters.  CBOR
support, on the other hand does matter.

I vote to CONDITIONALLY ACCEPT Boost.JSON, conditional on the inclusion
of code for parsing and serializing boost::json::value as CBOR.  I can
live with the added verbosity of Boost.JSON (although I'd rather see it
reduced where possible), but not without CBOR.

(If CBOR support is not added, this vote should count as an abstain vote
and not as a reject.  I don't think that Boost.JSON needs CBOR in order
to be useful to other people - I just don't want to vote to accept a
library that's not useful for me.)


--
Rainer Deyke ([hidden email])


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
On Tue, Sep 15, 2020 at 7:14 AM Rainer Deyke via Boost
<[hidden email]> wrote:
> I did have to add Boost.Container to the set of linked libraries.

Boost.Container develop branch has a fix for this, so linking with
that library will not be necessary if Boost.JSON is released with
Boost:
<https://github.com/boostorg/container/commit/0b297019ec43483f523a3270b632fecbc3ce5e63>

Thank you for your thoughtful review!

Regards

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Tue, Sep 15, 2020 at 7:14 AM Rainer Deyke via Boost
<[hidden email]> wrote:

>    - The choice of [u]int64_t seems arbitrary and restrictive.  It means
> that Boost.JSON will not use a 128 bit integer even where one is
> available, and it means that it cannot compile at all on implementations
> that don't provide int64_t.  It's good enough for my purposes, but I
> would like to see some discussion about this.  The json spec allows
> arbitrarily large integers.
>
>    - On a similar note, the use of double restricts floating point
> accuracy even when a higher-precision type is available.  The json spec
> allows arbitrarily precise decimal values.

The spec gives implementations freedom to place arbitrary upper limits
on precision. To be useful as a vocabulary type, the library prefers
homogeneity of interface over min/maxing. In other words more value is
placed on having the library use the same integer representation on
all platforms than using the largest integer width available. Another
point is that the sizes of types in the library is very tightly
controlled. `sizeof(value)` is 16 bytes on 32-bit platforms and 24
bytes on 64-bit platforms, and this is for a reason. It is to keep
performance high and memory consumption low. There is a direct, linear
falloff in general performance with increasing size of types.

>    - boost::json::value_to provides a single, clean way to extract
> values from json, but it also renders other parts of the library (e.g.
> number_cast) redundant except as an implementation detail.

Well, number_cast isn't redundant since it is the only interface which
offers the use of error codes rather than exceptions. We could have
gone with error codes in conversion to user-defined types but then the
interface would need some kind of expected<> return type and things
get messy there. Exceptions are a natural error handling mechanism.
However we recognize that in network programs there is a need to
convert numbers without using exceptions, thus number_cast is
available.

>    - boost::json::value_to provides a single, clean way to extract
> values from json, but it's syntactically long.  The same functionality
> in a member function of boost::json::value would be nicer to use.

Algorithms which can be implemented completely in terms of a class'
public interface are generally expressed as free functions in separate
header files. If we were to make `get` a member function of
`json::value`, then users who have no need to convert to and from
user-defined types would be unnecessarily including code they never
use.

Thanks

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list

>>     - boost::json::value_to provides a single, clean way to extract
>> values from json, but it's syntactically long.  The same functionality
>> in a member function of boost::json::value would be nicer to use.
> Algorithms which can be implemented completely in terms of a class'
> public interface are generally expressed as free functions in separate
> header files. If we were to make `get` a member function of
> `json::value`, then users who have no need to convert to and from
> user-defined types would be unnecessarily including code they never
> use.

To add to that: You can use ADL to avoid naming the namespace:
`value_to<std::string>(json_val)` which is not much longer than:
`json_val.get<std::string>()`
I could have been named it `get_as` though:
`get_as<std::string>(json_val)` but one is as good as the other




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Tue, Sep 15, 2020 at 7:14 AM Rainer Deyke via Boost
<[hidden email]> wrote:
> I can see the attraction of
> Boost.JSON's superior performance, and the attraction of incremental
> parsing and serialization, but for my usage none of this matters.  CBOR
> support, on the other hand does matter.

We are researching the topic. If you would like to weigh in, the issue
can be tracked here:
<https://github.com/CPPAlliance/json/issues/342>

One thing I will note, however. A Google search for "CBOR" produces
390,000 results while a Google search for JSON produces 188,000,000
results. Now there is surely some margin of error in these numbers but
I have to wonder how widespread is the use of CBOR.

Thanks

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Rainer Deyke wrote:

> I vote to CONDITIONALLY ACCEPT Boost.JSON, conditional on the inclusion of
> code for parsing and serializing boost::json::value as CBOR.

I find this condition is too strict. You basically say that you'd rather not
see the proposed Boost.JSON enter Boost until CBOR is implemented, which may
happen six months from now. So people who don't have a need for CBOR will
have to wait until Boost 1.77, which doesn't really help anyone.

I can understand requiring a firm commitment on the part of the authors to
add CBOR support, but postponing the acceptance until this is implemented...


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
On 15.09.20 16:45, Peter Dimov via Boost wrote:

> Rainer Deyke wrote:
>
>> I vote to CONDITIONALLY ACCEPT Boost.JSON, conditional on the
>> inclusion of code for parsing and serializing boost::json::value as CBOR.
>
> I find this condition is too strict. You basically say that you'd rather
> not see the proposed Boost.JSON enter Boost until CBOR is implemented,
> which may happen six months from now. So people who don't have a need
> for CBOR will have to wait until Boost 1.77, which doesn't really help
> anyone.

Actually I would like to see Boost.JSON in Boost, with or without CBOR.
However, I can't in good conscience vote to accept a library that I am
unwilling to use.  Boost should not be a graveyard of well-designed but
unused libraries.

As I explained in my review, if my condition is not met, I would like my
vote to be counted as ABSTAIN, not REJECT.  I am hoping that Boost.JSON
will get enough votes to accept to make it into Boost, with or without
CBOR support.

> I can understand requiring a firm commitment on the part of the authors
> to add CBOR support, but postponing the acceptance until this is
> implemented...

If such a commitment is made, I would also consider my condition for
acceptance met.


--
Rainer Deyke ([hidden email])


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 15.09.20 16:44, Vinnie Falco via Boost wrote:

> On Tue, Sep 15, 2020 at 7:14 AM Rainer Deyke via Boost
> <[hidden email]> wrote:
>> I can see the attraction of
>> Boost.JSON's superior performance, and the attraction of incremental
>> parsing and serialization, but for my usage none of this matters.  CBOR
>> support, on the other hand does matter.
>
> We are researching the topic. If you would like to weigh in, the issue
> can be tracked here:
> <https://github.com/CPPAlliance/json/issues/342>
>
> One thing I will note, however. A Google search for "CBOR" produces
> 390,000 results while a Google search for JSON produces 188,000,000
> results. Now there is surely some margin of error in these numbers but
> I have to wonder how widespread is the use of CBOR.

That's a valid point, but I would counter that most applications that
handle large amounts of JSON would benefit from using CBOR, and it's
often lack of library support that's holding them back.


--
Rainer Deyke ([hidden email])


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Rainer Deyke wrote:

> I vote to CONDITIONALLY ACCEPT Boost.JSON, conditional on the inclusion of
> code for parsing and serializing boost::json::value as CBOR.

The one interesting decision that needs to be made here is how to handle
CBOR byte strings (major type 3), as they aren't representable in JSON or in
the current boost::json::value.

I see that Niels Lohmann has added a "binary" 'kind' to the json value for
this purpose. Which would then invite the opposite question, what's a JSON
serializer supposed to do with kind_binary.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 16/09/2020 08:31, Rainer Deyke via Boost wrote:

> On 15.09.20 16:45, Peter Dimov via Boost wrote:
>> Rainer Deyke wrote:
>>
>>> I vote to CONDITIONALLY ACCEPT Boost.JSON, conditional on the
>>> inclusion of code for parsing and serializing boost::json::value as
>>> CBOR.
>>
>> I find this condition is too strict. You basically say that you'd
>> rather not see the proposed Boost.JSON enter Boost until CBOR is
>> implemented, which may happen six months from now. So people who don't
>> have a need for CBOR will have to wait until Boost 1.77, which doesn't
>> really help anyone.
>
> Actually I would like to see Boost.JSON in Boost, with or without CBOR.
> However, I can't in good conscience vote to accept a library that I am
> unwilling to use.  Boost should not be a graveyard of well-designed but
> unused libraries.
>
> As I explained in my review, if my condition is not met, I would like my
> vote to be counted as ABSTAIN, not REJECT.  I am hoping that Boost.JSON
> will get enough votes to accept to make it into Boost, with or without
> CBOR support.

If the proposed library were called Boost.Serialisation2 or something, I
would see your point.

But it's called Boost.JSON. It implements JSON. It does not implement
CBOR. I don't think it's reasonable to recommend a rejection for a
library not doing something completely different to what it does.

Speaking wider that this, if the format here were not specifically JSON,
I'd also be more sympathetic - binary as well as text
serialisation/deserialisation is important. But JSON is unique, most
users would not choose JSON except that they are forced to do so by
needing to talk to other stuff which mandates JSON.

At work we have this lovely very high performance custom DB based on
LLFIO. It has rocking stats. But it's exposed to clients via a REST API,
and that means everything goes via JSON. So the DB spends most of its
time fairly idle compared to what it is capable of, because JSON is so
very very slow in comparison.

If we could choose anything but JSON, we would, but the customer spec
requires an even nastier and slower text format than JSON. We expect to
win the argument to get them to "upgrade" to JSON, but anything better
than that is years away. Change is hard for big governmental orgs.

In any case, CBOR is actually a fairly lousy binary protocol. Very
inefficient compared to alternatives. But the alternatives all would
require you to design your software differently to what JSON's very
reference count centric design demands.

Niall

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
Niall Douglas wrote:

> In any case, CBOR is actually a fairly lousy binary protocol. Very
> inefficient compared to alternatives.

I actually quite like CBOR. Of all the "binary JSON" protocols, it's the
best. Or the least worst, if you will. (Except for MessagePack, which I like
even more, but it's arguably not a binary JSON.)


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Wed, Sep 16, 2020 at 6:33 AM Peter Dimov via Boost
<[hidden email]> wrote:
> The one interesting decision that needs to be made here is how to handle
> CBOR byte strings (major type 3), as they aren't representable in JSON or in
> the current boost::json::value.
>
> I see that Niels Lohmann has added a "binary" 'kind' to the json value for
> this purpose. Which would then invite the opposite question, what's a JSON
> serializer supposed to do with kind_binary.

If I make a library that has a public function which accepts a
parameter of type json::value, I don't expect to see binary objects in
it nor would I want to have to write code to handle something that is
not part of JSON. And I expect that things round-trip correctly, i.e.
if I serialize a json::value and then parse it, I get back the same
result. This is clearly impossible if the json::value contains a
"binary" string.

If CBOR was just another serialization format, then I might lean
towards implementing it. But CBOR is sounding more and more like it is
Not JSON and thus out-of-scope for Boost.JSON.

Thanks

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
Vinnie Falco wrote:

> If I make a library that has a public function which accepts a parameter
> of type json::value, I don't expect to see binary objects in it nor would
> I want to have to write code to handle something that is not part of JSON.
> And I expect that things round-trip correctly, i.e. if I serialize a
> json::value and then parse it, I get back the same result. This is clearly
> impossible if the json::value contains a "binary" string.

"Binary string" is basically JSON string with UTF-8 validation turned off;
it's a common thing to want to send/receive and the binary formats are
arguably correct in offering it as a specific type. Of course this doesn't
change the fact that it's not representable in standard JSON.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Tue, Sep 15, 2020 at 7:14 AM Rainer Deyke via Boost
<[hidden email]> wrote:
>    - The omission of binary serialization formats (CBOR et al) bothers
> me.  Not from a theoretical point of view, but because I have actual
> code that uses CBOR, and I won't be able to convert this code to
> Boost.JSON unless CBOR support is provided.

I've looked at the CBOR specification and some implementations in the
wild and these points stick out:

1. CBOR supports extensions, which cannot be represented in boost::json::value
2. CBOR also supports "binary" strings, which also cannot be
represented in boost::json::value
3. If boost.json's value container could hold these things, then it
would no longer serialize to standard JSON

Therefore, it seems to me that CBOR is not just a "binary
serialization format for JSON." It is in fact a completely different
format that only strongly resembles JSON. Or perhaps you could say it
is a superset of JSON. I think the best way to support this is as
follows:

1. Fork the Boost.JSON repository, rename it to Boost.CBOR
2. Add support for binary strings to the cbor::value type
3. Add support for extensions to the cbor::value type
4. Replace the parse, parser, serialize, and serializer interfaces
with CBOR equivalents
5. Propose this library as a new Boost library, with a separate review process

Then, we would have a first-class CBOR library whose interface and
implementation are optimized specifically for CBOR. Questions such as
what happens when you serialize a cbor::value to JSON would be moot.

This could be something that Krystian might take on as author and maintainer.

Thanks

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 16.09.20 15:33, Niall Douglas via Boost wrote:

> On 16/09/2020 08:31, Rainer Deyke via Boost wrote:
>> On 15.09.20 16:45, Peter Dimov via Boost wrote:
>>> Rainer Deyke wrote:
>>>
>>>> I vote to CONDITIONALLY ACCEPT Boost.JSON, conditional on the
>>>> inclusion of code for parsing and serializing boost::json::value as
>>>> CBOR.
>>>
>>> I find this condition is too strict. You basically say that you'd
>>> rather not see the proposed Boost.JSON enter Boost until CBOR is
>>> implemented, which may happen six months from now. So people who don't
>>> have a need for CBOR will have to wait until Boost 1.77, which doesn't
>>> really help anyone.
>>
>> Actually I would like to see Boost.JSON in Boost, with or without CBOR.
>> However, I can't in good conscience vote to accept a library that I am
>> unwilling to use.  Boost should not be a graveyard of well-designed but
>> unused libraries.
>>
>> As I explained in my review, if my condition is not met, I would like my
>> vote to be counted as ABSTAIN, not REJECT.  I am hoping that Boost.JSON
>> will get enough votes to accept to make it into Boost, with or without
>> CBOR support.
>
> If the proposed library were called Boost.Serialisation2 or something, I
> would see your point.
>
> But it's called Boost.JSON. It implements JSON. It does not implement
> CBOR. I don't think it's reasonable to recommend a rejection for a
> library not doing something completely different to what it does.

I see CBOR not as a separate format, but as an encoding for JSON (with
some additional features that can safely be ignored).  I use it to store
and transmit JSON data, and would not use it for anything else.

JSON data exists independently of the JSON serialization format.  This
is in fact a core principle of Boost.JSON: the data representation
exists independently of the serialization functions.

> Speaking wider that this, if the format here were not specifically JSON,
> I'd also be more sympathetic - binary as well as text
> serialisation/deserialisation is important. But JSON is unique, most
> users would not choose JSON except that they are forced to do so by
> needing to talk to other stuff which mandates JSON.
>
> At work we have this lovely very high performance custom DB based on
> LLFIO. It has rocking stats. But it's exposed to clients via a REST API,
> and that means everything goes via JSON. So the DB spends most of its
> time fairly idle compared to what it is capable of, because JSON is so
> very very slow in comparison.

This is exactly the sort of problem that CBOR excels at.  The server
produces JSON.  The client consumes JSON.  Flip a switch, and the server
produces CBOR instead.  Ideally the client doesn't have to be changed at
all.  One line of code changed in the server, and suddenly you have
twice the data throughput.

> In any case, CBOR is actually a fairly lousy binary protocol. Very
> inefficient compared to alternatives. But the alternatives all would
> require you to design your software differently to what JSON's very
> reference count centric design demands.

It may be lousy as a general-purpose binary protocol, but it's a fairly
good binary JSON representation.  Which is why it belongs in a JSON
library if it belongs anywhere.


--
Rainer Deyke ([hidden email])


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review][JSON] Review of JSON starts today: Sept 14 - Sept 23

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 16/09/2020 14:53, Peter Dimov via Boost wrote:
> Niall Douglas wrote:
>
>> In any case, CBOR is actually a fairly lousy binary protocol. Very
>> inefficient compared to alternatives.
>
> I actually quite like CBOR. Of all the "binary JSON" protocols, it's the
> best. Or the least worst, if you will. (Except for MessagePack, which I
> like even more, but it's arguably not a binary JSON.)

I know what you're saying. However, comparing **C++ implementations** of
CBOR to JSON ones does not yield much differential. For example,
simdjson will happily sustain 10 Gbit of textual JSON parsing per
session which is enough for most NICs. CBOR parsers, at least the ones
available to C++, are no better than this.

Our custom DB will push 20-25Gb/sec, so we'd need a 250 Gbit NIC and a
zero copy all binary network protocol to have the DB become the primary
bottleneck. I doubt any CBOR like design could achieve that, ever,
because that design is fundamentally anti-performance.

CBOR's primary gain for me is exact bit for bit value transfer, so
floating point numbers come out exactly as you sent them. That's rarely
needed outside scientific niches though, and even then, just send the FP
number as a string encoded in hexadecimal in JSON right?

In fact, for any format which looks better than JSON, encoding your
values as hexadecimal strings in JSON is an excellent workaround.
Hexadecimal string parsing is very, very fast on modern CPUs, you often
can add +20% to JSON bottlenecked performance by using hexadecimal for
everything.

Niall

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
1234