Crypto/hashes library?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Crypto/hashes library?

Boost - Dev mailing list
Hello,

The boost uuid module contains an implementation of the sha1 hash algorithm
for purposes of generating a uuid from arbitrary input.  I've opened an
issue on github to make the hash algorithm configurable:

https://github.com/boostorg/uuid/issues/26

I was hoping to find a concept and implementation of a few hashes somewhere
in boost that could be used instead, however I haven't found any.  Back in
2010 someone posted a link to a "boost::hashes" repository but I was unable
to see what the outcome of the discussion was other than that it was not
included.  A collection of hashing algorithms that are well tested would be
quite useful in a number of ways; and I like the hash size template style
used in the repo from 2010:

https://svn.boost.org/svn/boost/sandbox/hash/

Indeed there's mention of a discussion about the proper placement of sha1
inside boost::uuid in there:

https://svn.boost.org/svn/boost/sandbox/hash/libs/hash/doc/html/rationale.html

So I was curious where that effort ended up, and if there was a resolution
on whether an independent library for hashing was ultimately something that
folks wanted to pursue or avoid.

Thanks,

Jim

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
I think it would be reasonable to say that most people (tm) use either the
openssl or CryptoCpp libraries.

If you're using boost::asio with ssl then you may as well use openssl since
it's a dependency.

However, a uniform boost::crypto library would I think be a fantastic
addition to boost. It could be a uniform wrapper around whatever is normal
for the target (or configured at ./configure time).

It might even be the beginning of the road towards a std::crypto, which I
think is sorely needed.






On 7 September 2017 at 05:05, James E. King, III via Boost <
[hidden email]> wrote:

> Hello,
>
> The boost uuid module contains an implementation of the sha1 hash algorithm
> for purposes of generating a uuid from arbitrary input.  I've opened an
> issue on github to make the hash algorithm configurable:
>
> https://github.com/boostorg/uuid/issues/26
>
> I was hoping to find a concept and implementation of a few hashes somewhere
> in boost that could be used instead, however I haven't found any.  Back in
> 2010 someone posted a link to a "boost::hashes" repository but I was unable
> to see what the outcome of the discussion was other than that it was not
> included.  A collection of hashing algorithms that are well tested would be
> quite useful in a number of ways; and I like the hash size template style
> used in the repo from 2010:
>
> https://svn.boost.org/svn/boost/sandbox/hash/
>
> Indeed there's mention of a discussion about the proper placement of sha1
> inside boost::uuid in there:
>
> https://svn.boost.org/svn/boost/sandbox/hash/libs/hash/
> doc/html/rationale.html
>
> So I was curious where that effort ended up, and if there was a resolution
> on whether an independent library for hashing was ultimately something that
> folks wanted to pursue or avoid.
>
> Thanks,
>
> Jim
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/
> mailman/listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On Thu, Sep 7, 2017 at 2:50 AM, Richard Hodges via Boost
<[hidden email]> wrote:
> I think it would be reasonable to say that most people (tm) use either the
> openssl or CryptoCpp libraries.

I doubt either of these libraries would pass a Boost review.

Thanks

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On 09/07/17 16:59, Vinnie Falco via Boost wrote:
> On Thu, Sep 7, 2017 at 2:50 AM, Richard Hodges via Boost
> <[hidden email]> wrote:
>> I think it would be reasonable to say that most people (tm) use either the
>> openssl or CryptoCpp libraries.
>
> I doubt either of these libraries would pass a Boost review.

They don't need to.

I think a full implementation of cryptographic algorithms in Boost,
while would be nice in theory, is not very feasible in practice. First,
it would require a group of very skillful and active developers that
keep their hand on pulse in the security and cryptography field.
Vulnerabilities need to be acted upon fast, which also means that Boost
release schedule doesn't suit very well for such a library. The
implementation has to be robust and fast to compete with other
implementations (and by fast I mean including writing assembler routines
for many algorithms). Then, preferably, the implementation would have to
pass an independent audit to gain some trust in users.

A cryptography library in Boost is very much welcome, but, at least
initially, I think, it should focus primarilly on providing proper
interface that fits in C++ ecosystem well. The bulk of the work should
be done by a third party implementation, like OpenSSL or gnutls or
libnss or whatever the native API is on a given platform. Maybe, if the
API becomes popular and the library gains momentum, it could then offer
its own implementation of some algorithms, e.g. to remove external
dependencies. Then the difficulties I mentioned can be tackled gradually
over time.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On Thu, Sep 7, 2017 at 8:08 AM, Andrey Semashev via Boost
<[hidden email]> wrote:
> initially, I think, it should focus primarilly on providing proper interface

Are you suggesting that the first step is to define some concepts such
as BigInteger, Digest, BlockCipher, MessageAuthenticationCode, or
OneWayFunction?

Thanks

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On 09/07/17 18:12, Vinnie Falco via Boost wrote:
> On Thu, Sep 7, 2017 at 8:08 AM, Andrey Semashev via Boost
> <[hidden email]> wrote:
>> initially, I think, it should focus primarilly on providing proper interface
>
> Are you suggesting that the first step is to define some concepts such
> as BigInteger, Digest, BlockCipher, MessageAuthenticationCode, or
> OneWayFunction?

Yes, probably. And then implement those concepts on top of existing
crypto libraries.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
> I think a full implementation of cryptographic algorithms in Boost,
> while would be nice in theory, is not very feasible in practice. First,
> it would require a group of very skillful and active developers that
> keep their hand on pulse in the security and cryptography field.

It looks to me far better to implement a Boost.Hashing library, some
hashes in which might just happen to be crypto strength. Some recent
CPUs can do crypto hashing in hardware nowadays. Very nice.

I vaguely remember that Chandler or Howard had some WG21 proposal for a
standardised hashing infrastructure? One not fundamentally broken like
std::hash. One should probably study/implement that proposal.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On Thu, Sep 7, 2017 at 8:30 AM, Niall Douglas via Boost
<[hidden email]> wrote:
> I vaguely remember that...Howard had some WG21 proposal for a
> standardised hashing infrastructure? One not fundamentally broken like
> std::hash. One should probably study/implement that proposal.

You're thinking of hash_append, described here:

<http://htmlpreview.github.io/?https://github.com/HowardHinnant/papers/blob/master/hashing.html>

Repository:

<https://github.com/howardhinnant/hash_append>

This library is designed for non-cryptographic hashing. Specifically
for unordered containers (hash tables). The requirements for
cryptographic hash algorithms are different, and difficult to unify
with the use-case for hash tables.

For example when calculating the cryptographic digest on serialized
data, endianness of integers matters.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On 07/09/2017 17:11, Vinnie Falco via Boost wrote:
> On Thu, Sep 7, 2017 at 8:30 AM, Niall Douglas via Boost
> <[hidden email]> wrote:
>> I vaguely remember that...Howard had some WG21 proposal for a
>> standardised hashing infrastructure? One not fundamentally broken like
>> std::hash. One should probably study/implement that proposal.
>
> You're thinking of hash_append, described here:

Or a WG21 paper you authored yourself Vinnie!

https://isocpp.org/files/papers/n3980.html

(Thanks to Bjorn Reese for that link)

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Sep 7, 2017, at 12:11 PM, Vinnie Falco via Boost <[hidden email]> wrote:

>
> On Thu, Sep 7, 2017 at 8:30 AM, Niall Douglas via Boost
> <[hidden email]> wrote:
>> I vaguely remember that...Howard had some WG21 proposal for a
>> standardised hashing infrastructure? One not fundamentally broken like
>> std::hash. One should probably study/implement that proposal.
>
> You're thinking of hash_append, described here:
>
> <http://htmlpreview.github.io/?https://github.com/HowardHinnant/papers/blob/master/hashing.html>
>
> Repository:
>
> <https://github.com/howardhinnant/hash_append>
>
> This library is designed for non-cryptographic hashing. Specifically
> for unordered containers (hash tables). The requirements for
> cryptographic hash algorithms are different, and difficult to unify
> with the use-case for hash tables.
>
> For example when calculating the cryptographic digest on serialized
> data, endianness of integers matters.
endian taken into account here:

https://github.com/HowardHinnant/hash_append/blob/master/sha256.h#L30

(example sha256 hasher)

xstd::endian is now C++17 (as std::endian, 23.15.9 [meta.endian]).

Howard



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On Thu, Sep 7, 2017 at 9:26 AM, Howard Hinnant via Boost
<[hidden email]> wrote:
> endian taken into account here:

I use endianness only as an easily understood example of an issue that
makes the needs of a cryptographic hash interface different from an
unordered container hash interface. There are other issues, which
require a move involved discussion than I think is appropriate in this
thread. For example, we cannot have distinct cryptographic and
non-cryptographic serialization algorithms for the same type (using
n3980).

The bottom line is that n3980 needs to go through more iteration and
study with respect to its usage for cryptographic hash algorithms
before we can state that the interface is well suited for it.

Thanks

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On Sep 7, 2017, at 12:40 PM, Vinnie Falco via Boost <[hidden email]> wrote:

>
> On Thu, Sep 7, 2017 at 9:26 AM, Howard Hinnant via Boost
> <[hidden email]> wrote:
>> endian taken into account here:
>
> I use endianness only as an easily understood example of an issue that
> makes the needs of a cryptographic hash interface different from an
> unordered container hash interface. There are other issues, which
> require a move involved discussion than I think is appropriate in this
> thread. For example, we cannot have distinct cryptographic and
> non-cryptographic serialization algorithms for the same type (using
> n3980).
Demo for distinct serialization algorithms for varying hashers:

    #include "hash_append.h"
    #include "sha256.h"
    #include "fnv1a.h"
    #include <iostream>

    class X
    {
        std::array<unsigned char, 32> d_{};
    public:
        friend void hash_append(acme::sha256& h, X const& x)
        {
            std::cout << "hash with sha256: ";
            xstd::hash_append(h, x.d_);
        }

        template <class Hasher>
        friend void hash_append(Hasher& h, X const& x)
        {
            std::cout << "hash with everything else: ";
            xstd::hash_append(h, x.d_, x.d_.size());
        }
    };

    int
    main()
    {
        X x;
        xstd::uhash<acme::fnv1a> h1;
        std::cout << h1(x) << '\n';
        xstd::uhash<acme::sha256> h2;
        for (auto const& i : h2(x))
            std::cout << std::hex << (unsigned)i;
        std::cout << '\n';
    }

Output:

    hash with everything else: 7024978713738287141
    hash with sha256: 66687aadf862bd776c8fc18b8e9f8e2089714856ee233b3902a591dd5f2925


>
> The bottom line is that n3980 needs to go through more iteration and
> study with respect to its usage for cryptographic hash algorithms
> before we can state that the interface is well suited for it.

I welcome and encourage more study and discussion in any context, thread or list.

My only motivation here is to dispel mistaken believes that the current hash_append infrastructure can not handle issues such as endian or alternate serialization algorithms for different hashers.  Admittedly, the latest proposal (N3980) is sorely in need of updating and likely doesn’t have the latest details of the implementation at https://github.com/HowardHinnant/hash_append .  And thanks for the shout-out to this lib earlier in this thread.

Howard



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On Thu, Sep 7, 2017 at 1:07 PM, Howard Hinnant <[hidden email]> wrote:
> Demo for distinct serialization algorithms for varying hashers:
> ...
>         friend void hash_append(acme::sha256& h, X const& x)
>

Burdening a user defined type with the requirement to know exactly
which cryptographic hash functions will be used with it ahead of time,
and adjusting the serialization algorithm to compensate - is precisely
the style which n3980 aims to prevent.

> My only motivation here is to dispel mistaken believes that the current
> hash_append infrastructure can not handle issues such as endian or
> alternate serialization algorithms for different hashers.

The problem with the way endianness is handled in n3980 is that it
treats it as a property of the hash algorithm, when it is actually a
property of the protocol which makes the use of a cryptographic hash
function.

For example, imagine we have two protocols X and Y. Each defines some
object lets call it "Customer", with various fields including unsigned
or multi-precision integers. Each protocol also calculates the SHA256
digest of the serialized representation of a Customer. The question
arises, when a Customer is serialized, what endianness is used for the
integers?

The case where protocols X and Y each choose a different endianness in
the specification makes the current approach to endianness in n3980
less than ideal. The problem is that n3980 was not designed from the
ground up with the needs of cryptographic hashing in mind, so the
interface is not a perfect match for that use-case.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 7 September 2017 at 18:08, Andrey Semashev via Boost <
[hidden email]> wrote:

> I think a full implementation of cryptographic algorithms in Boost, while
> would be nice in theory, is not very feasible in practice.
>

I agree with you, but for an important other reason, over and above the
ones you mention. Crypto is tricky businness. If you don't know what your
doing (and most people don't), things can go horribly wrong. Supplying the
building blocks for doing Crypto (as suggested in follow-up post to this
one) is not "safe". The way these alogo's get integrated in the code-base
is hugely important, problems like side channel attacks spring to mind,
it's a different way of thinking about algorithms.

I think an approach like the one of libsodium
<https://www.gitbook.com/book/jedisct1/libsodium/details> is the correct
one. Per "task" (let's say password authentication) to be done, there are 1
or 2 (full and complete) options on offer, options that do those things
well, and in a safe way. This approach, though, is contrary to boosts'
philosophy (offer all and every option, so the user can do things "his/her"
way). Another good crypto library is Botan <https://botan.randombit.net/>.
The code is well organised and easy to grasp and extend. Botan is slightly
easier to use than CryptoCpp IMO.

degski
--
"*Ihre sogenannte Religion wirkt bloß wie ein Opiat reizend, betäubend,
Schmerzen aus Schwäche stillend.*" - Novalis 1798

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On Thu, Sep 7, 2017 at 10:12 PM, degski via Boost <[hidden email]> wrote:
> libsodium

A C library with no abstractions... nack

> <Botan <https://botan.randombit.net/>

C++, with some abstractions in the form of common base classes. This
looks promising :)

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Crypto/hashes library?

Boost - Dev mailing list
On 8 September 2017 at 08:33, Vinnie Falco via Boost <[hidden email]>
wrote:

> On Thu, Sep 7, 2017 at 10:12 PM, degski via Boost <[hidden email]>
> wrote:
> A C library with no abstractions... nack
>

C is *not* a dirty word (assuming it *is* a word), AFAIK! OpenSSL is C,
isn't it?

But, if you insist on C++, you could check out the Other Language Bindings
page: https://download.libsodium.org/doc/bindings_for_other_languages/ ,
where you'll find 3 wrapper libraries.

The importance is the different approach, I wrote about earlier.


> C++, with some abstractions in the form of common base classes. This
> looks promising :)
>

It's also been C++11, since, umh, well, 2011 :-).

degski
--
"*Ihre sogenannte Religion wirkt bloß wie ein Opiat reizend, betäubend,
Schmerzen aus Schwäche stillend.*" - Novalis 1798

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost