Maintainers: Adding Coverity Scan to your library

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Maintainers: Adding Coverity Scan to your library

Boost - Dev mailing list
I have completed the work necessary to add Coverity Scan to Boost.Uuid and
wanted to share the steps involved.  This work can be carried into other
libraries fairly easily.

1. Go to Coverity Scan: https://scan.coverity.com/
2. Sign up/in with your Github account.
3. Create a project for your boostorg/library.  It will take about 24 hours
for them to activate your account so that you can use the results browser.

To run a coverity scan manually:

1. Check out your project master (or whatever branch you want to scan).
2. cd into BOOST_ROOT (where "libs/" is located)
3. Run the libs/uuid/covscan.sh script with the following environment
variables set:
    A. COVERITY_SCAN_TOKEN: the project token in coverity scan
    B. COVERITY_SCAN_NOTIFICATION_EMAIL: your email
    C. SELF: your library name, for example SELF=uuid for Boost.Uuid
4. You will need to clean up some cruft it leaves behind as the script is
designed for Travis CI - just a couple directories starting with "cov" and
a tar.xz file.

To integrate coverity scans for every pull request or push into master with
Travis CI:

1. Make sure you have Travis CI set up properly to build your project.
2. Copy the Coverity Scan stanza from libs/uuid/.travis.yml into your
.travis.yml file.  Note that the Boost.Uuid travis configuration now uses
jobs instead of matrix, as jobs allow for conditions.  I would encourage
you to convert your .travis.yml to use jobs as well.
3. The next pull request into master, or the next push into master, will
trigger a build, and any build for master will run covscan.sh and upload
the data.

You can add component regular expressions to ignore/omit other boost
library errors if you wish.  Here are the results for Boost.Uuid.  I have
not excluded other components yet:

https://scan.coverity.com/projects/boostorg-uuid

Initially I saw a flurry of "unhandled exception" items which I dismissed
as intentional (i.e. I want the test to segfault if an unexpected exception
is thrown so that I can debug it), and that will likely be a common
practice for test files, but should not be for implementations.  I also
caution folks against marking something as a false positive.  Approach it
with the mindset of, "it must be a fault of the library" before jumping to,
"it must be a fault in Coverity" and really convince yourself it is not a
code issue.

Even if you do not have travis integration in your library, running a
manual scan will help you improve the quality of your library, so I
recommend everyone at least do that much.

- Jim

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Maintainers: Adding Coverity Scan to your library

Boost - Dev mailing list
On 12/10/2017 14:02, James E. King, III via Boost wrote:

> Even if you do not have travis integration in your library, running a
> manual scan will help you improve the quality of your library, so I
> recommend everyone at least do that much.

I don't have positive experiences with Coverity. Nor with their tech
support.

If you're going to run a lint tool, the most recent clang-tidy is hard
to beat, even compared to very expensive commercial lint tools. Plus, it
can rewrite your code to fix the lint problem for you. Much faster and
easier.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Maintainers: Adding Coverity Scan to your library

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Thu, Oct 12, 2017 at 6:02 AM, James E. King, III via Boost
<[hidden email]> wrote:
> I have completed the work necessary to add Coverity Scan

The last time I looked at Coverity Scan, there was a weekly limit on
the amount of build throughput (minutes?) per project. Has this
situation changed?

Thanks

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Maintainers: Adding Coverity Scan to your library

Boost - Dev mailing list
On Thu, Oct 12, 2017 at 9:22 AM, Vinnie Falco via Boost <
[hidden email]> wrote:

> On Thu, Oct 12, 2017 at 6:02 AM, James E. King, III via Boost
> <[hidden email]> wrote:
> > I have completed the work necessary to add Coverity Scan
>
> The last time I looked at Coverity Scan, there was a weekly limit on
> the amount of build throughput (minutes?) per project. Has this
> situation changed?
>
> Thanks
>
>
They have throttling on submission results.  The rate allowed depends on
the size of the project:
https://scan.coverity.com/faq#frequency

You can handle this a couple ways:

1. Create a coverity_scan branch and limit the travis integration to run
covscan.sh only on that branch.  You control when coverity scan runs.
2. Add travis integration to scan master, this will scan pull requests or
merges into master.  To kick off a build, submit a pull request of develop
into master as a preflight check.  This will run and update Coverity Scan
with results from what will merge into master.

Even if you run it once or twice a month, project quality will improve
based on action taken on the results.

It is free for open source projects, so everyone should take advantage of
it.

- Jim

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Maintainers: Adding Coverity Scan to your library

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list

12.10.2017 16:17, Niall Douglas via Boost пишет:

> On 12/10/2017 14:02, James E. King, III via Boost wrote:
>
>> Even if you do not have travis integration in your library, running a
>> manual scan will help you improve the quality of your library, so I
>> recommend everyone at least do that much.
> I don't have positive experiences with Coverity. Nor with their tech
> support.
>
> If you're going to run a lint tool, the most recent clang-tidy is hard
> to beat, even compared to very expensive commercial lint tools. Plus, it
> can rewrite your code to fix the lint problem for you. Much faster and
> easier.
>
> Niall
>

You can use free cppcheck or PVS-Studio (free for header files).

I have good experience with these tools.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Maintainers: Adding Coverity Scan to your library

Boost - Dev mailing list
On 12 October 2017 at 16:02, Alexander Zaitsev via Boost
<[hidden email]> wrote:

> 12.10.2017 16:17, Niall Douglas via Boost пишет:
>> On 12/10/2017 14:02, James E. King, III via Boost wrote:
>>
>>> Even if you do not have travis integration in your library, running a
>>> manual scan will help you improve the quality of your library, so I
>>> recommend everyone at least do that much.
>>
>> I don't have positive experiences with Coverity. Nor with their tech
>> support.
>>
>> If you're going to run a lint tool, the most recent clang-tidy is hard
>> to beat, even compared to very expensive commercial lint tools. Plus, it
>> can rewrite your code to fix the lint problem for you. Much faster and
>> easier.
>>
>> Niall
>>
>
> You can use free cppcheck or PVS-Studio (free for header files).

Any reference to support that?

AFAIK, there is more steps involved and no mention of special licensing
for headers or header-only libraries https://www.viva64.com/en/b/0457/

> I have good experience with these tools.

Me too.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost