[test] unit test command-line handling

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[test] unit test command-line handling

Boost - Dev mailing list
Hello,

this is a question concerning the Boost.Test framework.

I'd like to pass command-line arguments to my tests, and I'm unsure of
how to proceed. I have found
https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/adv_scenarios/obsolete_init_func.html,
which shows how to access the command-line programmatically, but that's
declared obsolete, and it's also lacking some important info, such as
how I should handle the argv vector. Are there any examples that
demonstrate this use case ? Specifically:

* What (non-deprecated) API can I use to access command-line arguments
during test initialization ?

* How does my own handling of command-line arguments interact with
Boost.Test's own command-line argument handling ?

Thanks,


Stefan

--

      ...ich hab' noch einen Koffer in Berlin...
   


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [test] unit test command-line handling

Boost - Dev mailing list
On 12.09.18 00:21, Stefan Seefeld via Boost wrote:

> Hello,
>
> this is a question concerning the Boost.Test framework.
>
> I'd like to pass command-line arguments to my tests, and I'm unsure of
> how to proceed. I have found
> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/adv_scenarios/obsolete_init_func.html,
> which shows how to access the command-line programmatically, but that's
> declared obsolete, and it's also lacking some important info, such as
> how I should handle the argv vector. Are there any examples that
> demonstrate this use case ? Specifically:
>
> * What (non-deprecated) API can I use to access command-line arguments
> during test initialization ?

By "test initialization", do you mean before the first test starts?
The argc/argv are available to the master test suite:

https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_organization/test_suite/master_test_suite.html#boost_test.tests_organization.test_suite.master_test_suite.command_line_arguments_access_in

In 1.68, it is also possible to use this for data driven test cases,
using the make_delayed (if you need a test generator that is dependent
on your argc/argv).

https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_log.html#boost_test.change_log.new_features

> * How does my own handling of command-line arguments interact with
> Boost.Test's own command-line argument handling ?

I am surprised I did not create a doc entry about this. The way it works
is indicated in the change logs of boost.test:

https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_log.html#boost_test.change_log.boost_test_v3_1_boost_1_60

Anything after -- is passed to the master test suite argc/argv and is
available to your test module.

I will update the doc on the two points above to make it clearer.

Raffi

>
> Thanks,
>
>
> Stefan
>



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [test] unit test command-line handling

Boost - Dev mailing list
On 09/12/18 14:28, Raffi Enficiaud wrote:
> On 12.09.18 00:21, Stefan Seefeld via Boost wrote:
>>
>> * What (non-deprecated) API can I use to access command-line arguments
>> during test initialization ?
>
> By "test initialization", do you mean before the first test starts?

Yes.

> The argc/argv are available to the master test suite:
>
> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_organization/test_suite/master_test_suite.html#boost_test.tests_organization.test_suite.master_test_suite.command_line_arguments_access_in
>

I see. That code could use a bit of an explanation. You present a
function that takes "argc" and "argv" as input, but rather than using
those variables themselves, you access the argument vector via
"framework::master_test_suite().argc" etc.. I find that a bit...em...
counter-intuitive. What's the rationale for that ? Is the function
argument the full argument vector, including the ones already consumed
by Boost.Test itself ?

> In 1.68, it is also possible to use this for data driven test cases,
> using the make_delayed (if you need a test generator that is dependent
> on your argc/argv).
>
> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_log.html#boost_test.change_log.new_features
>
>
>> * How does my own handling of command-line arguments interact with
>> Boost.Test's own command-line argument handling ?
>
> I am surprised I did not create a doc entry about this. The way it
> works is indicated in the change logs of boost.test:
>
> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_log.html#boost_test.change_log.boost_test_v3_1_boost_1_60
>
>
> Anything after -- is passed to the master test suite argc/argv and is
> available to your test module.

That makes sense.

>
> I will update the doc on the two points above to make it clearer.

Thank you. Please consider adding a top-level section on passing
command-line arguments, perhaps under "Runtime parameters". I believe
that's a common enough use-case that it might be worthwhile for this not
to be hidden in a (seemingly unrelated) section about the master test suite.

Thanks for your help !


Stefan

--

      ...ich hab' noch einen Koffer in Berlin...
   


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [test] unit test command-line handling

Boost - Dev mailing list
On 09/12/18 18:50, Stefan Seefeld wrote:

> On 09/12/18 14:28, Raffi Enficiaud wrote:
>
>> The argc/argv are available to the master test suite:
>>
>> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_organization/test_suite/master_test_suite.html#boost_test.tests_organization.test_suite.master_test_suite.command_line_arguments_access_in
>>
>
> I see. That code could use a bit of an explanation. You present a
> function that takes "argc" and "argv" as input, but rather than using
> those variables themselves, you access the argument vector via
> "framework::master_test_suite().argc" etc.. I find that a bit...em...
> counter-intuitive. What's the rationale for that ? Is the function
> argument the full argument vector, including the ones already consumed
> by Boost.Test itself ?

...and what is the appropriate way to signal an error, such as a missing
or unsupported command-line argument ? (All the examples in the docs I
could find return '0', so another related question is: what is the
meaning of the return value ?)

Thanks,

Stefan

--

      ...ich hab' noch einen Koffer in Berlin...
   


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [test] unit test command-line handling

Boost - Dev mailing list
On 13.09.18 03:21, Stefan Seefeld via Boost wrote:

> On 09/12/18 18:50, Stefan Seefeld wrote:
>> On 09/12/18 14:28, Raffi Enficiaud wrote:
>>
>>> The argc/argv are available to the master test suite:
>>>
>>> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_organization/test_suite/master_test_suite.html#boost_test.tests_organization.test_suite.master_test_suite.command_line_arguments_access_in
>>>
>>
>> I see. That code could use a bit of an explanation. You present a
>> function that takes "argc" and "argv" as input, but rather than using
>> those variables themselves, you access the argument vector via
>> "framework::master_test_suite().argc" etc.. I find that a bit...em...
>> counter-intuitive. What's the rationale for that ? Is the function
>> argument the full argument vector, including the ones already consumed
>> by Boost.Test itself ?
>
> ...and what is the appropriate way to signal an error, such as a missing
> or unsupported command-line argument ? (All the examples in the docs I
> could find return '0', so another related question is: what is the
> meaning of the return value ?)
>

There is this exception you may throw:

https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost/unit_test/framework/setup_error.html

Again, this is not so well documented here:

https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/adv_scenarios/obsolete_init_func.html

Raffi


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [test] unit test command-line handling

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 13.09.18 00:50, Stefan Seefeld via Boost wrote:

> On 09/12/18 14:28, Raffi Enficiaud wrote:
>> On 12.09.18 00:21, Stefan Seefeld via Boost wrote:
>>>
>>> * What (non-deprecated) API can I use to access command-line arguments
>>> during test initialization ?
>>
>> By "test initialization", do you mean before the first test starts?
>
> Yes.
>
>> The argc/argv are available to the master test suite:
>>
>> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_organization/test_suite/master_test_suite.html#boost_test.tests_organization.test_suite.master_test_suite.command_line_arguments_access_in
>>
>
> I see. That code could use a bit of an explanation. You present a
> function that takes "argc" and "argv" as input, but rather than using
> those variables themselves, you access the argument vector via
> "framework::master_test_suite().argc" etc.. I find that a bit...em...
> counter-intuitive. What's the rationale for that ? Is the function
> argument the full argument vector, including the ones already consumed
> by Boost.Test itself ?

Apart from the historical side of this design, this has two benefits IMO:
1- the argc/argv passed to the rest of the test module at runtime may
have been altered, either by the test module itself (dropping some
params in a global fixture), or by the boost.test framework. The
argc/argv seen by the test module are the ones that appear after the --
2- the master test suite is a singleton, so you do not need to pass
around argc/argv

>
>> In 1.68, it is also possible to use this for data driven test cases,
>> using the make_delayed (if you need a test generator that is dependent
>> on your argc/argv).
>>
>> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_log.html#boost_test.change_log.new_features
>>
>>
>>> * How does my own handling of command-line arguments interact with
>>> Boost.Test's own command-line argument handling ?
>>
>> I am surprised I did not create a doc entry about this. The way it
>> works is indicated in the change logs of boost.test:
>>
>> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/change_log.html#boost_test.change_log.boost_test_v3_1_boost_1_60
>>
>>
>> Anything after -- is passed to the master test suite argc/argv and is
>> available to your test module.
>
> That makes sense.
>
>>
>> I will update the doc on the two points above to make it clearer.
>
> Thank you. Please consider adding a top-level section on passing
> command-line arguments, perhaps under "Runtime parameters". I believe
> that's a common enough use-case that it might be worthwhile for this not
> to be hidden in a (seemingly unrelated) section about the master test suite.

Yes, will do, I regularly have questions about this part.

> Thanks for your help !

You're very welcome.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [test] unit test command-line handling

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
>> ...and what is the appropriate way to signal an error, such as a missing
>> or unsupported command-line argument ? (All the examples in the docs I
>> could find return '0', so another related question is: what is the
>> meaning of the return value ?)
>>
>
> There is this exception you may throw:
>
> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost/unit_test/framework/setup_error.html 
>
>
> Again, this is not so well documented here:
>
> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/adv_scenarios/obsolete_init_func.html 

I assume throwing setup_error works only with the obsolete init_func?
How would it be with the 'new' way described at the caution note on
bottom of the last URL.

Thanks,
Olaf

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [test] unit test command-line handling

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list


On 2018-09-13 02:53 PM, Raffi Enficiaud via Boost wrote:

> On 13.09.18 00:50, Stefan Seefeld via Boost wrote:
>> On 09/12/18 14:28, Raffi Enficiaud wrote:
>>> On 12.09.18 00:21, Stefan Seefeld via Boost wrote:
>>>>
>>>> * What (non-deprecated) API can I use to access command-line arguments
>>>> during test initialization ?
>>>
>>> By "test initialization", do you mean before the first test starts?
>>
>> Yes.
>>
>>> The argc/argv are available to the master test suite:
>>>
>>> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_organization/test_suite/master_test_suite.html#boost_test.tests_organization.test_suite.master_test_suite.command_line_arguments_access_in 
>>>
>>>
>>
>> I see. That code could use a bit of an explanation. You present a
>> function that takes "argc" and "argv" as input, but rather than using
>> those variables themselves, you access the argument vector via
>> "framework::master_test_suite().argc" etc.. I find that a bit...em...
>> counter-intuitive. What's the rationale for that ? Is the function
>> argument the full argument vector, including the ones already consumed
>> by Boost.Test itself ?
>
> Apart from the historical side of this design, this has two benefits IMO:
> 1- the argc/argv passed to the rest of the test module at runtime may
> have been altered, either by the test module itself (dropping some
> params in a global fixture), or by the boost.test framework. The
> argc/argv seen by the test module are the ones that appear after the --
> 2- the master test suite is a singleton, so you do not need to pass
> around argc/argv

And what about the return value ? What does it mean to return 0, or
something else ?

Thanks,

Stefan

--

       ...ich hab' noch einen Koffer in Berlin...
     


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost