[boost.test] parsing runtime arguments

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[boost.test] parsing runtime arguments

Boost - Users mailing list
Hi,
how get an 'user' API for the boost.test like this:

testsuite_xyz -- --help

or even

testsuite_xyz -- --save-path=/foo

to get the user aspects. I know I can get the args by

     int argc{ framework::master_test_suite().argc };
     char** argv{ framework::master_test_suite().argv };

Further, at test/include/boost/test/utils/runtime/cla/parser.hpp are
some utils used by boost.test self. Anyway, the problem rises to get the
behaviour desribed above, simply using a global fixture with:

fixture::fixture()
{
     int argc{ utf::framework::master_test_suite().argc };
     char** argv{ utf::framework::master_test_suite().argv };

        for (int i = 0; i != argc; ++i) {
            std::cerr << "test_observer_fixture: " << argv[i] << "\n";
        }
}

doesn't give the expected results - all the BOOST_AUTO_TEST_SUITE test
cases are called even they depend on this arguments.

Thanks,
Olaf

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [boost.test] parsing runtime arguments

Boost - Users mailing list
On 12.08.18 20:17, Olaf Peter via Boost-users wrote:

> Hi,
> how get an 'user' API for the boost.test like this:
>
> testsuite_xyz -- --help
>
> or even
>
> testsuite_xyz -- --save-path=/foo
>
> to get the user aspects. I know I can get the args by
>
>      int argc{ framework::master_test_suite().argc };
>      char** argv{ framework::master_test_suite().argv };
>
> Further, at test/include/boost/test/utils/runtime/cla/parser.hpp are
> some utils used by boost.test self. Anyway, the problem rises to get the
> behaviour desribed above, simply using a global fixture with:
>
> fixture::fixture()
> {
>      int argc{ utf::framework::master_test_suite().argc };
>      char** argv{ utf::framework::master_test_suite().argv };
>
>         for (int i = 0; i != argc; ++i) {
>             std::cerr << "test_observer_fixture: " << argv[i] << "\n";
>         }
> }
>
> doesn't give the expected results - all the BOOST_AUTO_TEST_SUITE test
> cases are called even they depend on this arguments.
>
> Thanks,
> Olaf

What about using the new API for fixtures and accessing argc/argv from
the "setup" member function?

https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_organization/fixtures/models.html

If you are constructing a global fixture, the class is instanciated
before the framework starts, hence the observed behaviour.

Best,
Raffi


_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [boost.test] parsing runtime arguments

Boost - Users mailing list
In reply to this post by Boost - Users mailing list
On 12.08.18 20:17, Olaf Peter via Boost-users wrote:

> Hi,
> how get an 'user' API for the boost.test like this:
>
> testsuite_xyz -- --help
>
> or even
>
> testsuite_xyz -- --save-path=/foo
>
> to get the user aspects. I know I can get the args by
>
>      int argc{ framework::master_test_suite().argc };
>      char** argv{ framework::master_test_suite().argv };
>
> Further, at test/include/boost/test/utils/runtime/cla/parser.hpp are
> some utils used by boost.test self. Anyway, the problem rises to get the
> behaviour desribed above, simply using a global fixture with:
>
> fixture::fixture()
> {
>      int argc{ utf::framework::master_test_suite().argc };
>      char** argv{ utf::framework::master_test_suite().argv };
>
>         for (int i = 0; i != argc; ++i) {
>             std::cerr << "test_observer_fixture: " << argv[i] << "\n";
>         }
> }
>
> doesn't give the expected results - all the BOOST_AUTO_TEST_SUITE test
> cases are called even they depend on this arguments.
>
> Thanks,
> Olaf

Also, for installing the global fixture, please use the new macro
BOOST_TEST_GLOBAL_FIXTURE.

Raffi

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [boost.test] parsing runtime arguments

Boost - Users mailing list
In reply to this post by Boost - Users mailing list
Hello Raffi,

> What about using the new API for fixtures and accessing argc/argv from
> the "setup" member function?
>
> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_organization/fixtures/models.html 
>
>
> If you are constructing a global fixture, the class is instanciated
> before the framework starts, hence the observed behaviour.

Ok, so the constructor isn't the right place for this, thanks for
pointing out.

At this time I did some experiments with your suggestions using a global
fixture.

So, what I finally like to have as behaviour is shown below (actually
using quick&dirty CLI11, but this detail is information only):

$ bin\testrunner_a.exe  -- --help
My Test Module/Suite Name
Usage: bin\testrunner_a.exe [OPTIONS]

Options:
   -h,--help                   Print this help message and exit
   -s,--source-dir DIR         directory to the test case sources.
   --input-extension TEXT      file extension at read time.
   -d,--destination-dir DIR    directory where to write the test results.
   --output-extension TEXT     file extension at write time.
$ bin\testrunner_b.exe  -- --help
...
$ bin\testrunner_a.exe -- -s ../source/testsuite/
....


with a re-usable (maybe) fixture for the CLI parameter where these
parameters are accessible later.

What I have is:

- a class implementing the delayed dataset API
(https://www.boost.org/doc/libs/1_68_0/boost/test/data/monomorphic/delayed.hpp)
where the test data (input and reference/gold) is loaded at runtime
given by CLI args (--source-dir and --input-extension). It's obvious
that --source-dir is required.
- a failure_diagnostic_fixture saving data in case of test failure,
where the save path/file is also created at runtime from CLI args
(--destination-dir and --output-extension).

At his time, each of these classes is re-usable for different test
suites and have their own command line parser member functions and are
used like:

---8<---
BOOST_DATA_TEST_CASE_F( failure_diagnostic_fixture,
        labels_ok,
     utf_data::make_delayed<dataset_loader>(
        "test_case/labels_ok" ),
     input, expected, test_case_name)
{
     btt::output_test_stream                     os;

...
     // call failure_diagnostic_fixture's member and save
     // context in case of failure for later inspection
     failure_closure(test_case_name, input, os.str());
}
--->8---

Getting the command line options can be accomplished using a global
fixture, than using a static member which can be used by the two helper
mentioned above. But, this prevents to get the test_suite/module name
for the help message (since isn't exist yet). Also, I didn't found a way
to stop the further tests after --help requested.

Moving the argc/argv code into the two helper classes doesn't make the
things easier, since I would have two help messages than.

I don't like thing more complicate than necessary, but I have some
testuite_xyz which I want to run independent from cmake/ctest; so the
command line arguments are mandatory (even --help :-).

Thanks,
Olaf
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users