Boost.Test BOOST_TEST/BOOST_REQUIRE_MESSAGE message arg problems

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Boost.Test BOOST_TEST/BOOST_REQUIRE_MESSAGE message arg problems

Boost - Users mailing list
Hello Boost Users,

for my project I'm testing using BOOST_AUTO_TEST_SUITE and data driven
test sets (BOOST_DATA_TEST_CASE). The goal is to check the proper
working of a parser by checking success as such and compare the result
with a "expected" data (string) vector. In case of parse error, I want
to write the captured cout/cerr by boost.test into a file. In case of
compare error the parsed result also shall be written to a file. The
fragments show the concept/current state:

namespace x3_test { // aka utils ....

/* [Detect if boost test case failed](
  *
https://stackoverflow.com/questions/18466857/detect-if-boost-test-case-failed?answertab=active#tab-top)
*/
bool current_test_passing()
{
   using namespace boost::unit_test;
   auto const id = framework::current_test_case().p_id;
   auto const test_results = results_collector.results(id);
   return test_results.passed();
}

std::string report_diagnostic(fs::path const& test_case_name,
std::string const& input, std::string const& result)
{
     ...
     if(!x3_test::current_test_passing()) {     // only write in case of
failed test
         test_case_result_writer result_writer(test_case_name);
         result_writer.write(result);
     }
     return ss.str();
}

} // x3_test

...

BOOST_AUTO_TEST_SUITE(....

BOOST_DATA_TEST_CASE( declaration,
        dataset.input()
     ^ dataset.expect()
     ^ dataset.test_case_name(),
     code, expect_AST, test_case_name)
{
     using attribute_type = ast::declaration;
     auto const parser = parser::declaration;

     x3_test::testing_parser<attribute_type> parse;
     auto [parse_ok, parsed_AST] = parse(code, parser);

     BOOST_TEST(parse_ok);
     BOOST_REQUIRE_MESSAGE(parse_ok,
         x3_test::report_diagnostic(test_case_name, code, parsed_AST)
     );

     BOOST_TEST(parsed_AST == expect_AST, btt::per_element());
     BOOST_REQUIRE_MESSAGE(x3_test::current_test_passing(),
         x3_test::report_diagnostic(test_case_name, code, parsed_AST)
     );
}

BOOST_AUTO_TEST_SUITE_END()

After some testing, the error count is twice the real (obviously here)
but it's all above a work arround.

It seems, that BOOST_REQUIRE_MESSAGE evaluates always the message
predicate (shown by using --log_level=all) nevertheless the test case
successed or failed. If failed it is streamed to the console.

My first thoughs where that the message is called only in case of test
failure which isn't true.

The problem is now, that my report_diagnostic function write the test
output (parsed_AST) to a special dir/file for later evaluating etc. The
intuitive way would be:

std::string report_diagnostic(fs::path const& test_case_name,
std::string const& input, std::string const& result)
{
     ...
     test_case_result_writer result_writer(test_case_name);
     result_writer.write(result);

     return ss.str();
}

BOOST_REQUIRE_MESSAGE(parse_ok,
         // call diagnostic only in failure case and write then
         x3_test::report_diagnostic(test_case_name, code, parsed_AST)
     );
BOOST_REQUIRE_MESSAGE((parsed_AST == expect_AST, btt::per_element()),
         // call diagnostic only in failure case and write then
         x3_test::report_diagnostic(test_case_name, code, parsed_AST)
     );

Hence not depending on current_test_passing() any more.

How to get this?


Further, how can I get the captured std::cerr in case of '!parse_ok'
inside report_diagnostic?

Thanks,

Olaf

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users