Home > The Unit Test Framework > User's guide > Test organization > Expected failures specification
PrevNext

Expected failures specification

While in a perfect world all test assertions should pass in order for a test module to pass, in some situations it is desirable to temporarily allow particular tests to fail. For example, where a particular feature is not implemented yet and one needs to prepare a library for the release or when particular test fails on some platforms. To avoid a nagging red box in regression tests table, you can use the expected failures feature.
在完美的世界中,所有的測試斷言都應該通過,這樣整個測試模塊才會通過,但在某種情況下,需要特定的測試暫時失敗。 例如,某個特性並沒有被實現,但需要準備一個發佈庫,在這種情況下,某些測試就會失敗。 為了防止在回歸測試表格中惱人的紅色警報,你可以使用期望失敗特性。

This feature allows specifying an expected number of failed assertions per test unit. The value is specified during test tree construction, and can't be updated during test execution.
這個特性允許對每個測試單元指定期望失敗斷言的數量。這個值在測試樹構造過程中指定,在測試執行中不會更改。

The feature is not intended to be used to check for expected functionality failures. To check that a particular input is causing an exception to be thrown use BOOST_CHECK_THROW family of testing tools.
這個特性並不試圖用於檢查期望的功能性失敗。 要檢查對於特定的輸入將引起異常拋出,使用 BOOST_CHECK_THROW 一族的測試工具。

The usage of this feature should be limited and employed only after careful consideration. In general you should only use this feature when it is necessary to force a test module to pass without actually fixing the problem. Obviously, an excessive usage of expected failures defeats the purpose of the unit test. In most cases it only needs be applied temporarily.
這個特性的使用應當受到限制並只有在仔細考慮後才使用。 通常只有當不實際修正問題,但確實需要測試模塊通過時,才使用這個特性。 顯然,期望失敗的過量使用會妨礙單元測試的目的。 在多數情況下,這個特性只是暫時使用。

You also need to remember that the expected failure specification is per test case. This means that any failed assertion within that test case can satisfy the expected failures quota. Meaning it is possible for an unexpected failure to occur to satisfy this quota.
同時需要記得,期望失敗定義是每個測試用例相關的。 這意味著在測試內的任何失敗斷言都會滿足期望失敗的配額。 這意味著非期望的失敗發生可能會滿足失敗的配額。

[Note] Note

If an assertion at fault is fixed and passed, while an expected failures specification still present, the test case is going to fail, since the number of failures is smaller than expected.
如果失敗的斷言被修正,可以通過,但期望失敗說明仍然存在,那麼測試用例仍然會失敗,因為失敗的數量少於期望的。

Usage with manually registered test cases

To set the value of expected failures for the manually registered test unit pass it as a second argument for the test_suite::add call during test unit registration.
想要為手動註冊的測試單元設置期望失敗的數量,在註冊測試單元時將它作為第二個參數傳入 test_suite::add 方法中。

Example 21. Expected failures specification for manually registered test case

#include <boost/test/included/unit_test.hpp>
using namespace boost::unit_test;

//____________________________________________________________________________//

void free_test_function()
{
    BOOST_CHECK( 2 == 1 );    
}

//____________________________________________________________________________//

test_suite*
init_unit_test_suite( int, char* [] ) {
    framework::master_test_suite().
        add( BOOST_TEST_CASE( &free_test_function ), 2 );

    return 0;
}

//____________________________________________________________________________//
Source code | Show output
> example --log_level=message
Running 1 test case...
test.cpp(8): error in "free_test_function": check 2 == 1 failed
Test case has less failures then expected

*** No errors detected

Usage with automatically registered test cases

To set the number of expected failures for the automatically registered test unit use the macro BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES before the test case definition.
想要為自動註冊的測試單元設置期望失敗的數量,在測試用例定義前使用宏 BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES。

BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES(test_case_name, number_of_expected_failures)

You can use this macro both on a file scope and inside a test suite. Moreover you can use it even if name of test units coincide in different test suites. Expected failures specification applies to the test unit belonging to the same test suite where BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES resides.
你可以在文件範圍或測試套件內使用這個宏。即使是和不同測試套件的測試單元同名也可以使用。 期望失敗定義應用於和 BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES 相同測試套件內的測試單元。

Example 22. Expected failures specification for automatically registered test case

#define BOOST_TEST_MODULE example
#include <boost/test/included/unit_test.hpp>

//____________________________________________________________________________//

BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES( my_test1, 1 )

BOOST_AUTO_TEST_CASE( my_test1 )
{
    BOOST_CHECK( 2 == 1 );
}

//____________________________________________________________________________//

BOOST_AUTO_TEST_SUITE( internal )

BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES( my_test1, 2 )

BOOST_AUTO_TEST_CASE( my_test1 )
{
    BOOST_CHECK_EQUAL( sizeof(int), sizeof(char) );
    BOOST_CHECK_EQUAL( sizeof(int*), sizeof(char) );
}

//____________________________________________________________________________//

BOOST_AUTO_TEST_SUITE_END()
Source code | Show output
> example --report_level=short
Running 2 test cases...
test.cpp(10): error in "my_test1": check 2 == 1 failed
test.cpp(21): error in "my_test1": check sizeof(int) == sizeof(char) failed [4 != 1]
test.cpp(22): error in "my_test1": check sizeof(int*) == sizeof(char) failed [4 != 1]

Test suite "example" passed with:
  3 assertions out of 3 failed
  3 failures expected
  2 test cases out of 2 passed


PrevUpHomeNext