kernel/tools/testing/selftests/kselftest.h, branch linux-5.2.y

selftests: Add test plan API to kselftest.h and adjust callers

2019-04-25T19:15:46Z

The test plan for TAP needs to be declared immediately after the header. This adds the test plan API to kselftest.h and updates all callers to declare their expected test counts. Signed-off-by: Kees Cook Signed-off-by: Shuah Khan

selftests: Move test output to diagnostic lines

2019-04-25T19:15:19Z

This changes the selftest output so that each test's output is prefixed with "# " as a TAP "diagnostic line". This creates a bit of a kernel-specific TAP dialect where the diagnostics precede the results. The TAP spec isn't entirely clear about this, though, so I think it's the correct solution so as to keep interactive runs making sense. If the output _followed_ the result line in the spec-suggested YAML form, each test would dump all of its output at once instead of as it went, making debugging harder. This does, however, solve the recursive TAP output problem, as sub-tests will simply be prefixed by "# ". Parsing sub-tests becomes a simple problem of just removing the first two characters of a given top-level test's diagnostic output, and parsing the results. Note that the shell construct needed to both get an exit code from the first command in a pipe and still filter the pipe (to add the "# " prefix) uses a POSIX solution rather than the bash "pipefail" option which is not supported by dash. Since some test environments may have a very minimal set of utilities available, the new prefixing code will fall back to doing line-at-a-time prefixing if perl and/or stdbuf are not available. Signed-off-by: Kees Cook Signed-off-by: Shuah Khan

selftests: Add plan line and fix result line syntax

2019-04-25T19:14:45Z

The TAP version 13 spec requires a "plan" line, which has been missing. Since we always know how many tests we're going to run, emit the count on the plan line. This also fixes the result lines to remove the "1.." prefix which is against spec, and to mark skips with the correct "# SKIP" suffix. Signed-off-by: Kees Cook Signed-off-by: Shuah Khan

selftest: include stdio.h in kselftest.h

2019-02-13T15:34:47Z

While playing around with a way to skip the seccomp get_metadata test, I noticed that this header uses printf() without defining it, leading to, ../kselftest.h: In function ‘ksft_print_header’: ../kselftest.h:61:3: warning: implicit declaration of function ‘printf’ [-Wimplicit-function-declaration] printf("TAP version 13\n"); ^~~~~~ ../kselftest.h:61:3: warning: incompatible implicit declaration of built-in function ‘printf’ ../kselftest.h:61:3: note: include ‘’ or provide a declaration of ‘printf’ if user code doesn't also use printf. Signed-off-by: Tycho Andersen Acked-by: Kees Cook Signed-off-by: Shuah Khan

selftests: kselftest: Remove outdated comment

2018-08-27T20:07:46Z

Commit 3c07aaef6598 ("selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS") reverted commit 11867a77eb85 ("selftests: kselftest framework: change skip exit code to 0") but missed removing the comment which that commit added, so do that now. Signed-off-by: Thiago Jung Bauermann Signed-off-by: Shuah Khan (Samsung OSG)

selftests: kselftest: change KSFT_SKIP=4 instead of KSFT_PASS

2018-05-30T21:21:52Z

KSFT_SKIP points to KSFT_PASS resulting in reporting skipped tests as Passed, when test programs exit with KSFT_SKIP or call ksft_exit_skip(). If tests are skipped because of unmet dependencies and/or unsupported configuration, reporting them as passed leads to too many false positives. Fix it to return a skip code of 4 to clearly differentiate the skipped tests. Signed-off-by: Shuah Khan (Samsung OSG)

selftests: kselftest framework: add handling for TAP header level

2018-03-06T02:10:27Z

Introduce environment variable KSFT_TAP_LEVEL to avoid printing nested TAP headers for each test. lib.mk run_tests target prints TAP header before invoking the test program or test script. Tests need a way to suppress TAP headers if it is already printed out. This new environment variable adds a way for ksft_print_header() print TAP header only when KSFT_TAP_LEVEL isn't set. lib.mk run_tests and test program should print TAP header and set KSFT_TAP_LEVEL to avoid a second TAP header to be printed. selftests Makefile should export KSFT_TAP_LEVEL and add TAP Header echo to the run_kselftest.sh script from emit_tests target handling. Signed-off-by: Shuah Khan

selftests: kselftest.h: Add SPDX license identifier

2018-01-12T18:46:21Z

Replace GPL license statement with SPDX GPL-2.0 license identifier. Signed-off-by: Shuah Khan

selftests: kselftest framework: change skip exit code to 0

2017-08-25T16:02:00Z

When a test is skipped, instead of using a special exit code of 4, treat it as pass condition and use exit code of 0. It makes sense to treat skip as pass since the test couldn't be run as opposed to a failed test. Signed-off-by: Shuah Khan

selftests: kselftest framework: add error counter

2017-08-09T16:39:05Z

Some tests track errors in addition to test failures. Add ksft_error counter, ksft_get_error_cnt(), and ksft_test_result_error() API to get the counter value and print error message. Update ksft_print_cnts(), and ksft_test_num() to include error counter. Signed-off-by: Shuah Khan