kernel/tools/testing/selftests/bpf/test_progs.h, branch linux-6.2.y

selftests/bpf: add generic BPF program tester-loader

2022-12-08T01:01:21Z

It's become a common pattern to have a collection of small BPF programs in one BPF object file, each representing one test case. On user-space side of such tests we maintain a table of program names and expected failure or success, along with optional expected verifier log message. This works, but each set of tests reimplement this mundane code over and over again, which is a waste of time for anyone trying to add a new set of tests. Furthermore, it's quite error prone as it's way too easy to miss some entries in these manually maintained test tables (as evidences by dynptr_fail tests, in which ringbuf_release_uninit_dynptr subtest was accidentally missed; this is fixed in next patch). So this patch implements generic test_loader, which accepts skeleton name and handles the rest of details: opens and loads BPF object file, making sure each program is tested in isolation. Optionally each test case can specify expected BPF verifier log message. In case of failure, tester makes sure to report verifier log, but it also reports verifier log in verbose mode unconditionally. Now, the interesting deviation from existing custom implementations is the use of btf_decl_tag attribute to specify expected-to-fail vs expected-to-succeed markers and, optionally, expected log message directly next to BPF program source code, eliminating the need to manually create and update table of tests. We define few macros wrapping btf_decl_tag with a convention that all values of btf_decl_tag start with "comment:" prefix, and then utilizing a very simple "just_some_text_tag" or "some_key_name=" pattern to define things like expected success/failure, expected verifier message, extra verifier log level (if necessary). This approach is demonstrated by next patch in which two existing sets of failure tests are converted. Tester supports both expected-to-fail and expected-to-succeed programs, though this patch set didn't convert any existing expected-to-succeed programs yet, as existing tests couple BPF program loading with their further execution through attach or test_prog_run. One way to allow testing scenarios like this would be ability to specify custom callback, executed for each successfully loaded BPF program. This is left for follow up patches, after some more analysis of existing test cases. This test_loader is, hopefully, a start of a test_verifier-like runner, but integrated into test_progs infrastructure. It will allow much better "user experience" of defining low-level verification tests that can take advantage of all the libbpf-provided nicety features on BPF side: global variables, declarative maps, etc. All while having a choice of defining it in C or as BPF assembly (through __attribute__((naked)) functions and using embedded asm), depending on what makes most sense in each particular case. This will be explored in follow up patches as well. Acked-by: John Fastabend Signed-off-by: Andrii Nakryiko Link: https://lore.kernel.org/r/20221207201648.2990661-1-andrii@kernel.org Signed-off-by: Alexei Starovoitov

selftests/bpf: Deduplicate write_sysctl() to test_progs.c

2022-09-09T17:40:45Z

This helper is needed in multiple tests. Instead of copying it over and over, better to deduplicate this helper to test_progs.c. test_progs.c is chosen over testing_helpers.c because of this helper's use of CHECK / ASSERT_*, and the CHECK was modified to use ASSERT_* so it does not rely on a duration variable. Suggested-by: Martin KaFai Lau Signed-off-by: YiFei Zhu Link: https://lore.kernel.org/r/9b4fc9a27bd52f771b657b4c4090fc8d61f3a6b5.1662682323.git.zhuyifei@google.com Signed-off-by: Martin KaFai Lau

selftests/bpf: Remove filtered subtests from output

2022-05-20T23:25:29Z

Currently filtered subtests show up in the output as skipped. Before: $ sudo ./test_progs -t log_fixup/missing_map #94 /1 log_fixup/bad_core_relo_trunc_none:SKIP #94 /2 log_fixup/bad_core_relo_trunc_partial:SKIP #94 /3 log_fixup/bad_core_relo_trunc_full:SKIP #94 /4 log_fixup/bad_core_relo_subprog:SKIP #94 /5 log_fixup/missing_map:OK #94 log_fixup:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED After: $ sudo ./test_progs -t log_fixup/missing_map #94 /5 log_fixup/missing_map:OK #94 log_fixup:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Mykola Lysenko Signed-off-by: Andrii Nakryiko Acked-by: Yonghong Song Link: https://lore.kernel.org/bpf/20220520061303.4004808-1-mykolal@fb.com

bpf/selftests: Add granular subtest output for prog_test

2022-04-28T02:03:58Z

Implement per subtest log collection for both parallel and sequential test execution. This allows granular per-subtest error output in the 'All error logs' section. Add subtest log transfer into the protocol during the parallel test execution. Move all test log printing logic into dump_test_log function. One exception is the output of test names when verbose printing is enabled. Move test name/result printing into separate functions to avoid repetition. Print all successful subtest results in the log. Print only failed test logs when test does not have subtests. Or only failed subtests' logs when test has subtests. Disable 'All error logs' output when verbose mode is enabled. This functionality was already broken and is causing confusion. Signed-off-by: Mykola Lysenko Signed-off-by: Andrii Nakryiko Link: https://lore.kernel.org/bpf/20220427041353.246007-1-mykolal@fb.com

selftests/bpf: Add libbpf's log fixup logic selftests

2022-04-26T22:41:46Z

Add tests validating that libbpf is indeed patching up BPF verifier log with CO-RE relocation details. Also test partial and full truncation scenarios. This test might be a bit fragile due to changing BPF verifier log format. If that proves to be frequently breaking, we can simplify tests or remove the truncation subtests. But for now it seems useful to test it in those conditions that are otherwise rarely occuring in practice. Also test CO-RE relo failure in a subprog as that excercises subprogram CO-RE relocation mapping logic which doesn't work out of the box without extra relo storage previously done only for gen_loader case. Signed-off-by: Andrii Nakryiko Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20220426004511.2691730-11-andrii@kernel.org

selftests/bpf: Refactor prog_tests logging and test execution

2022-04-19T04:22:13Z

This is a pre-req to add separate logging for each subtest in test_progs. Move all the mutable test data to the test_result struct. Move per-test init/de-init into the run_one_test function. Consolidate data aggregation and final log output in calculate_and_print_summary function. As a side effect, this patch fixes double counting of errors for subtests and possible duplicate output of subtest log on failures. Also, add prog_tests_framework.c test to verify some of the counting logic. As part of verification, confirmed that number of reported tests is the same before and after the change for both parallel and sequential test execution. Signed-off-by: Mykola Lysenko Signed-off-by: Andrii Nakryiko Link: https://lore.kernel.org/bpf/20220418222507.1726259-1-mykolal@fb.com

selftests/bpf: Drop duplicate max/min definitions

2022-04-11T15:18:09Z

Drop duplicate macros min() and MAX() definitions in prog_tests and use MIN() or MAX() in sys/param.h instead. Signed-off-by: Geliang Tang Signed-off-by: Daniel Borkmann Acked-by: Song Liu Link: https://lore.kernel.org/bpf/1ae276da9925c2de59b5bdc93b693b4c243e692e.1649462033.git.geliang.tang@suse.com

selftests/bpf: Improve by-name subtest selection logic in prog_tests

2022-04-11T03:08:20Z

Improve subtest selection logic when using -t/-a/-d parameters. In particular, more than one subtest can be specified or a combination of tests / subtests. -a send_signal -d send_signal/send_signal_nmi* - runs send_signal test without nmi tests -a send_signal/send_signal_nmi*,find_vma - runs two send_signal subtests and find_vma test -a 'send_signal*' -a find_vma -d send_signal/send_signal_nmi* - runs 2 send_signal test and find_vma test. Disables two send_signal nmi subtests -t send_signal -t find_vma - runs two *send_signal* tests and one *find_vma* test This will allow us to have granular control over which subtests to disable in the CI system instead of disabling whole tests. Also, add new selftest to avoid possible regression when changing prog_test test name selection logic. Signed-off-by: Mykola Lysenko Signed-off-by: Andrii Nakryiko Link: https://lore.kernel.org/bpf/20220409001750.529930-1-mykolal@fb.com

selftests/bpf: Define SYS_NANOSLEEP_KPROBE_NAME for aarch64

2022-04-04T21:57:29Z

attach_probe selftest fails on aarch64 with `failed to create kprobe 'sys_nanosleep+0x0' perf event: No such file or directory`. This is because, like on several other architectures, nanosleep has a prefix. Signed-off-by: Ilya Leoshkevich Signed-off-by: Andrii Nakryiko Tested-by: Alan Maguire Link: https://lore.kernel.org/bpf/20220404142101.27900-1-iii@linux.ibm.com

selftests/bpf: Add parallelism to test_progs

2021-10-08T21:37:56Z

This patch adds "-j" mode to test_progs, executing tests in multiple process. "-j" mode is optional, and works with all existing test selection mechanism, as well as "-v", "-l" etc. In "-j" mode, main process use UDS/SEQPACKET to communicate to each forked worker, commanding it to run tests and collect logs. After all tests are finished, a summary is printed. main process use multiple competing threads to dispatch work to worker, trying to keep them all busy. The test status will be printed as soon as it is finished, if there are error logs, it will be printed after the final summary line. By specifying "--debug", additional debug information on server/worker communication will be printed. Example output: > ./test_progs -n 15-20 -j [ 12.801730] bpf_testmod: loading out-of-tree module taints kernel. Launching 8 workers. #20 btf_split:OK #16 btf_endian:OK #18 btf_module:OK #17 btf_map_in_map:OK #19 btf_skc_cls_ingress:OK #15 btf_dump:OK Summary: 6/20 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Yucong Sun Signed-off-by: Andrii Nakryiko Link: https://lore.kernel.org/bpf/20211006185619.364369-2-fallentree@fb.com