kernel/tools/testing/selftests/tc-testing/tdc.py, branch linux-6.2.y

selftests: tc-testing: Add matchJSON to tdc

2022-10-27T03:22:33Z

This allows the use of a matchJSON field in tests to match against JSON output from the command under test, if that command outputs JSON. You specify what you want to match against as a JSON array or object in the test's matchJSON field. You can leave out any fields you don't want to match against that are present in the output and they will be skipped. An example matchJSON value would look like this: "matchJSON": [ { "Value": { "neighIP": { "family": 4, "addr": "AQIDBA==", "width": 32 }, "nsflags": 142, "ncflags": 0, "LLADDR": "ESIzRFVm" } } ] The real output from the command under test might have some extra fields that we don't care about for matching, and since we didn't include them in our matchJSON value, those fields will not be attempted to be matched. If everything we included above has the same values as the real command output, the test will pass. The matchJSON field's type must be the same as the command output's type, otherwise the test will fail. So if the command outputs an array, then the value of matchJSON must also be an array. If matchJSON is an array, it must not contain more elements than the command output's array, otherwise the test will fail. Signed-off-by: Jeremy Carter Signed-off-by: Victor Nogueira Acked-by: Jamal Hadi Salim Link: https://lore.kernel.org/r/20221024111603.2185410-1-victor@mojatatu.com Signed-off-by: Jakub Kicinski

selftests/tc-testing: add exit code

2021-12-03T11:46:41Z

Mark the summary result as FAIL to prevent from confusing the selftest framework if some of them are failed. Previously, the selftest framework always treats it as *ok* even though some of them are failed actually. That's because the script tdc.sh always return 0. # All test results: # # 1..97 # ok 1 83be - Create FQ-PIE with invalid number of flows # ok 2 8b6e - Create RED with no flags [...snip] # ok 6 5f15 - Create RED with flags ECN, harddrop # ok 7 53e8 - Create RED with flags ECN, nodrop # ok 8 d091 - Fail to create RED with only nodrop flag # ok 9 af8e - Create RED with flags ECN, nodrop, harddrop # not ok 10 ce7d - Add mq Qdisc to multi-queue device (4 queues) # Could not match regex pattern. Verify command output: # qdisc mq 1: root # qdisc fq_codel 0: parent 1:4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 # qdisc fq_codel 0: parent 1:3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 [...snip] # ok 96 6979 - Change quantum of a strict ETS band # ok 97 9a7d - Change ETS strict band without quantum # # # # ok 1 selftests: tc-testing: tdc.sh <<< summary result CC: Philip Li Reported-by: kernel test robot Signed-off-by: Li Zhijian Acked-by: Davide Caratti Acked-by: Jamal Hadi Salim Signed-off-by: David S. Miller

tc-testing: remove duplicate code in tdc.py

2020-04-08T01:38:55Z

In set_operation_mode() function remove duplicated check for args.list parameter, which is already done one line before. Signed-off-by: Roman Mashak Signed-off-by: David S. Miller

tc-testing: Clarify the use of tdc's -d option

2019-07-31T22:54:34Z

The -d command line argument to tdc requires the name of a physical device on the system where the tests will be run. If -d has not been used, tdc will skip tests that require a physical device. This patch is intended to better document what the -d option does and how it is used. Signed-off-by: Lucas Bates Signed-off-by: David S. Miller

tc-testing: Allow tdc plugins to see test case data

2019-07-09T21:07:20Z

Instead of only passing the test case name and ID, pass the entire current test case down to the plugins. This change allows plugins to start accepting commands and directives from the test cases themselves, for greater flexibility in testing. Signed-off-by: Lucas Bates Signed-off-by: David S. Miller

tc-testing: Restore original behaviour for namespaces in tdc

2019-06-25T19:57:29Z

This patch restores the original behaviour for tdc prior to the introduction of the plugin system, where the network namespace functionality was split from the main script. It introduces the concept of required plugins for testcases, and will automatically load any plugin that isn't already enabled when said plugin is required by even one testcase. Additionally, the -n option for the nsPlugin is deprecated so the default action is to make use of the namespaces. Instead, we introduce -N to not use them, but still create the veth pair. buildebpfPlugin's -B option is also deprecated. If a test cases requires the features of a specific plugin in order to pass, it should instead include a new key/value pair describing plugin interactions: "plugins": { "requires": "buildebpfPlugin" }, A test case can have more than one required plugin: a list can be inserted as the value for 'requires'. Signed-off-by: Lucas Bates Acked-by: Davide Caratti Tested-by: Nicolas Dichtel Signed-off-by: David S. Miller

tc-testing: Allow test cases to be skipped

2019-03-02T07:05:06Z

By adding a check for an optional key/value pair to the test case data, individual test cases may be skipped to prevent tdc from aborting a test run due to setup or teardown failure. If a test case is skipped, it will still appear in the results output to allow for a consistent number of executed tests in each run. However, the test will be marked as skipped. This support for skipping extends to any plugins that may generate additional results for each executed test. Signed-off-by: Lucas Bates Signed-off-by: David S. Miller

tc-testing: Implement the TdcResults module in tdc

2018-12-08T00:39:03Z

In tdc and the valgrind plugin, begin using the TdcResults module to track executed test cases. Signed-off-by: Lucas Bates Signed-off-by: David S. Miller

tc-testing: Add command timeout feature to tdc

2018-12-08T00:39:03Z

Using an attribute set in the tdc_config.py file, limit the amount of time tdc will wait for an executed command to complete and prevent the script from hanging entirely. This timeout will be applied to all executed commands. Signed-off-by: Lucas Bates Signed-off-by: David S. Miller

tc-testing: tdc.py: Guard against lack of returncode in executed command

2018-11-18T05:54:53Z

Add some defensive coding in case one of the subprocesses created by tdc returns nothing. If no object is returned from exec_cmd, then tdc will halt with an unhandled exception. Signed-off-by: Brenda J. Butler Signed-off-by: Lucas Bates Signed-off-by: David S. Miller