kernel/tools/testing/selftests/vm/run_vmtests.sh, branch linux-6.2.y

selftests/vm: add KSM unmerge tests

2022-12-12T02:12:07Z

Patch series "mm/ksm: break_ksm() cleanups and fixes", v2. This series cleans up and fixes break_ksm(). In summary, we no longer use fake write faults to break COW but instead FAULT_FLAG_UNSHARE. Further, we move away from using follow_page() --- that we can hopefully remove completely at one point --- and use new walk_page_range_vma() instead. Fortunately, we can get rid of VM_FAULT_WRITE and FOLL_MIGRATION in common code now. Extend the existing ksm tests by an unmerge benchmark, and a some new unmerge tests. Also, add a selftest to measure MADV_UNMERGEABLE performance. In my setup (AMD Ryzen 9 3900X), running the KSM selftest to test unmerge performance on 2 GiB (taskset 0x8 ./ksm_tests -D -s 2048), this results in a performance degradation of ~6% -- 7% (old: ~5250 MiB/s, new: ~4900 MiB/s). I don't think we particularly care for now, but it's good to be aware of the implication. This patch (of 9): Let's add three unmerge tests (MADV_UNMERGEABLE unmerging all pages in the range). test_unmerge(): basic unmerge tests test_unmerge_discarded(): have some pte_none() entries in the range test_unmerge_uffd_wp(): protect the merged pages using uffd-wp ksm_tests.c currently contains a mixture of benchmarks and tests, whereby each test is carried out by executing the ksm_tests binary with specific parameters. Let's add new ksm_functional_tests.c that performs multiple, smaller functional tests all at once. Link: https://lkml.kernel.org/r/20221021101141.84170-1-david@redhat.com Link: https://lkml.kernel.org/r/20221021101141.84170-5-david@redhat.com Signed-off-by: David Hildenbrand Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Jason Gunthorpe Cc: John Hubbard Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Shuah Khan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

selftests/vm: enable running select groups of tests

2022-12-12T02:12:07Z

Our memory management kernel CI testing at Red Hat uses the VM selftests and we have run into two problems: First, our LTP tests overlap with the VM selftests. We want to avoid unhelpful redundancy in our testing practices. Second, we have observed the current run_vmtests.sh to report overall failure/ambiguous results in the case that a machine lacks the necessary hardware to perform one or more of the tests. E.g. ksm tests that require more than one numa node. We want to be able to run the vm selftests suitable to particular hardware. Add the ability to run one or more groups of vm tests via run_vmtests.sh instead of simply all-or-none in order to solve these problems. Preserve existing default behavior of running all tests when the script is invoked with no arguments. Documentation of test groups is included in the patch as follows: # ./run_vmtests.sh [ -h || --help ] usage: ./tools/testing/selftests/vm/run_vmtests.sh [ -h | -t ""] -t: specify specific categories to tests to run -h: display this message The default behavior is to run all tests. Alternatively, specific groups tests can be run by passing a string to the -t argument containing one or more of the following categories separated by spaces: - mmap tests for mmap(2) - gup_test tests for gup using gup_test interface - userfaultfd tests for userfaultfd(2) - compaction a test for the patch "Allow compaction of unevictable pages" - mlock tests for mlock(2) - mremap tests for mremap(2) - hugevm tests for very large virtual address space - vmalloc vmalloc smoke tests - hmm hmm smoke tests - madv_populate test memadvise(2) MADV_POPULATE_{READ,WRITE} options - memfd_secret test memfd_secret(2) - process_mrelease test process_mrelease(2) - ksm ksm tests that do not require >=2 NUMA nodes - ksm_numa ksm tests that require >=2 NUMA nodes - pkey memory protection key tests - soft_dirty test soft dirty page bit semantics - anon_cow test anonymous copy-on-write semantics example: ./run_vmtests.sh -t "hmm mmap ksm" Link: https://lkml.kernel.org/r/20221018231222.1884715-1-jsavitz@redhat.com Signed-off-by: Joel Savitz Cc: Joel Savitz Cc: Nico Pache Cc: Shuah Khan Signed-off-by: Andrew Morton

selftests/vm: anon_cow: prepare for non-anonymous COW tests

2022-11-30T23:58:56Z

Patch series "mm/gup: remove FOLL_FORCE usage from drivers (reliable R/O long-term pinning)". For now, we did not support reliable R/O long-term pinning in COW mappings. That means, if we would trigger R/O long-term pinning in MAP_PRIVATE mapping, we could end up pinning the (R/O-mapped) shared zeropage or a pagecache page. The next write access would trigger a write fault and replace the pinned page by an exclusive anonymous page in the process page table; whatever the process would write to that private page copy would not be visible by the owner of the previous page pin: for example, RDMA could read stale data. The end result is essentially an unexpected and hard-to-debug memory corruption. Some drivers tried working around that limitation by using "FOLL_FORCE|FOLL_WRITE|FOLL_LONGTERM" for R/O long-term pinning for now. FOLL_WRITE would trigger a write fault, if required, and break COW before pinning the page. FOLL_FORCE is required because the VMA might lack write permissions, and drivers wanted to make that working as well, just like one would expect (no write access, but still triggering a write access to break COW). However, that is not a practical solution, because (1) Drivers that don't stick to that undocumented and debatable pattern would still run into that issue. For example, VFIO only uses FOLL_LONGTERM for R/O long-term pinning. (2) Using FOLL_WRITE just to work around a COW mapping + page pinning limitation is unintuitive. FOLL_WRITE would, for example, mark the page softdirty or trigger uffd-wp, even though, there actually isn't going to be any write access. (3) The purpose of FOLL_FORCE is debug access, not access without lack of VMA permissions by arbitrarty drivers. So instead, make R/O long-term pinning work as expected, by breaking COW in a COW mapping early, such that we can remove any FOLL_FORCE usage from drivers and make FOLL_FORCE ptrace-specific (renaming it to FOLL_PTRACE). More details in patch #8. This patch (of 19): Originally, the plan was to have a separate tests for testing COW of non-anonymous (e.g., shared zeropage) pages. Turns out, that we'd need a lot of similar functionality and that there isn't a really good reason to separate it. So let's prepare for non-anon tests by renaming to "cow". Link: https://lkml.kernel.org/r/20221116102659.70287-1-david@redhat.com Link: https://lkml.kernel.org/r/20221116102659.70287-2-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Vlastimil Babka Cc: Alexander Shishkin Cc: Alexander Viro Cc: Alex Williamson Cc: Andrea Arcangeli Cc: Andy Walls Cc: Anton Ivanov Cc: Arnaldo Carvalho de Melo Cc: Arnd Bergmann Cc: Bernard Metzler Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Benvenuti Cc: Christian Gmeiner Cc: Christophe Leroy Cc: Christoph Hellwig Cc: Daniel Vetter Cc: Daniel Vetter Cc: Dave Hansen Cc: David Airlie Cc: David S. Miller Cc: Dennis Dalessandro Cc: "Eric W . Biederman" Cc: Greg Kroah-Hartman Cc: Hans Verkuil Cc: "H. Peter Anvin" Cc: Hugh Dickins Cc: Ingo Molnar Cc: Inki Dae Cc: Ivan Kokshaysky Cc: James Morris Cc: Jason Gunthorpe Cc: Jiri Olsa Cc: Johannes Berg Cc: John Hubbard Cc: Kees Cook Cc: Kentaro Takeda Cc: Krzysztof Kozlowski Cc: Kyungmin Park Cc: Leon Romanovsky Cc: Leon Romanovsky Cc: Linus Torvalds Cc: Lucas Stach Cc: Marek Szyprowski Cc: Mark Rutland Cc: Matthew Wilcox Cc: Matt Turner Cc: Mauro Carvalho Chehab Cc: Michael Ellerman Cc: Mike Kravetz Cc: Muchun Song Cc: Nadav Amit Cc: Namhyung Kim Cc: Nelson Escobar Cc: Nicholas Piggin Cc: Oded Gabbay Cc: Oleg Nesterov Cc: Paul Moore Cc: Peter Xu Cc: Peter Zijlstra Cc: Richard Henderson Cc: Richard Weinberger Cc: Russell King Cc: Serge Hallyn Cc: Seung-Woo Kim Cc: Shuah Khan Cc: Tetsuo Handa Cc: Thomas Bogendoerfer Cc: Thomas Gleixner Cc: Tomasz Figa Cc: Will Deacon Signed-off-by: Andrew Morton

selftests/vm: anon_cow: test COW handling of anonymous memory

2022-11-09T01:37:14Z

Patch series "selftests/vm: test COW handling of anonymous memory". This is my current set of tests for testing COW handling of anonymous memory, especially when interacting with GUP. I developed these tests while working on PageAnonExclusive and managed to clean them up just now. On current upstream Linux, all tests pass except the hugetlb tests that rely on vmsplice -- these tests should pass as soon as vmsplice properly uses FOLL_PIN instead of FOLL_GET. I'm working on additional tests for COW handling in private mappings, focusing on long-term R/O pinning e.g., of the shared zeropage, pagecache pages and KSM pages. These tests, however, will go into a different file. So this is everything I have regarding tests for anonymous memory. This patch (of 7): Let's start adding tests for our COW handling of anonymous memory. We'll focus on basic tests that we can achieve without additional libraries or gup_test extensions. We'll add THP and hugetlb tests separately. [david@redhat.com: s/size_t/ssize_t/ on `cur', `total', `transferred';] Link: https://lkml.kernel.org/r/51302b9e-dc69-d709-3214-f23868028555@redhat.com Link: https://lkml.kernel.org/r/20220927110120.106906-1-david@redhat.com Link: https://lkml.kernel.org/r/20220927110120.106906-2-david@redhat.com Signed-off-by: David Hildenbrand Cc: Andrea Arcangeli Cc: Christoph von Recklinghausen Cc: Don Dutile Cc: Jason Gunthorpe Cc: John Hubbard Cc: Mike Rapoport Cc: Nadav Amit Cc: Peter Xu Cc: Shuah Khan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

selftests/vm: drop mnt point for hugetlb in run_vmtests.sh

2022-11-09T01:37:11Z

After converting all the three relevant testcases (uffd, madvise, mremap) to use memfd, no test will need the hugetlb mount point anymore. Drop the code. Link: https://lkml.kernel.org/r/20221014144015.94039-1-peterx@redhat.com Signed-off-by: Peter Xu Reviewed-by: Axel Rasmussen Cc: Mike Kravetz Signed-off-by: Andrew Morton

selftests: vm: add /dev/userfaultfd test cases to run_vmtests.sh

2022-09-12T03:25:49Z

This new mode was recently added to the userfaultfd selftest. We want to exercise both userfaultfd(2) as well as /dev/userfaultfd, so add both test cases to the script. Link: https://lkml.kernel.org/r/20220808175614.3885028-6-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Reviewed-by: Shuah Khan Acked-by: Peter Xu Cc: Al Viro Cc: Dave Hansen Cc: Dmitry V. Levin Cc: Gleb Fotengauer-Malinovskiy Cc: Hugh Dickins Cc: Jan Kara Cc: Jonathan Corbet Cc: Mel Gorman Cc: Mike Kravetz Cc: Mike Rapoport Cc: Nadav Amit Cc: Shuah Khan Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Zhang Yi Cc: Mike Rapoport Signed-off-by: Andrew Morton

selftests: vm: add hugetlb_shared userfaultfd test to run_vmtests.sh

2022-09-12T03:25:48Z

Patch series "userfaultfd: add /dev/userfaultfd for fine grained access control", v7. Why not ...? ============ - Why not /proc/[pid]/userfaultfd? Two main points (additional discussion [1]): - /proc/[pid]/* files are all owned by the user/group of the process, and they don't really support chmod/chown. So, without extending procfs it doesn't solve the problem this series is trying to solve. - The main argument *for* this was to support creating UFFDs for remote processes. But, that use case clearly calls for CAP_SYS_PTRACE, so to support this we could just use the UFFD syscall as-is. - Why not use a syscall? Access to syscalls is generally controlled by capabilities. We don't have a capability which is used for userfaultfd access without also granting more / other permissions as well, and adding a new capability was rejected [2]. - It's possible a LSM could be used to control access instead, but I have some concerns. I don't think this approach would be as easy to use, particularly if we were to try to solve this with something heavyweight like SELinux. Maybe we could pursue adding a new LSM specifically for this user case, but it may be too narrow of a case to justify that. [1]: https://patchwork.kernel.org/project/linux-mm/cover/20220719195628.3415852-1-axelrasmussen@google.com/ [2]: https://lore.kernel.org/lkml/686276b9-4530-2045-6bd8-170e5943abe4@schaufler-ca.com/T/ This patch (of 5): This not being included was just a simple oversight. There are certain features (like minor fault support) which are only enabled on shared mappings, so without including hugetlb_shared we actually lose a significant amount of test coverage. Link: https://lkml.kernel.org/r/20220808175614.3885028-1-axelrasmussen@google.com Link: https://lkml.kernel.org/r/20220808175614.3885028-2-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Reviewed-by: Shuah Khan Reviewed-by: Peter Xu Cc: Al Viro Cc: Dave Hansen Cc: Dmitry V. Levin Cc: Gleb Fotengauer-Malinovskiy Cc: Hugh Dickins Cc: Jan Kara Cc: Jonathan Corbet Cc: Mel Gorman Cc: Mike Kravetz Cc: Mike Rapoport Cc: Nadav Amit Cc: Shuah Khan Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Zhang Yi Cc: Mike Rapoport Signed-off-by: Andrew Morton

selftests: add soft-dirty into run_vmtests.sh

2022-07-30T01:07:18Z

Link: https://lkml.kernel.org/r/20220725142048.30450-4-peterx@redhat.com Signed-off-by: Peter Xu Reviewed-by: David Hildenbrand Cc: Andrea Arcangeli Cc: Nadav Amit Signed-off-by: Andrew Morton

selftests/vm: add protection_keys tests to run_vmtests

2022-07-18T00:14:27Z

Add "protected_keys" tests to "run_vmtests.sh" would help run all VM related tests from a single shell script. [kalpana.shetty@amd.com: Shuah Khan's review comments incorporated, added -x executable check] Link: https://lkml.kernel.org/r/20220617202931.357-1-kalpana.shetty@amd.com Link: https://lkml.kernel.org/r/20220610090704.296-1-kalpana.shetty@amd.com Link: https://lkml.kernel.org/r/20220531102556.388-1-kalpana.shetty@amd.com Signed-off-by: Kalpana Shetty Cc: Shuah Khan Signed-off-by: Andrew Morton

selftests/vm: only run 128TBswitch with 5-level paging

2022-07-04T01:08:52Z

The test va_128TBswitch.c expects to be able to pass mmap an address hint and length that cross the address 1<<47. On x86_64, this is not possible without 5-level page tables, so the test fails. The test is already only run on 64-bit powerpc and x86_64 archs, but this patch adds an additional check on x86_64 that skips the test if PG_TABLE_LEVELS < 5. There is precedent for checking /proc/config.gz in selftests, e.g. in selftests/firmware. Running the tests produces the desired output: sudo make -C tools/testing/selftests TARGETS=vm run_tests --------------------------- running ./va_128TBswitch.sh --------------------------- ./va_128TBswitch.sh: PG_TABLE_LEVELS=4, must be >= 5 to run this test [SKIP] ------------------------------- [adam@wowsignal.io: restrict the check to x86_64] Link: https://lkml.kernel.org/r/20220628163654.337600-1-adam@wowsignal.io [adam@wowsignal.io: fix formatting issues, rename "die" to "fail"] Link: https://lkml.kernel.org/r/20220701163030.415735-1-adam@wowsignal.io Link: https://lkml.kernel.org/r/20220627163912.5581-1-adam@wowsignal.io Signed-off-by: Adam Sindelar Cc: Adam Sindelar Cc: David Vernet Cc: Aneesh Kumar K.V Signed-off-by: Andrew Morton