kernel/tools/testing/selftests/rseq/rseq-x86.h, branch linux-rolling-stable

selftests/rseq: Use rseq_unqual_scalar_typeof in macros

2023-08-08T20:42:12Z

Use rseq_unqual_scalar_typeof() rather than typeof() in macros to remove the volatile qualifier (if there is one in the input argument), thus generating better assembly code in those scenarios. Also add extra brackets around the "p" parameter in RSEQ_READ_ONCE(), RSEQ_WRITE_ONCE(), and rseq_unqual_scalar_typeof() across architectures to preserve expectations of operator priority. Here is an example that shows how operator priority may be an issue with missing parentheses: #define m(p) \ do { \ __typeof__(*p) v = 0; \ } while (0) void fct(unsigned long long *p1) { m(p1 + 1); /* works */ m(1 + p1); /* broken */ } Signed-off-by: Mathieu Desnoyers Cc: Peter Zijlstra Signed-off-by: Shuah Khan

selftests/rseq: x86: Template memory ordering and percpu access mode

2022-12-27T11:52:12Z

Introduce a rseq-x86-bits.h template header which is internally included to generate the static inline functions covering: - relaxed and release memory ordering, - per-cpu-id and per-mm-cid per-cpu data access. This introduces changes to the rseq.h selftests API which require to update the rseq selftest programs. Similar API/templating changes need to be done for other architectures. Signed-off-by: Mathieu Desnoyers Signed-off-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/20221122203932.231377-12-mathieu.desnoyers@efficios.com

selftests/rseq: Remove RSEQ_SKIP_FASTPATH code

2022-12-27T11:52:12Z

This code is not currently build by the test Makefile, adds complexity, and is not overall useful considering that the abort handling loops to retry the fast-path. Signed-off-by: Mathieu Desnoyers Signed-off-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/20221122203932.231377-10-mathieu.desnoyers@efficios.com

selftests/rseq: Change type of rseq_offset to ptrdiff_t

2022-02-11T22:30:08Z

Just before the 2.35 release of glibc, the __rseq_offset userspace ABI was changed from int to ptrdiff_t. Adapt to this change in the kernel selftests. Signed-off-by: Mathieu Desnoyers Signed-off-by: Peter Zijlstra (Intel) Link: https://sourceware.org/pipermail/libc-alpha/2022-February/136024.html

selftests/rseq: x86-32: use %gs segment selector for accessing rseq thread area

2022-02-02T12:11:37Z

Rather than use rseq_get_abi() and pass its result through a register to the inline assembler, directly access the per-thread rseq area through a memory reference combining the %gs segment selector, the constant offset of the field in struct rseq, and the rseq_offset value (in a register). Signed-off-by: Mathieu Desnoyers Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20220124171253.22072-16-mathieu.desnoyers@efficios.com

selftests/rseq: x86-64: use %fs segment selector for accessing rseq thread area

2022-02-02T12:11:37Z

Rather than use rseq_get_abi() and pass its result through a register to the inline assembler, directly access the per-thread rseq area through a memory reference combining the %fs segment selector, the constant offset of the field in struct rseq, and the rseq_offset value (in a register). Signed-off-by: Mathieu Desnoyers Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20220124171253.22072-15-mathieu.desnoyers@efficios.com

selftests/rseq: Fix: work-around asm goto compiler bugs

2022-02-02T12:11:37Z

gcc and clang each have their own compiler bugs with respect to asm goto. Implement a work-around for compiler versions known to have those bugs. gcc prior to 4.8.2 miscompiles asm goto. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58670 gcc prior to 8.1.0 miscompiles asm goto at O1. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103908 clang prior to version 13.0.1 miscompiles asm goto at O2. https://github.com/llvm/llvm-project/issues/52735 Work around these issues by adding a volatile inline asm with memory clobber in the fallthrough after the asm goto and at each label target. Emit this for all compilers in case other similar issues are found in the future. Signed-off-by: Mathieu Desnoyers Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20220124171253.22072-14-mathieu.desnoyers@efficios.com

selftests/rseq: Fix warnings about #if checks of undefined tokens

2022-02-02T12:11:36Z

Signed-off-by: Mathieu Desnoyers Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20220124171253.22072-12-mathieu.desnoyers@efficios.com

selftests/rseq: Fix ppc32 offsets by using long rather than off_t

2022-02-02T12:11:36Z

The semantic of off_t is for file offsets. We mean to use it as an offset from a pointer. We really expect it to fit in a single register, and not use a 64-bit type on 32-bit architectures. Fix runtime issues on ppc32 where the offset is always 0 due to inconsistency between the argument type (off_t -> 64-bit) and type expected by the inline assembler (32-bit). Signed-off-by: Mathieu Desnoyers Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20220124171253.22072-11-mathieu.desnoyers@efficios.com

selftests/rseq: Introduce rseq_get_abi() helper

2022-02-02T12:11:35Z

This is done in preparation for the selftest uplift to become compatible with glibc-2.35. glibc-2.35 exposes the rseq per-thread data in the TCB, accessible at an offset from the thread pointer, rather than through an actual Thread-Local Storage (TLS) variable, as the kernel selftests initially expected. Introduce a rseq_get_abi() helper, initially using the __rseq_abi TLS variable, in preparation for changing this userspace ABI for one which is compatible with glibc-2.35. Note that the __rseq_abi TLS and glibc-2.35's ABI for per-thread data cannot actively coexist in a process, because the kernel supports only a single rseq registration per thread. Signed-off-by: Mathieu Desnoyers Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20220124171253.22072-6-mathieu.desnoyers@efficios.com