kernel/arch/s390/include/asm/checksum.h, branch linux-6.2.y

s390/checksum: always use cksm instruction

2023-05-11T14:11:28Z

[ Upstream commit e42ac7789df64120d7d3d57433dfc9f37ec0cb99 ] Commit dfe843dce775 ("s390/checksum: support GENERIC_CSUM, enable it for KASAN") switched s390 to use the generic checksum functions, so that KASAN instrumentation also works checksum functions by avoiding architecture specific inline assemblies. There is however the problem that the generic csum_partial() function returns a 32 bit value with a 16 bit folded checksum, while the original s390 variant does not fold to 16 bit. This in turn causes that the ipib_checksum in lowcore contains different values depending on kernel config options. The ipib_checksum is used by system dumpers to verify if pointers in lowcore point to valid data. Verification is done by comparing checksum values. The system dumpers still use 32 bit checksum values which are not folded, and therefore the checksum verification fails (incorrectly). Symptom is that reboot after dump does not work anymore when a KASAN instrumented kernel is dumped. Fix this by not using the generic checksum implementation. Instead add an explicit kasan_check_read() so that KASAN knows about the read access from within the inline assembly. Reported-by: Alexander Egorenkov Fixes: dfe843dce775 ("s390/checksum: support GENERIC_CSUM, enable it for KASAN") Tested-by: Alexander Egorenkov Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik Signed-off-by: Sasha Levin

s390/checksum: support GENERIC_CSUM, enable it for KASAN

2022-12-02T17:25:04Z

This is the s390 variant of commit d911c67e10b4 ("x86: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for KASAN/KMSAN"). Even though most of the s390 specific checksum code is written in C there is still the csum_partial() inline assembly which could prevent KASAN and KMSAN from seeing all memory accesses. Therefore switch to GENERIC_CSUM if KASAN is enabled just like x86. Reviewed-by: Vasily Gorbik Signed-off-by: Heiko Carstens Signed-off-by: Alexander Gordeev

s390/checksum: use register pair instead of register asm

2021-06-18T14:41:21Z

Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

Merge tag 's390-5.10-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux

2020-10-16T19:36:38Z

Pull s390 updates from Vasily Gorbik: - Remove address space overrides using set_fs() - Convert to generic vDSO - Convert to generic page table dumper - Add ARCH_HAS_DEBUG_WX support - Add leap seconds handling support - Add NVMe firmware-assisted kernel dump support - Extend NVMe boot support with memory clearing control and addition of kernel parameters - AP bus and zcrypt api code rework. Add adapter configure/deconfigure interface. Extend debug features. Add failure injection support - Add ECC secure private keys support - Add KASan support for running protected virtualization host with 4-level paging - Utilize destroy page ultravisor call to speed up secure guests shutdown - Implement ioremap_wc() and ioremap_prot() with MIO in PCI code - Various checksum improvements - Other small various fixes and improvements all over the code * tag 's390-5.10-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux: (85 commits) s390/uaccess: fix indentation s390/uaccess: add default cases for __put_user_fn()/__get_user_fn() s390/zcrypt: fix wrong format specifications s390/kprobes: move insn_page to text segment s390/sie: fix typo in SIGP code description s390/lib: fix kernel doc for memcmp() s390/zcrypt: Introduce Failure Injection feature s390/zcrypt: move ap_msg param one level up the call chain s390/ap/zcrypt: revisit ap and zcrypt error handling s390/ap: Support AP card SCLP config and deconfig operations s390/sclp: Add support for SCLP AP adapter config/deconfig s390/ap: add card/queue deconfig state s390/ap: add error response code field for ap queue devices s390/ap: split ap queue state machine state from device state s390/zcrypt: New config switch CONFIG_ZCRYPT_DEBUG s390/zcrypt: introduce msg tracking in zcrypt functions s390/startup: correct early pgm check info formatting s390: remove orphaned extern variables declarations s390/kasan: make sure int handler always run with DAT on s390/ipl: add support to control memory clearing for nvme re-IPL ...

s390/checksum: coding style changes

2020-08-26T16:47:20Z

Add some coding style changes which hopefully make the code look a bit less odd. Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/checksum: have consistent calculations

2020-08-26T16:47:20Z

Use "|" instead of "+" within csum_fold() for consistency reasons, like in the rest of the file. Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/checksum: make ip_fast_csum() faster

2020-08-26T16:47:20Z

Convert ip_fast_csum() so it doesn't call csum_partial(), but instead open code the checksum calculation. The problem with csum_partial() is that it makes use of the cksm instruction, which has high startup costs and therefore is only very fast if used on larger memory regions. IPv4 headers however are small in size (5-16 32-bit words). The open coded variant calculates the checksum in ~30% of the time compared to the old variant (z14, march=z196). Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/checksum: rewrite csum_tcpudp_nofold()

2020-08-26T16:47:20Z

Rewrite csum_tcpudp_nofold() so that the generated code will not contain branches. The old implementation was also optimized for machines which came with "add logical with carry" instructions, however the compiler doesn't generate them anymore. This is most likely because those instructions are slower. However with the old code the compiler generates a lot of branches, which isn't too helpful usually. Therefore rewrite the code. In a tight loop this doesn't make any difference since the branch prediction unit does its job. Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/checksum: provide csum_ipv6_magic()

2020-08-26T16:47:20Z

This implementation needs only ~30% of the time to calculate the checksum compared to the generic variant. In addition the compiler also generates only ~30% of the instructions compared to the generic variant (on z14, compiled with march=z196). Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

unify generic instances of csum_partial_copy_nocheck()

2020-08-20T19:45:14Z

quite a few architectures have the same csum_partial_copy_nocheck() - simply memcpy() the data and then return the csum of the copy. hexagon, parisc, ia64, s390, um: explicitly spelled out that way. arc, arm64, csky, h8300, m68k/nommu, microblaze, mips/GENERIC_CSUM, nds32, nios2, openrisc, riscv, unicore32: end up picking the same thing spelled out in lib/checksum.h (with varying amounts of perversions along the way). everybody else (alpha, arm, c6x, m68k/mmu, mips/!GENERIC_CSUM, powerpc, sh, sparc, x86, xtensa) have non-generic variants. For all except c6x the declaration is in their asm/checksum.h. c6x uses the wrapper from asm-generic/checksum.h that would normally lead to the lib/checksum.h instance, but in case of c6x we end up using an asm function from arch/c6x instead. Screw that mess - have architectures with private instances define _HAVE_ARCH_CSUM_AND_COPY in their asm/checksum.h and have the default one right in net/checksum.h conditional on _HAVE_ARCH_CSUM_AND_COPY *not* defined. Signed-off-by: Al Viro