kernel/arch/riscv/include/asm/smp.h, branch linux-rolling-stable

riscv: Use IPIs for remote cache/TLB flushes by default

2024-04-29T17:49:26Z

An IPI backend is always required in an SMP configuration, but an SBI implementation is not. For example, SBI will be unavailable when the kernel runs in M mode. For this reason, consider IPI delivery of cache and TLB flushes to be the base case, and any other implementation (such as the SBI remote fence extension) to be an optimization. Generally, if IPIs can be delivered without firmware assistance, they are assumed to be faster than SBI calls due to the SBI context switch overhead. However, when SBI is used as the IPI backend, then the context switch cost must be paid anyway, and performing the cache/TLB flush directly in the SBI implementation is more efficient than injecting an interrupt to S-mode. This is the only existing scenario where riscv_ipi_set_virq_range() is called with use_for_rfence set to false. sbi_ipi_init() already checks riscv_ipi_have_virq_range(), so it only calls riscv_ipi_set_virq_range() when no other IPI device is available. This allows moving the static key and dropping the use_for_rfence parameter. This decouples the static key from the irqchip driver probe order. Furthermore, the static branch only makes sense when CONFIG_RISCV_SBI is enabled. Optherwise, IPIs must be used. Add a fallback definition of riscv_use_sbi_for_rfence() which handles this case and removes the need to check CONFIG_RISCV_SBI elsewhere, such as in cacheflush.c. Reviewed-by: Anup Patel Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti Link: https://lore.kernel.org/r/20240327045035.368512-4-samuel.holland@sifive.com Signed-off-by: Palmer Dabbelt

riscv: Switch to hotplug core state synchronization

2023-05-15T11:44:59Z

Switch to the CPU hotplug core state tracking and synchronization mechanim. No functional change intended. Signed-off-by: Thomas Gleixner Signed-off-by: Peter Zijlstra (Intel) Acked-by: Palmer Dabbelt Tested-by: Michael Kelley Tested-by: Oleksandr Natalenko Tested-by: Helge Deller # parisc Tested-by: Guilherme G. Piccoli # Steam Deck Link: https://lore.kernel.org/r/20230512205256.916055844@linutronix.de

RISC-V: Allow marking IPIs as suitable for remote FENCEs

2023-04-08T10:26:24Z

To do remote FENCEs (i.e. remote TLB flushes) using IPI calls on the RISC-V kernel, we need hardware mechanism to directly inject IPI from the supervisor mode (i.e. RISC-V kernel) instead of using SBI calls. The upcoming AIA IMSIC devices allow direct IPI injection from the supervisor mode (i.e. RISC-V kernel). To support this, we extend the riscv_ipi_set_virq_range() function so that IPI provider (i.e. irqchip drivers can mark IPIs as suitable for remote FENCEs. Signed-off-by: Anup Patel Reviewed-by: Atish Patra Acked-by: Palmer Dabbelt Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/20230328035223.1480939-5-apatel@ventanamicro.com

RISC-V: Treat IPIs as normal Linux IRQs

2023-04-08T10:26:24Z

Currently, the RISC-V kernel provides arch specific hooks (i.e. struct riscv_ipi_ops) to register IPI handling methods. The stats gathering of IPIs is also arch specific in the RISC-V kernel. Other architectures (such as ARM, ARM64, and MIPS) have moved away from custom arch specific IPI handling methods. Currently, these architectures have Linux irqchip drivers providing a range of Linux IRQ numbers to be used as IPIs and IPI triggering is done using generic IPI APIs. This approach allows architectures to treat IPIs as normal Linux IRQs and IPI stats gathering is done by the generic Linux IRQ subsystem. We extend the RISC-V IPI handling as-per above approach so that arch specific IPI handling methods (struct riscv_ipi_ops) can be removed and the IPI handling is done through the Linux IRQ subsystem. Signed-off-by: Anup Patel Acked-by: Palmer Dabbelt Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/20230328035223.1480939-4-apatel@ventanamicro.com

riscv: kexec: Fixup crash_smp_send_stop without multi cores

2022-11-30T05:50:59Z

Current crash_smp_send_stop is the same as the generic one in kernel/panic and misses crash_save_cpu in percpu. This patch is inspired by 78fd584cdec0 ("arm64: kdump: implement machine_crash_shutdown()") and adds the same mechanism for riscv. Before this patch, test result: crash> help -r CPU 0: [OFFLINE] CPU 1: epc : ffffffff80009ff0 ra : ffffffff800b789a sp : ff2000001098bb40 gp : ffffffff815fca60 tp : ff60000004680000 t0 : 6666666666663c5b t1 : 0000000000000000 t2 : 666666666666663c s0 : ff2000001098bc90 s1 : ffffffff81600798 a0 : ff2000001098bb48 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000001 a4 : 0000000000000000 a5 : ff60000004690800 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ff2000001098bb48 s3 : ffffffff81093ec8 s4 : ffffffff816004ac s5 : 0000000000000000 s6 : 0000000000000007 s7 : ffffffff80e7f720 s8 : 00fffffffffff3f0 s9 : 0000000000000007 s10: 00aaaaaaaab98700 s11: 0000000000000001 t3 : ffffffff819a8097 t4 : ffffffff819a8097 t5 : ffffffff819a8098 t6 : ff2000001098b9a8 CPU 2: [OFFLINE] CPU 3: [OFFLINE] After this patch, test result: crash> help -r CPU 0: epc : ffffffff80003f34 ra : ffffffff808caa7c sp : ffffffff81403eb0 gp : ffffffff815fcb48 tp : ffffffff81413400 t0 : 0000000000000000 t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffff81403ec0 s1 : 0000000000000000 a0 : 0000000000000000 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000 a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ffffffff816001c8 s3 : ffffffff81600370 s4 : ffffffff80c32e18 s5 : ffffffff819d3018 s6 : ffffffff810e2110 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000080039eac s10: 0000000000000000 s11: 0000000000000000 t3 : 0000000000000000 t4 : 0000000000000000 t5 : 0000000000000000 t6 : 0000000000000000 CPU 1: epc : ffffffff80003f34 ra : ffffffff808caa7c sp : ff2000000068bf30 gp : ffffffff815fcb48 tp : ff6000000240d400 t0 : 0000000000000000 t1 : 0000000000000000 t2 : 0000000000000000 s0 : ff2000000068bf40 s1 : 0000000000000001 a0 : 0000000000000000 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000 a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ffffffff816001c8 s3 : ffffffff81600370 s4 : ffffffff80c32e18 s5 : ffffffff819d3018 s6 : ffffffff810e2110 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000080039ea8 s10: 0000000000000000 s11: 0000000000000000 t3 : 0000000000000000 t4 : 0000000000000000 t5 : 0000000000000000 t6 : 0000000000000000 CPU 2: epc : ffffffff80003f34 ra : ffffffff808caa7c sp : ff20000000693f30 gp : ffffffff815fcb48 tp : ff6000000240e900 t0 : 0000000000000000 t1 : 0000000000000000 t2 : 0000000000000000 s0 : ff20000000693f40 s1 : 0000000000000002 a0 : 0000000000000000 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000 a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ffffffff816001c8 s3 : ffffffff81600370 s4 : ffffffff80c32e18 s5 : ffffffff819d3018 s6 : ffffffff810e2110 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000080039eb0 s10: 0000000000000000 s11: 0000000000000000 t3 : 0000000000000000 t4 : 0000000000000000 t5 : 0000000000000000 t6 : 0000000000000000 CPU 3: epc : ffffffff8000a1e4 ra : ffffffff800b7bba sp : ff200000109bbb40 gp : ffffffff815fcb48 tp : ff6000000373aa00 t0 : 6666666666663c5b t1 : 0000000000000000 t2 : 666666666666663c s0 : ff200000109bbc90 s1 : ffffffff816007a0 a0 : ff200000109bbb48 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000001 a4 : 0000000000000000 a5 : ff60000002c61c00 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ff200000109bbb48 s3 : ffffffff810941a8 s4 : ffffffff816004b4 s5 : 0000000000000000 s6 : 0000000000000007 s7 : ffffffff80e7f7a0 s8 : 00fffffffffff3f0 s9 : 0000000000000007 s10: 00aaaaaaaab98700 s11: 0000000000000001 t3 : ffffffff819a8097 t4 : ffffffff819a8097 t5 : ffffffff819a8098 t6 : ff200000109bb9a8 Fixes: ad943893d5f1 ("RISC-V: Fixup schedule out issue in machine_crash_shutdown()") Reviewed-by: Xianting Tian Signed-off-by: Guo Ren Signed-off-by: Guo Ren Cc: Nick Kossifidis Link: https://lore.kernel.org/r/20221020141603.2856206-3-guoren@kernel.org Signed-off-by: Palmer Dabbelt

riscv: smp: Add 64bit hartid support on RV64

2022-07-19T23:39:09Z

The hartid can be a 64bit value on RV64 platforms. Modify the hartid parameter in riscv_hartid_to_cpuid() as unsigned long so that it can hold 64bit value on RV64 platforms. Signed-off-by: Sunil V L Reviewed-by: Heinrich Schuchardt Reviewed-by: Atish Patra Link: https://lore.kernel.org/r/20220527051743.2829940-4-sunilvl@ventanamicro.com Signed-off-by: Palmer Dabbelt

RISC-V: Do not use cpumask data structure for hartid bitmap

2022-01-20T17:27:22Z

Currently, SBI APIs accept a hartmask that is generated from struct cpumask. Cpumask data structure can hold upto NR_CPUs value. Thus, it is not the correct data structure for hartids as it can be higher than NR_CPUs for platforms with sparse or discontguous hartids. Remove all association between hartid mask and struct cpumask. Reviewed-by: Anup Patel (For Linux RISC-V changes) Acked-by: Anup Patel (For KVM RISC-V changes) Signed-off-by: Atish Patra Signed-off-by: Palmer Dabbelt

RISC-V: Use common riscv_cpuid_to_hartid_mask() for both SMP=y and SMP=n

2022-01-09T20:13:31Z

Use what is currently the SMP=y version of riscv_cpuid_to_hartid_mask() for both SMP=y and SMP=n to fix a build failure with KVM=m and SMP=n due to boot_cpu_hartid not being exported. This also fixes a second bug where the SMP=n version assumes the sole CPU in the system is in the incoming mask, which may not hold true in kvm_riscv_vcpu_sbi_ecall() if the KVM guest VM has multiple vCPUs (on a SMP=n system). Fixes: 1ef46c231df4 ("RISC-V: Implement new SBI v0.2 extensions") Reported-by: Adam Borowski Reviewed-by: Anup Patel Signed-off-by: Sean Christopherson Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt

riscv: remove cpu_stop()

2022-01-09T19:02:39Z

Except arch_cpu_idle_dead(), no users of this function. So remove cpu_stop() and fold its code into arch_cpu_idle_dead(). Signed-off-by: Jisheng Zhang Signed-off-by: Palmer Dabbelt

riscv: Constify sbi_ipi_ops

2021-04-26T15:25:10Z

Constify the sbi_ipi_ops so that it will be placed in the .rodata section. This will cause attempts to modify it to fail when strict page permissions are in place. Signed-off-by: Jisheng Zhang Signed-off-by: Palmer Dabbelt