Skip to contentSkip to conversation

riscv: add support for SBI Supervisor Software Events

PATCH Superseded Clément Léger 7 participants 21 messages Sep 8, 2025 – Oct 29, 2025

Subsystems (4)

v7 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v7 cover riscv: add support for SBI Supervisor Software Events

The SBI Supervisor Software Events (SSE) extensions provides a mechanism to inject software events from an SBI implementation to supervisor software such that it preempts all other supervisor level traps and interrupts. This extension is introduced by the SBI v3.0 specification[1].

Various events are defined and can be send asynchronously to supervisor software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well as platform specific events. Events can be either local (per-hart) or global. Events can be nested on top of each other based on priority and can interrupt the kernel at any time.

First patch adds the SSE definitions. Second one adds support for SSE at arch level (entry code and stack allocations) and third one at driver level. Finally, the last patch add support for SSE events in the SBI PMU driver. Additional testing for that part is highly welcomed since there are a lot of possible path that needs to be exercised.

Amongst the specific points that needs to be handle is the interruption at any point of the kernel execution and more specifically at the beginning of exception handling. Due to the fact that the exception entry implementation uses the SCRATCH CSR as both the current task struct and as the temporary register to switch the stack and save register, it is difficult to reliably get the current task struct if we get interrupted at this specific moment (ie, it might contain 0, the task pointer or tp). A fixup-like mechanism is not possible due to the nested nature of SSE which makes it really hard to obtain the original interruption site. In order to retrieve the task in a reliable manner, add an additional __sse_entry_task per_cpu array which stores the current task. Ideally, we would need to modify the way we retrieve/store the current task in exception handling so that it does not depend on the place where it's interrupted.

Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are handled and does not adds any overhead to existing code. Moreover, it provides "true" NMI-like interrupts which can interrupt the kernel at any time (even in exception handling). This is particularly crucial for RAS errors which needs to be handled as fast as possible to avoid any fault propagation.

A test suite is available as a separate kselftest module. In order to build it, you can use the following command:

$ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests

Then load the module using:

$ sh run_sse_test.sh

A KVM SBI SSE extension implementation is available at [2].

---

Changes in v7:
- Check return values of sse_on_each_cpu()
- Fix typos in commit
- Rename SBI_SSE_EVENT_SIGNAL to SBI_SSE_EVENT_INJECT
- Rename SBI_SSE_EVENT_HART_UNMASK/MASK to SBI_SSE_HART_UNMASK/MASK
- Add tlb flush for vmap stack to avoid taking exception during sse handler upon stack access. (Alex)
- Move some assembly instruction to slow path
- Renamed sse.c to sbi_sse.c, ditto for other files
- Renamed RISCV_SSE to RISCV_SBI_SSE
- Renamed sse_event_handler to sse_event_handler_fn
- Put ifdef around sse_evt in PMU SBI driver

Changes in v6:
- Fix comment in assembly argument
- Check hart id to be the expected one in order to skip CPU id matching in sse assembly.

Changes in v5:
- Added a SSE test module in kselftests
- Removed an unused variable
- Applied checkpatch.pl --strict and fix all errors
- Use scope_guard(cpus_read_lock) instead of manual cpus_read_lock()
- Fix wrong variable returned in sse_get_event
- Remove useless init of events list
- Remove useless empty for loop on cpus
- Set sse_available as __ro_after_init
- Changed a few pr_debug to pr_warn
- Fix event enabled stated updated in case of failure
- Change no_lock to nolock
- Rename attr_buf to attr
- renamed sse_get_event_phys() to sse_event_get_attr_phys() and removed the second argument
- Simplify return value in sse_event_attr_set_nolock()
- Remove while loop(-EINVAL) for event cpu set call
- Renamed interrupted_state_phys to interrupted_phys
- Use scoped_guards/guard for sse_mutex
- Remove useless struct forward declaration in sse.h
- Add more explanations as to why we set SIE bit in IP
- Unconditionnally set SIE in SIP
- Move SSE_STACK_SIZE adjustement in sse_stack_alloc/free()
- Replace move instructions with mv
- Rename NR_CPUS asm symbol to ASM_NR_CPUS
- Restore SSTATUS first in sse_entry return path so that it works for double trap without any modification later.
- Implement proper per cpu revert if enable/register fails

Changes in v4:
- Fix a bug when using per_cpu ptr for local event (Andrew)
- Add sse_event_disable/enable_local()
- Add pmu_disable/pmu_enable() to disable/enable SSE event
- Update event ID description according to the latest spec
- Fix comment about arguments in handle_sse()
- Added Himanchu as a SSE reviewer
- Used SYM_DATA_*() macros instead of hardcoded labels
- Invoke softirqs only if not returning to kernel with irqs disabled
- Remove invalid state check for write attribute function.
- Remove useless bneq statement in sse_entry.S

Changes in v3:
- Split arch/driver support
- Fix potential register failure reporting
- Set a few pr_err as pr_debug
- Allow CONFIG_RISCV_SSE to be disabled
- Fix build without CONFIG_RISCV_SSE
- Remove fixup-like mechanism and use a per-cpu array
- Fixed SSCRATCH being corrupted when interrupting the kernel in early exception path.
- Split SSE assembly from entry.S
- Add Himanchu SSE mask/unmask and runtime PM support.
- Disable user memory access/floating point/vector in SSE handler
- Rebased on master

v2: https://lore.kernel.org/linux-riscv/[email protected]/

Changes in v2:
- Implemented specification v2
- Fix various error handling cases
- Added shadow stack support

v1: https://lore.kernel.org/linux-riscv/[email protected]/

Clément Léger (5):
riscv: add SBI SSE extension definitions
riscv: add support for SBI Supervisor Software Events extension
drivers: firmware: add riscv SSE support
perf: RISC-V: add support for SSE event selftests/riscv: add SSE test module

 MAINTAINERS                                   |  15 +
 arch/riscv/include/asm/asm.h                  |  14 +-
 arch/riscv/include/asm/sbi.h                  |  61 ++
 arch/riscv/include/asm/scs.h                  |   7 +
 arch/riscv/include/asm/sse.h                  |  47 ++
 arch/riscv/include/asm/switch_to.h            |  14 +
 arch/riscv/include/asm/thread_info.h          |   1 +
 arch/riscv/kernel/Makefile                    |   1 +
 arch/riscv/kernel/asm-offsets.c               |  14 +
 arch/riscv/kernel/sbi_sse.c                   | 174 +++++
 arch/riscv/kernel/sbi_sse_entry.S             | 178 +++++
 drivers/firmware/Kconfig                      |   1 +
 drivers/firmware/Makefile                     |   1 +
 drivers/firmware/riscv/Kconfig                |  15 +
 drivers/firmware/riscv/Makefile               |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c        | 701 ++++++++++++++++++
 drivers/perf/Kconfig                          |  10 +
 drivers/perf/riscv_pmu.c                      |  23 +
 drivers/perf/riscv_pmu_sbi.c                  |  71 +-
 include/linux/perf/riscv_pmu.h                |   5 +
 include/linux/riscv_sbi_sse.h                 |  57 ++
 tools/testing/selftests/riscv/Makefile        |   2 +-
 tools/testing/selftests/riscv/sse/Makefile    |   5 +
 .../selftests/riscv/sse/module/Makefile       |  16 +
 .../riscv/sse/module/riscv_sse_test.c         | 513 +++++++++++++
 .../selftests/riscv/sse/run_sse_test.sh       |  44 ++
 26 files changed, 1979 insertions(+), 14 deletions(-)
 create mode 100644 arch/riscv/include/asm/sse.h
 create mode 100644 arch/riscv/kernel/sbi_sse.c
 create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h
 create mode 100644 tools/testing/selftests/riscv/sse/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
 create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh
Signature · 1 line
2.43.0
#1 lore raw

PATCH v7 1/5 riscv: add SBI SSE extension definitions

Add needed definitions for SBI Supervisor Software Events extension [1]. This extension enables the SBI to inject events into supervisor software much like ARM SDEI.

[1] https://lists.riscv.org/g/tech-prs/message/515

---
 arch/riscv/include/asm/sbi.h | 61 ++++++++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)
arch/riscv/include/asm/sbi.h +61 -0
index 341e74238aa0..874cc1d7603a 100644
@@ -36,6 +36,7 @@ enum sbi_ext_id {
 	SBI_EXT_STA = 0x535441,
 	SBI_EXT_NACL = 0x4E41434C,
 	SBI_EXT_FWFT = 0x46574654,
+	SBI_EXT_SSE = 0x535345,
 
 	/* Experimentals extensions must lie within this range */
 	SBI_EXT_EXPERIMENTAL_START = 0x08000000,
@@ -430,6 +431,66 @@ enum sbi_fwft_feature_t {
 
 #define SBI_FWFT_SET_FLAG_LOCK			BIT(0)
 
+enum sbi_ext_sse_fid {
+	SBI_SSE_EVENT_ATTR_READ = 0,
+	SBI_SSE_EVENT_ATTR_WRITE,
+	SBI_SSE_EVENT_REGISTER,
+	SBI_SSE_EVENT_UNREGISTER,
+	SBI_SSE_EVENT_ENABLE,
+	SBI_SSE_EVENT_DISABLE,
+	SBI_SSE_EVENT_COMPLETE,
+	SBI_SSE_EVENT_INJECT,
+	SBI_SSE_HART_UNMASK,
+	SBI_SSE_HART_MASK,
+};
+
+enum sbi_sse_state {
+	SBI_SSE_STATE_UNUSED     = 0,
+	SBI_SSE_STATE_REGISTERED = 1,
+	SBI_SSE_STATE_ENABLED    = 2,
+	SBI_SSE_STATE_RUNNING    = 3,
+};
+
+/* SBI SSE Event Attributes. */
+enum sbi_sse_attr_id {
+	SBI_SSE_ATTR_STATUS		= 0x00000000,
+	SBI_SSE_ATTR_PRIO		= 0x00000001,
+	SBI_SSE_ATTR_CONFIG		= 0x00000002,
+	SBI_SSE_ATTR_PREFERRED_HART	= 0x00000003,
+	SBI_SSE_ATTR_ENTRY_PC		= 0x00000004,
+	SBI_SSE_ATTR_ENTRY_ARG		= 0x00000005,
+	SBI_SSE_ATTR_INTERRUPTED_SEPC	= 0x00000006,
+	SBI_SSE_ATTR_INTERRUPTED_FLAGS	= 0x00000007,
+	SBI_SSE_ATTR_INTERRUPTED_A6	= 0x00000008,
+	SBI_SSE_ATTR_INTERRUPTED_A7	= 0x00000009,
+
+	SBI_SSE_ATTR_MAX		= 0x0000000A
+};
+
+#define SBI_SSE_ATTR_STATUS_STATE_OFFSET	0
+#define SBI_SSE_ATTR_STATUS_STATE_MASK		0x3
+#define SBI_SSE_ATTR_STATUS_PENDING_OFFSET	2
+#define SBI_SSE_ATTR_STATUS_INJECT_OFFSET	3
+
+#define SBI_SSE_ATTR_CONFIG_ONESHOT	BIT(0)
+
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPP	BIT(0)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPIE	BIT(1)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_HSTATUS_SPV	BIT(2)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_HSTATUS_SPVP	BIT(3)
+
+#define SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS	0x00000000
+#define SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP		0x00000001
+#define SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS	0x00008000
+#define SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW	0x00010000
+#define SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS	0x00100000
+#define SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS	0x00108000
+#define SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED	0xffff0000
+#define SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED	0xffff8000
+
+#define SBI_SSE_EVENT_PLATFORM		BIT(14)
+#define SBI_SSE_EVENT_GLOBAL		BIT(15)
+
 /* SBI spec version fields */
 #define SBI_SPEC_VERSION_DEFAULT	0x1
 #define SBI_SPEC_VERSION_MAJOR_SHIFT	24
Signature · 1 line
2.43.0
#2 lore raw

PATCH v7 2/5 riscv: add support for SBI Supervisor Software Events extension

In reply to Clément Léger

The SBI SSE extension allows the supervisor software to be notified by the SBI of specific events that are not maskable. The context switch is handled partially by the firmware which will save registers a6 and a7. When entering kernel we can rely on these 2 registers to setup the stack and save all the registers.

Since SSE events can be delivered at any time to the kernel (including during exception handling, we need a way to locate the current_task for context tracking. On RISC-V, it is sotred in scratch when in user space or tp when in kernel space (in which case SSCRATCH is zero). But at a at the beginning of exception handling, SSCRATCH is used to swap tp and check the origin of the exception. If interrupted at that point, then, there is no way to reliably know were is located the current task_struct. Even checking the interruption location won't work as SSE event can be nested on top of each other so the original interruption site might be lost at some point. In order to retrieve it reliably, store the current task in an additional __sse_entry_task per_cpu array. This array is then used to retrieve the current task based on the hart ID that is passed to the SSE event handler in a6.

That being said, the way the current task struct is stored should probably be reworked to find a better reliable alternative.

Since each events (and each CPU for local events) have their own context and can preempt each other, allocate a stack (and a shadow stack if needed for each of them (and for each cpu for local events).

When completing the event, if we were coming from kernel with interrupts disabled, simply return there. If coming from userspace or kernel with interrupts enabled, simulate an interrupt exception by setting IE_SIE in CSR_IP to allow delivery of signals to user task. For instance this can happen, when a RAS event has been generated by a user application and a SIGBUS has been sent to a task.

---
 arch/riscv/include/asm/asm.h         |  14 ++-
 arch/riscv/include/asm/scs.h         |   7 ++
 arch/riscv/include/asm/sse.h         |  47 +++++++
 arch/riscv/include/asm/switch_to.h   |  14 +++
 arch/riscv/include/asm/thread_info.h |   1 +
 arch/riscv/kernel/Makefile           |   1 +
 arch/riscv/kernel/asm-offsets.c      |  14 +++
 arch/riscv/kernel/sbi_sse.c          | 174 ++++++++++++++++++++++++++
 arch/riscv/kernel/sbi_sse_entry.S    | 178 +++++++++++++++++++++++++++
 9 files changed, 447 insertions(+), 3 deletions(-)
 create mode 100644 arch/riscv/include/asm/sse.h
 create mode 100644 arch/riscv/kernel/sbi_sse.c
 create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
arch/riscv/include/asm/asm.h +11 -3
index 2a16e88e13de..416dddd37d67 100644
@@ -90,16 +90,24 @@
 #define PER_CPU_OFFSET_SHIFT 3
 #endif
 
-.macro asm_per_cpu dst sym tmp
-	lw    \tmp, TASK_TI_CPU_NUM(tp)
-	slli  \tmp, \tmp, PER_CPU_OFFSET_SHIFT
+.macro asm_per_cpu_with_cpu dst sym tmp cpu
+	slli  \tmp, \cpu, PER_CPU_OFFSET_SHIFT
 	la    \dst, __per_cpu_offset
 	add   \dst, \dst, \tmp
 	REG_L \tmp, 0(\dst)
 	la    \dst, \sym
 	add   \dst, \dst, \tmp
 .endm
+
+.macro asm_per_cpu dst sym tmp
+	lw \tmp, TASK_TI_CPU_NUM(tp)
+	asm_per_cpu_with_cpu \dst \sym \tmp \tmp
+.endm
 #else /* CONFIG_SMP */
+.macro asm_per_cpu_with_cpu dst sym tmp cpu
+	la    \dst, \sym
+.endm
+
 .macro asm_per_cpu dst sym tmp
 	la    \dst, \sym
 .endm
arch/riscv/include/asm/scs.h +7 -0
index 0e45db78b24b..62344daad73d 100644
@@ -18,6 +18,11 @@
 	load_per_cpu gp, irq_shadow_call_stack_ptr, \tmp
 .endm
 
+/* Load the per-CPU IRQ shadow call stack to gp. */
+.macro scs_load_sse_stack reg_evt
+	REG_L gp, SSE_REG_EVT_SHADOW_STACK(\reg_evt)
+.endm
+
 /* Load task_scs_sp(current) to gp. */
 .macro scs_load_current
 	REG_L	gp, TASK_TI_SCS_SP(tp)
@@ -41,6 +46,8 @@
 .endm
 .macro scs_load_irq_stack tmp
 .endm
+.macro scs_load_sse_stack reg_evt
+.endm
 .macro scs_load_current
 .endm
 .macro scs_load_current_if_task_changed prev
arch/riscv/include/asm/sse.h new file +47 -0

47 lines not on this page · raw message

arch/riscv/include/asm/switch_to.h +14 -0
index 0e71eb82f920..70e68e630216 100644
@@ -88,6 +88,19 @@ static inline void __switch_to_envcfg(struct task_struct *next)
 			:: "r" (next->thread.envcfg) : "memory");
 }
 
+#ifdef CONFIG_RISCV_SBI_SSE
+DECLARE_PER_CPU(struct task_struct *, __sbi_sse_entry_task);
+
+static inline void __switch_sbi_sse_entry_task(struct task_struct *next)
+{
+	__this_cpu_write(__sbi_sse_entry_task, next);
+}
+#else
+static inline void __switch_sbi_sse_entry_task(struct task_struct *next)
+{
+}
+#endif
+
 extern struct task_struct *__switch_to(struct task_struct *,
 				       struct task_struct *);
 
@@ -122,6 +135,7 @@ do {							\
 	if (switch_to_should_flush_icache(__next))	\
 		local_flush_icache_all();		\
 	__switch_to_envcfg(__next);			\
+	__switch_sbi_sse_entry_task(__next);			\
 	((last) = __switch_to(__prev, __next));		\
 } while (0)
 
arch/riscv/include/asm/thread_info.h +1 -0
index f5916a70879a..28e9805e61fc 100644
@@ -36,6 +36,7 @@
 #define OVERFLOW_STACK_SIZE     SZ_4K
 
 #define IRQ_STACK_SIZE		THREAD_SIZE
+#define SSE_STACK_SIZE		THREAD_SIZE
 
 #ifndef __ASSEMBLY__
 
arch/riscv/kernel/Makefile +1 -0
index c7b542573407..16637e01a6b3 100644
@@ -99,6 +99,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
 obj-$(CONFIG_PERF_EVENTS)	+= perf_callchain.o
 obj-$(CONFIG_HAVE_PERF_REGS)	+= perf_regs.o
 obj-$(CONFIG_RISCV_SBI)		+= sbi.o sbi_ecall.o
+obj-$(CONFIG_RISCV_SBI_SSE)	+= sbi_sse.o sbi_sse_entry.o
 ifeq ($(CONFIG_RISCV_SBI), y)
 obj-$(CONFIG_SMP)		+= sbi-ipi.o
 obj-$(CONFIG_SMP) += cpu_ops_sbi.o
arch/riscv/kernel/asm-offsets.c +14 -0
index 6e8c0d6feae9..1b0d8624ef6e 100644
@@ -14,6 +14,8 @@
 #include <asm/ptrace.h>
 #include <asm/cpu_ops_sbi.h>
 #include <asm/stacktrace.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
 #include <asm/suspend.h>
 
 void asm_offsets(void);
@@ -528,4 +530,16 @@ void asm_offsets(void)
 	DEFINE(FREGS_A6,	    offsetof(struct __arch_ftrace_regs, a6));
 	DEFINE(FREGS_A7,	    offsetof(struct __arch_ftrace_regs, a7));
 #endif
+
+#ifdef CONFIG_RISCV_SBI_SSE
+	OFFSET(SSE_REG_EVT_STACK, sse_event_arch_data, stack);
+	OFFSET(SSE_REG_EVT_SHADOW_STACK, sse_event_arch_data, shadow_stack);
+	OFFSET(SSE_REG_EVT_TMP, sse_event_arch_data, tmp);
+	OFFSET(SSE_REG_HART_ID, sse_event_arch_data, hart_id);
+	OFFSET(SSE_REG_CPU_ID, sse_event_arch_data, cpu_id);
+
+	DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
+	DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
+	DEFINE(ASM_NR_CPUS, NR_CPUS);
+#endif
 }
arch/riscv/kernel/sbi_sse.c new file +174 -0

174 lines not on this page · raw message

arch/riscv/kernel/sbi_sse_entry.S new file +178 -0

178 lines not on this page · raw message

Signature · 1 line
2.43.0
#3 lore raw

Re: [External] [PATCH v7 2/5] riscv: add support for SBI Supervisor Software Events extension

Hi Clément,

Quoted 289 lines · Clément Léger

The SBI SSE extension allows the supervisor software to be notified by the SBI of specific events that are not maskable. The context switch is handled partially by the firmware which will save registers a6 and a7. When entering kernel we can rely on these 2 registers to setup the stack and save all the registers.

Since SSE events can be delivered at any time to the kernel (including during exception handling, we need a way to locate the current_task for context tracking. On RISC-V, it is sotred in scratch when in user space or tp when in kernel space (in which case SSCRATCH is zero). But at a at the beginning of exception handling, SSCRATCH is used to swap tp and check the origin of the exception. If interrupted at that point, then, there is no way to reliably know were is located the current task_struct. Even checking the interruption location won't work as SSE event can be nested on top of each other so the original interruption site might be lost at some point. In order to retrieve it reliably, store the current task in an additional __sse_entry_task per_cpu array. This array is then used to retrieve the current task based on the hart ID that is passed to the SSE event handler in a6.

That being said, the way the current task struct is stored should probably be reworked to find a better reliable alternative.

Since each events (and each CPU for local events) have their own context and can preempt each other, allocate a stack (and a shadow stack if needed for each of them (and for each cpu for local events).

When completing the event, if we were coming from kernel with interrupts disabled, simply return there. If coming from userspace or kernel with interrupts enabled, simulate an interrupt exception by setting IE_SIE in CSR_IP to allow delivery of signals to user task. For instance this can happen, when a RAS event has been generated by a user application and a SIGBUS has been sent to a task.

---
 arch/riscv/include/asm/asm.h         |  14 ++-
 arch/riscv/include/asm/scs.h         |   7 ++
 arch/riscv/include/asm/sse.h         |  47 +++++++
 arch/riscv/include/asm/switch_to.h   |  14 +++
 arch/riscv/include/asm/thread_info.h |   1 +
 arch/riscv/kernel/Makefile           |   1 +
 arch/riscv/kernel/asm-offsets.c      |  14 +++
 arch/riscv/kernel/sbi_sse.c          | 174 ++++++++++++++++++++++++++
 arch/riscv/kernel/sbi_sse_entry.S    | 178 +++++++++++++++++++++++++++
 9 files changed, 447 insertions(+), 3 deletions(-)
 create mode 100644 arch/riscv/include/asm/sse.h
 create mode 100644 arch/riscv/kernel/sbi_sse.c
 create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
arch/riscv/include/asm/asm.h +11 -3
index 2a16e88e13de..416dddd37d67 100644
@@ -90,16 +90,24 @@
 #define PER_CPU_OFFSET_SHIFT 3
 #endif
 
-.macro asm_per_cpu dst sym tmp
-       lw    \tmp, TASK_TI_CPU_NUM(tp)
-       slli  \tmp, \tmp, PER_CPU_OFFSET_SHIFT
+.macro asm_per_cpu_with_cpu dst sym tmp cpu
+       slli  \tmp, \cpu, PER_CPU_OFFSET_SHIFT
        la    \dst, __per_cpu_offset
        add   \dst, \dst, \tmp
        REG_L \tmp, 0(\dst)
        la    \dst, \sym
        add   \dst, \dst, \tmp
 .endm
+
+.macro asm_per_cpu dst sym tmp
+       lw \tmp, TASK_TI_CPU_NUM(tp)
+       asm_per_cpu_with_cpu \dst \sym \tmp \tmp
+.endm
 #else /* CONFIG_SMP */
+.macro asm_per_cpu_with_cpu dst sym tmp cpu
+       la    \dst, \sym
+.endm
+
 .macro asm_per_cpu dst sym tmp
        la    \dst, \sym
 .endm
arch/riscv/include/asm/scs.h +7 -0
index 0e45db78b24b..62344daad73d 100644
@@ -18,6 +18,11 @@
        load_per_cpu gp, irq_shadow_call_stack_ptr, \tmp
 .endm
 
+/* Load the per-CPU IRQ shadow call stack to gp. */
+.macro scs_load_sse_stack reg_evt
+       REG_L gp, SSE_REG_EVT_SHADOW_STACK(\reg_evt)
+.endm
+
 /* Load task_scs_sp(current) to gp. */
 .macro scs_load_current
        REG_L   gp, TASK_TI_SCS_SP(tp)
@@ -41,6 +46,8 @@
 .endm
 .macro scs_load_irq_stack tmp
 .endm
+.macro scs_load_sse_stack reg_evt
+.endm
 .macro scs_load_current
 .endm
 .macro scs_load_current_if_task_changed prev
arch/riscv/include/asm/sse.h new file +47 -0
new file mode 100644
index 000000000000..d3ce8c2b5221
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+#ifndef __ASM_SSE_H
+#define __ASM_SSE_H
+
+#include <asm/sbi.h>
+
+#ifdef CONFIG_RISCV_SBI_SSE
+
+struct sse_event_interrupted_state {
+       unsigned long a6;
+       unsigned long a7;
+};
+
+struct sse_event_arch_data {
+       void *stack;
+       void *shadow_stack;
+       unsigned long tmp;
+       struct sse_event_interrupted_state interrupted;
+       unsigned long interrupted_phys;
+       u32 evt_id;
+       unsigned int hart_id;
+       unsigned int cpu_id;
+};
+
+static inline bool sse_event_is_global(u32 evt)
+{
+       return !!(evt & SBI_SSE_EVENT_GLOBAL);
+}
+
+void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int cpu);
+int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id,
+                       int cpu);
+void arch_sse_free_event(struct sse_event_arch_data *arch_evt);
+int arch_sse_register_event(struct sse_event_arch_data *arch_evt);
+
+void sse_handle_event(struct sse_event_arch_data *arch_evt,
+                     struct pt_regs *regs);
+asmlinkage void handle_sse(void);
+asmlinkage void do_sse(struct sse_event_arch_data *arch_evt,
+                      struct pt_regs *reg);
+
+#endif
+
+#endif
arch/riscv/include/asm/switch_to.h +14 -0
index 0e71eb82f920..70e68e630216 100644
@@ -88,6 +88,19 @@ static inline void __switch_to_envcfg(struct task_struct *next)
                        :: "r" (next->thread.envcfg) : "memory");
 }
 
+#ifdef CONFIG_RISCV_SBI_SSE
+DECLARE_PER_CPU(struct task_struct *, __sbi_sse_entry_task);
+
+static inline void __switch_sbi_sse_entry_task(struct task_struct *next)
+{
+       __this_cpu_write(__sbi_sse_entry_task, next);
+}
+#else
+static inline void __switch_sbi_sse_entry_task(struct task_struct *next)
+{
+}
+#endif
+
 extern struct task_struct *__switch_to(struct task_struct *,
                                       struct task_struct *);
 
@@ -122,6 +135,7 @@ do {                                                        \
        if (switch_to_should_flush_icache(__next))      \
                local_flush_icache_all();               \
        __switch_to_envcfg(__next);                     \
+       __switch_sbi_sse_entry_task(__next);                    \
        ((last) = __switch_to(__prev, __next));         \
 } while (0)
 
arch/riscv/include/asm/thread_info.h +1 -0
index f5916a70879a..28e9805e61fc 100644
@@ -36,6 +36,7 @@
 #define OVERFLOW_STACK_SIZE     SZ_4K
 
 #define IRQ_STACK_SIZE         THREAD_SIZE
+#define SSE_STACK_SIZE         THREAD_SIZE
 
 #ifndef __ASSEMBLY__
 
arch/riscv/kernel/Makefile +1 -0
index c7b542573407..16637e01a6b3 100644
@@ -99,6 +99,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE)  += mcount-dyn.o
 obj-$(CONFIG_PERF_EVENTS)      += perf_callchain.o
 obj-$(CONFIG_HAVE_PERF_REGS)   += perf_regs.o
 obj-$(CONFIG_RISCV_SBI)                += sbi.o sbi_ecall.o
+obj-$(CONFIG_RISCV_SBI_SSE)    += sbi_sse.o sbi_sse_entry.o
 ifeq ($(CONFIG_RISCV_SBI), y)
 obj-$(CONFIG_SMP)              += sbi-ipi.o
 obj-$(CONFIG_SMP) += cpu_ops_sbi.o
arch/riscv/kernel/asm-offsets.c +14 -0
index 6e8c0d6feae9..1b0d8624ef6e 100644
@@ -14,6 +14,8 @@
 #include <asm/ptrace.h>
 #include <asm/cpu_ops_sbi.h>
 #include <asm/stacktrace.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
 #include <asm/suspend.h>
 
 void asm_offsets(void);
@@ -528,4 +530,16 @@ void asm_offsets(void)
        DEFINE(FREGS_A6,            offsetof(struct __arch_ftrace_regs, a6));
        DEFINE(FREGS_A7,            offsetof(struct __arch_ftrace_regs, a7));
 #endif
+
+#ifdef CONFIG_RISCV_SBI_SSE
+       OFFSET(SSE_REG_EVT_STACK, sse_event_arch_data, stack);
+       OFFSET(SSE_REG_EVT_SHADOW_STACK, sse_event_arch_data, shadow_stack);
+       OFFSET(SSE_REG_EVT_TMP, sse_event_arch_data, tmp);
+       OFFSET(SSE_REG_HART_ID, sse_event_arch_data, hart_id);
+       OFFSET(SSE_REG_CPU_ID, sse_event_arch_data, cpu_id);
+
+       DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
+       DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
+       DEFINE(ASM_NR_CPUS, NR_CPUS);
+#endif
 }
arch/riscv/kernel/sbi_sse.c new file +36 -0
new file mode 100644
index 000000000000..626912a0927d
@@ -0,0 +1,174 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+#include <linux/nmi.h>
+#include <linux/scs.h>
+#include <linux/bitfield.h>
+#include <linux/percpu-defs.h>
+
+#include <asm/asm-prototypes.h>
+#include <asm/switch_to.h>
+#include <asm/irq_stack.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+DEFINE_PER_CPU(struct task_struct *, __sbi_sse_entry_task);
+
+void __weak sse_handle_event(struct sse_event_arch_data *arch_evt, struct pt_regs *regs)
+{
+}
+
+void do_sse(struct sse_event_arch_data *arch_evt, struct pt_regs *regs)
+{
+       nmi_enter();
+
+       /* Retrieve missing GPRs from SBI */
+       sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, arch_evt->evt_id,
+                 SBI_SSE_ATTR_INTERRUPTED_A6,
+                 (SBI_SSE_ATTR_INTERRUPTED_A7 - SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
+                 arch_evt->interrupted_phys, 0, 0);
+
+       memcpy(&regs->a6, &arch_evt->interrupted, sizeof(arch_evt->interrupted));
+
+       sse_handle_event(arch_evt, regs);
+
+       /*

arch/riscv/kernel/sbi_sse.c line 46

Quoted 10 lines
arch/riscv/kernel/sbi_sse.c new file +10 -0
+        * The SSE delivery path does not uses the "standard" exception path
+        * (see sse_entry.S) and does not process any pending signal/softirqs
+        * due to being similar to a NMI.
+        * Some drivers (PMU, RAS) enqueue pending work that needs to be handled
+        * as soon as possible by bottom halves. For that purpose, set the SIP
+        * software interrupt pending bit which will force a software interrupt
+        * to be serviced once interrupts are reenabled in the interrupted
+        * context if they were masked or directly if unmasked.
+        */
+       csr_set(CSR_IP, IE_SIE);

IE_SIE may not always be enabled in CSR_IE(for example when we disable CONFIG_ACLINT_SSWI and use imsic for ipi). Maybe we should send ipi to the current cpu here.

Best regards,
Xu Lu

Quoted 315 lines
arch/riscv/kernel/sbi_sse.c new file +128 -0
+
+       nmi_exit();
+}
+
+static void *alloc_to_stack_pointer(void *alloc)
+{
+       return alloc ? alloc + SSE_STACK_SIZE : NULL;
+}
+
+static void *stack_pointer_to_alloc(void *stack)
+{
+       return stack - SSE_STACK_SIZE;
+}
+
+#ifdef CONFIG_VMAP_STACK
+static void *sse_stack_alloc(unsigned int cpu)
+{
+       void *stack = arch_alloc_vmap_stack(SSE_STACK_SIZE, cpu_to_node(cpu));
+
+       return alloc_to_stack_pointer(stack);
+}
+
+static void sse_stack_free(void *stack)
+{
+       vfree(stack_pointer_to_alloc(stack));
+}
+
+static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
+{
+       void *p_stack = arch_evt->stack;
+       unsigned long stack = (unsigned long) stack_pointer_to_alloc(p_stack);
+       unsigned long stack_end = stack + SSE_STACK_SIZE;
+
+       /*
+        * Flush the tlb to avoid taking any exception when accessing the
+        * vmapped stack inside the SSE handler
+        */
+       if (sse_event_is_global(arch_evt->evt_id))
+               flush_tlb_kernel_range(stack, stack_end);
+       else
+               local_flush_tlb_kernel_range(stack, (unsigned long) stack_end);
+}
+#else /* CONFIG_VMAP_STACK */
+static void *sse_stack_alloc(unsigned int cpu)
+{
+       void *stack = kmalloc(SSE_STACK_SIZE, GFP_KERNEL);
+
+       return alloc_to_stack_pointer(stack);
+}
+
+static void sse_stack_free(void *stack)
+{
+       kfree(stack_pointer_to_alloc(stack));
+}
+
+static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt) {}
+#endif /* CONFIG_VMAP_STACK */
+
+static int sse_init_scs(int cpu, struct sse_event_arch_data *arch_evt)
+{
+       void *stack;
+
+       if (!scs_is_enabled())
+               return 0;
+
+       stack = scs_alloc(cpu_to_node(cpu));
+       if (!stack)
+               return -ENOMEM;
+
+       arch_evt->shadow_stack = stack;
+
+       return 0;
+}
+
+void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int cpu)
+{
+       arch_evt->cpu_id = cpu;
+       arch_evt->hart_id = cpuid_to_hartid_map(cpu);
+}
+
+int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id,
+                       int cpu)
+{
+       void *stack;
+
+       arch_evt->evt_id = evt_id;
+       stack = sse_stack_alloc(cpu);
+       if (!stack)
+               return -ENOMEM;
+
+       arch_evt->stack = stack;
+
+       if (sse_init_scs(cpu, arch_evt)) {
+               sse_stack_free(arch_evt->stack);
+               return -ENOMEM;
+       }
+
+       if (sse_event_is_global(evt_id)) {
+               arch_evt->interrupted_phys =
+                                       virt_to_phys(&arch_evt->interrupted);
+       } else {
+               arch_evt->interrupted_phys =
+                               per_cpu_ptr_to_phys(&arch_evt->interrupted);
+       }
+
+       arch_sse_event_update_cpu(arch_evt, cpu);
+
+       return 0;
+}
+
+void arch_sse_free_event(struct sse_event_arch_data *arch_evt)
+{
+       scs_free(arch_evt->shadow_stack);
+       sse_stack_free(arch_evt->stack);
+}
+
+int arch_sse_register_event(struct sse_event_arch_data *arch_evt)
+{
+       struct sbiret sret;
+
+       arch_sse_stack_cpu_sync(arch_evt);
+
+       sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_REGISTER, arch_evt->evt_id,
+                        (unsigned long)handle_sse, (unsigned long)arch_evt, 0,
+                        0, 0);
+
+       return sbi_err_map_linux_errno(sret.error);
+}
arch/riscv/kernel/sbi_sse_entry.S new file +178 -1
new file mode 100644
index 000000000000..612510b98445
@@ -0,0 +1,178 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+
+#include <asm/asm.h>
+#include <asm/csr.h>
+#include <asm/scs.h>
+
+/* When entering handle_sse, the following registers are set:
+ * a6: contains the hartid
+ * a7: contains a sse_event_arch_data struct pointer
+ */
+SYM_CODE_START(handle_sse)
+       /* Save stack temporarily */
+       REG_S sp, SSE_REG_EVT_TMP(a7)
+       /* Set entry stack */
+       REG_L sp, SSE_REG_EVT_STACK(a7)
+
+       addi sp, sp, -(PT_SIZE_ON_STACK)
+       REG_S ra, PT_RA(sp)
+       REG_S s0, PT_S0(sp)
+       REG_S s1, PT_S1(sp)
+       REG_S s2, PT_S2(sp)
+       REG_S s3, PT_S3(sp)
+       REG_S s4, PT_S4(sp)
+       REG_S s5, PT_S5(sp)
+       REG_S s6, PT_S6(sp)
+       REG_S s7, PT_S7(sp)
+       REG_S s8, PT_S8(sp)
+       REG_S s9, PT_S9(sp)
+       REG_S s10, PT_S10(sp)
+       REG_S s11, PT_S11(sp)
+       REG_S tp, PT_TP(sp)
+       REG_S t0, PT_T0(sp)
+       REG_S t1, PT_T1(sp)
+       REG_S t2, PT_T2(sp)
+       REG_S t3, PT_T3(sp)
+       REG_S t4, PT_T4(sp)
+       REG_S t5, PT_T5(sp)
+       REG_S t6, PT_T6(sp)
+       REG_S gp, PT_GP(sp)
+       REG_S a0, PT_A0(sp)
+       REG_S a1, PT_A1(sp)
+       REG_S a2, PT_A2(sp)
+       REG_S a3, PT_A3(sp)
+       REG_S a4, PT_A4(sp)
+       REG_S a5, PT_A5(sp)
+
+       /* Retrieve entry sp */
+       REG_L a4, SSE_REG_EVT_TMP(a7)
+       /* Save CSRs */
+       csrr a0, CSR_EPC
+       csrr a1, CSR_SSTATUS
+       csrr a2, CSR_STVAL
+       csrr a3, CSR_SCAUSE
+
+       REG_S a0, PT_EPC(sp)
+       REG_S a1, PT_STATUS(sp)
+       REG_S a2, PT_BADADDR(sp)
+       REG_S a3, PT_CAUSE(sp)
+       REG_S a4, PT_SP(sp)
+
+       /* Disable user memory access and floating/vector computing */
+       li t0, SR_SUM | SR_FS_VS
+       csrc CSR_STATUS, t0
+
+       load_global_pointer
+       scs_load_sse_stack a7
+
+#ifdef CONFIG_SMP
+       lw t4, SSE_REG_HART_ID(a7)
+       lw t3, SSE_REG_CPU_ID(a7)
+
+       bne t4, a6, .Lfind_hart_id_slowpath
+
+.Lcpu_id_found:
+#else
+       mv t3, zero
+#endif
+
+       asm_per_cpu_with_cpu t2 __sbi_sse_entry_task t1 t3
+       REG_L tp, 0(t2)
+
+       mv a1, sp /* pt_regs on stack */
+
+       /*
+        * Save sscratch for restoration since we might have interrupted the
+        * kernel in early exception path and thus, we don't know the content of
+        * sscratch.
+        */
+       csrrw s4, CSR_SSCRATCH, x0
+
+       mv a0, a7
+
+       call do_sse
+
+       csrw CSR_SSCRATCH, s4
+
+       REG_L a0, PT_STATUS(sp)
+       REG_L a1, PT_EPC(sp)
+       REG_L a2, PT_BADADDR(sp)
+       REG_L a3, PT_CAUSE(sp)
+       csrw CSR_SSTATUS, a0
+       csrw CSR_EPC, a1
+       csrw CSR_STVAL, a2
+       csrw CSR_SCAUSE, a3
+
+       REG_L ra, PT_RA(sp)
+       REG_L s0, PT_S0(sp)
+       REG_L s1, PT_S1(sp)
+       REG_L s2, PT_S2(sp)
+       REG_L s3, PT_S3(sp)
+       REG_L s4, PT_S4(sp)
+       REG_L s5, PT_S5(sp)
+       REG_L s6, PT_S6(sp)
+       REG_L s7, PT_S7(sp)
+       REG_L s8, PT_S8(sp)
+       REG_L s9, PT_S9(sp)
+       REG_L s10, PT_S10(sp)
+       REG_L s11, PT_S11(sp)
+       REG_L tp, PT_TP(sp)
+       REG_L t0, PT_T0(sp)
+       REG_L t1, PT_T1(sp)
+       REG_L t2, PT_T2(sp)
+       REG_L t3, PT_T3(sp)
+       REG_L t4, PT_T4(sp)
+       REG_L t5, PT_T5(sp)
+       REG_L t6, PT_T6(sp)
+       REG_L gp, PT_GP(sp)
+       REG_L a0, PT_A0(sp)
+       REG_L a1, PT_A1(sp)
+       REG_L a2, PT_A2(sp)
+       REG_L a3, PT_A3(sp)
+       REG_L a4, PT_A4(sp)
+       REG_L a5, PT_A5(sp)
+
+       REG_L sp, PT_SP(sp)
+
+       li a7, SBI_EXT_SSE
+       li a6, SBI_SSE_EVENT_COMPLETE
+       ecall
+
+#ifdef CONFIG_SMP
+.Lfind_hart_id_slowpath:
+
+       /* Restore current task struct from __sbi_sse_entry_task */
+       li t1, ASM_NR_CPUS
+       /* Slowpath to find the CPU id associated to the hart id */
+       la t0, __cpuid_to_hartid_map
+
+.Lhart_id_loop:
+       REG_L t2, 0(t0)
+       beq t2, a6, .Lcpu_id_found
+
+       /* Increment pointer and CPU number */
+       addi t3, t3, 1
+       addi t0, t0, RISCV_SZPTR
+       bltu t3, t1, .Lhart_id_loop
+
+       /*
+        * This should never happen since we expect the hart_id to match one
+        * of our CPU, but better be safe than sorry
+        */
+       la tp, init_task
+       la a0, sse_hart_id_panic_string
+       la t0, panic
+       jalr t0
+#endif
+
+SYM_CODE_END(handle_sse)
+
+SYM_DATA_START_LOCAL(sse_hart_id_panic_string)
+    .ascii "Unable to match hart_id with cpu\0"
+SYM_DATA_END(sse_hart_id_panic_string)
--

2.43.0

#4 lore raw

Re: [External] [PATCH v7 2/5] riscv: add support for SBI Supervisor Software Events extension

Quoted 20 lines · Xu Lu
Quoted 20 lines
arch/riscv/kernel/sbi_sse.c new file +20 -0
+void do_sse(struct sse_event_arch_data *arch_evt, struct pt_regs *regs)
+{
+       nmi_enter();
+
+       /* Retrieve missing GPRs from SBI */
+       sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, arch_evt->evt_id,
+                 SBI_SSE_ATTR_INTERRUPTED_A6,
+                 (SBI_SSE_ATTR_INTERRUPTED_A7 - SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
+                 arch_evt->interrupted_phys, 0, 0);
+
+       memcpy(&regs->a6, &arch_evt->interrupted, sizeof(arch_evt->interrupted));
+
+       sse_handle_event(arch_evt, regs);
+
+       /*
+        * The SSE delivery path does not uses the "standard" exception path
+        * (see sse_entry.S) and does not process any pending signal/softirqs
+        * due to being similar to a NMI.
+        * Some drivers (PMU, RAS) enqueue pending work that needs to be handled
+        * as soon as possible by bottom halves. For that purpose, set the SIP
Quoted 8 lines

arch/riscv/kernel/sbi_sse.c line 46

Quoted 5 lines
arch/riscv/kernel/sbi_sse.c new file +5 -0
+        * software interrupt pending bit which will force a software interrupt
+        * to be serviced once interrupts are reenabled in the interrupted
+        * context if they were masked or directly if unmasked.
+        */
+       csr_set(CSR_IP, IE_SIE);

IE_SIE may not always be enabled in CSR_IE(for example when we disable CONFIG_ACLINT_SSWI and use imsic for ipi). Maybe we should send ipi to the current cpu here.

Hi Xu,

Indeed, that's a good catch. Sending an IPI will be more generic.

Thanks,

Clément

Quoted 3 lines

Best regards,
Xu Lu

#5 lore raw

PATCH v7 3/5 drivers: firmware: add riscv SSE support

In reply to Clément Léger

Add driver level interface to use RISC-V SSE arch support. This interface allows registering SSE handlers, and receive them. This will be used by PMU and GHES driver.

---
 MAINTAINERS                            |  15 +
 drivers/firmware/Kconfig               |   1 +
 drivers/firmware/Makefile              |   1 +
 drivers/firmware/riscv/Kconfig         |  15 +
 drivers/firmware/riscv/Makefile        |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c | 701 +++++++++++++++++++++++++
 include/linux/riscv_sbi_sse.h          |  57 ++
 7 files changed, 793 insertions(+)
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h
MAINTAINERS +15 -0
index fe168477caa4..684d23f852c3 100644
@@ -21648,6 +21648,13 @@ T:	git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
 F:	Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
 F:	drivers/iommu/riscv/
 
+RISC-V FIRMWARE DRIVERS
+M:	Conor Dooley <[email protected]>
+L:	[email protected]
+S:	Maintained
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git
+F:	drivers/firmware/riscv/*
+
 RISC-V MICROCHIP FPGA SUPPORT
 M:	Conor Dooley <[email protected]>
 M:	Daire McNamara <[email protected]>
@@ -21712,6 +21719,14 @@ F:	arch/riscv/boot/dts/spacemit/
 N:	spacemit
 K:	spacemit
 
+RISC-V SSE DRIVER
+M:	Clément Léger <[email protected]>
+R:	Himanshu Chauhan <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	drivers/firmware/riscv/riscv_sse.c
+F:	include/linux/riscv_sse.h
+
 RISC-V THEAD SoC SUPPORT
 M:	Drew Fustini <[email protected]>
 M:	Guo Ren <[email protected]>
drivers/firmware/Kconfig +1 -0
index bbd2155d8483..1894df87b08e 100644
@@ -294,6 +294,7 @@ source "drivers/firmware/meson/Kconfig"
 source "drivers/firmware/microchip/Kconfig"
 source "drivers/firmware/psci/Kconfig"
 source "drivers/firmware/qcom/Kconfig"
+source "drivers/firmware/riscv/Kconfig"
 source "drivers/firmware/samsung/Kconfig"
 source "drivers/firmware/smccc/Kconfig"
 source "drivers/firmware/tegra/Kconfig"
drivers/firmware/Makefile +1 -0
index 4ddec2820c96..6cdd84570ea7 100644
@@ -34,6 +34,7 @@ obj-y				+= efi/
 obj-y				+= imx/
 obj-y				+= psci/
 obj-y				+= qcom/
+obj-y				+= riscv/
 obj-y				+= samsung/
 obj-y				+= smccc/
 obj-y				+= tegra/
drivers/firmware/riscv/Kconfig new file +15 -0
new file mode 100644
index 000000000000..ed5b663ac5f9
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menu "Risc-V Specific firmware drivers"
+depends on RISCV
+
+config RISCV_SBI_SSE
+	bool "Enable SBI Supervisor Software Events support"
+	depends on RISCV_SBI
+	default y
+	help
+	  The Supervisor Software Events support allows the SBI to deliver
+	  NMI-like notifications to the supervisor mode software. When enabled,
+	  this option provides support to register callbacks on specific SSE
+	  events.
+
+endmenu
drivers/firmware/riscv/Makefile new file +3 -0
new file mode 100644
index 000000000000..c8795d4bbb2e
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RISCV_SBI_SSE)		+= riscv_sbi_sse.o
drivers/firmware/riscv/riscv_sbi_sse.c new file +701 -0

701 lines not on this page · raw message

include/linux/riscv_sbi_sse.h new file +57 -0

57 lines not on this page · raw message

Signature · 1 line
2.43.0
reviewer
#6 lore raw

Re: [PATCH v7 3/5] drivers: firmware: add riscv SSE support

Quoted 164 lines · Clément Léger

Add driver level interface to use RISC-V SSE arch support. This interface allows registering SSE handlers, and receive them. This will be used by PMU and GHES driver.

---
 MAINTAINERS                            |  15 +
 drivers/firmware/Kconfig               |   1 +
 drivers/firmware/Makefile              |   1 +
 drivers/firmware/riscv/Kconfig         |  15 +
 drivers/firmware/riscv/Makefile        |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c | 701 +++++++++++++++++++++++++
 include/linux/riscv_sbi_sse.h          |  57 ++
 7 files changed, 793 insertions(+)
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h
MAINTAINERS +15 -0
index fe168477caa4..684d23f852c3 100644
@@ -21648,6 +21648,13 @@ T:     git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
 F:     Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
 F:     drivers/iommu/riscv/
 
+RISC-V FIRMWARE DRIVERS
+M:     Conor Dooley <[email protected]>
+L:     [email protected]
+S:     Maintained
+T:     git git://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git
+F:     drivers/firmware/riscv/*
+
 RISC-V MICROCHIP FPGA SUPPORT
 M:     Conor Dooley <[email protected]>
 M:     Daire McNamara <[email protected]>
@@ -21712,6 +21719,14 @@ F:     arch/riscv/boot/dts/spacemit/
 N:     spacemit
 K:     spacemit
 
+RISC-V SSE DRIVER
+M:     Clément Léger <[email protected]>
+R:     Himanshu Chauhan <[email protected]>
+L:     [email protected]
+S:     Maintained
+F:     drivers/firmware/riscv/riscv_sse.c
+F:     include/linux/riscv_sse.h
+
 RISC-V THEAD SoC SUPPORT
 M:     Drew Fustini <[email protected]>
 M:     Guo Ren <[email protected]>
drivers/firmware/Kconfig +1 -0
index bbd2155d8483..1894df87b08e 100644
@@ -294,6 +294,7 @@ source "drivers/firmware/meson/Kconfig"
 source "drivers/firmware/microchip/Kconfig"
 source "drivers/firmware/psci/Kconfig"
 source "drivers/firmware/qcom/Kconfig"
+source "drivers/firmware/riscv/Kconfig"
 source "drivers/firmware/samsung/Kconfig"
 source "drivers/firmware/smccc/Kconfig"
 source "drivers/firmware/tegra/Kconfig"
drivers/firmware/Makefile +1 -0
index 4ddec2820c96..6cdd84570ea7 100644
@@ -34,6 +34,7 @@ obj-y                         += efi/
 obj-y                          += imx/
 obj-y                          += psci/
 obj-y                          += qcom/
+obj-y                          += riscv/
 obj-y                          += samsung/
 obj-y                          += smccc/
 obj-y                          += tegra/
drivers/firmware/riscv/Kconfig new file +15 -0
new file mode 100644
index 000000000000..ed5b663ac5f9
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menu "Risc-V Specific firmware drivers"
+depends on RISCV
+
+config RISCV_SBI_SSE
+       bool "Enable SBI Supervisor Software Events support"
+       depends on RISCV_SBI
+       default y
+       help
+         The Supervisor Software Events support allows the SBI to deliver
+         NMI-like notifications to the supervisor mode software. When enabled,
+         this option provides support to register callbacks on specific SSE
+         events.
+
+endmenu
drivers/firmware/riscv/Makefile new file +3 -0
new file mode 100644
index 000000000000..c8795d4bbb2e
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RISCV_SBI_SSE)            += riscv_sbi_sse.o
drivers/firmware/riscv/riscv_sbi_sse.c new file +48 -0
new file mode 100644
index 000000000000..57b6dad92482
@@ -0,0 +1,701 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/cpu_pm.h>
+#include <linux/hardirq.h>
+#include <linux/list.h>
+#include <linux/percpu-defs.h>
+#include <linux/reboot.h>
+#include <linux/riscv_sbi_sse.h>
+#include <linux/slab.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_event {
+       struct list_head list;
+       u32 evt_id;
+       u32 priority;
+       sse_event_handler_fn *handler;
+       void *handler_arg;
+       /* Only valid for global events */
+       unsigned int cpu;
+
+       union {
+               struct sse_registered_event *global;
+               struct sse_registered_event __percpu *local;
+       };
+};
+
+static int sse_hp_state;
+static bool sse_available __ro_after_init;
+static DEFINE_SPINLOCK(events_list_lock);
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+struct sse_registered_event {
+       struct sse_event_arch_data arch;
+       struct sse_event *event;
+       unsigned long attr;
+       bool is_enabled;
+};
+

<snip>

drivers/firmware/riscv/riscv_sbi_sse.c line 666

Quoted 7 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +7 -0
+
+static int __init sse_init(void)
+{
+       int ret;
+
+       if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+               pr_err("Missing SBI SSE extension\n");

We might have older firmware which implements an older SBI spec version so we should not throw errors over here. Please change this to pr_info().

Regards,
Anup

#7 lore raw

Re: [PATCH v7 3/5] drivers: firmware: add riscv SSE support

Quoted 168 lines · Anup Patel
Quoted 164 lines · Clément Léger

Add driver level interface to use RISC-V SSE arch support. This interface allows registering SSE handlers, and receive them. This will be used by PMU and GHES driver.

---
 MAINTAINERS                            |  15 +
 drivers/firmware/Kconfig               |   1 +
 drivers/firmware/Makefile              |   1 +
 drivers/firmware/riscv/Kconfig         |  15 +
 drivers/firmware/riscv/Makefile        |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c | 701 +++++++++++++++++++++++++
 include/linux/riscv_sbi_sse.h          |  57 ++
 7 files changed, 793 insertions(+)
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h
MAINTAINERS +15 -0
index fe168477caa4..684d23f852c3 100644
@@ -21648,6 +21648,13 @@ T:     git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
 F:     Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
 F:     drivers/iommu/riscv/
 
+RISC-V FIRMWARE DRIVERS
+M:     Conor Dooley <[email protected]>
+L:     [email protected]
+S:     Maintained
+T:     git git://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git
+F:     drivers/firmware/riscv/*
+
 RISC-V MICROCHIP FPGA SUPPORT
 M:     Conor Dooley <[email protected]>
 M:     Daire McNamara <[email protected]>
@@ -21712,6 +21719,14 @@ F:     arch/riscv/boot/dts/spacemit/
 N:     spacemit
 K:     spacemit
 
+RISC-V SSE DRIVER
+M:     Clément Léger <[email protected]>
+R:     Himanshu Chauhan <[email protected]>
+L:     [email protected]
+S:     Maintained
+F:     drivers/firmware/riscv/riscv_sse.c
+F:     include/linux/riscv_sse.h
+
 RISC-V THEAD SoC SUPPORT
 M:     Drew Fustini <[email protected]>
 M:     Guo Ren <[email protected]>
drivers/firmware/Kconfig +1 -0
index bbd2155d8483..1894df87b08e 100644
@@ -294,6 +294,7 @@ source "drivers/firmware/meson/Kconfig"
 source "drivers/firmware/microchip/Kconfig"
 source "drivers/firmware/psci/Kconfig"
 source "drivers/firmware/qcom/Kconfig"
+source "drivers/firmware/riscv/Kconfig"
 source "drivers/firmware/samsung/Kconfig"
 source "drivers/firmware/smccc/Kconfig"
 source "drivers/firmware/tegra/Kconfig"
drivers/firmware/Makefile +1 -0
index 4ddec2820c96..6cdd84570ea7 100644
@@ -34,6 +34,7 @@ obj-y                         += efi/
 obj-y                          += imx/
 obj-y                          += psci/
 obj-y                          += qcom/
+obj-y                          += riscv/
 obj-y                          += samsung/
 obj-y                          += smccc/
 obj-y                          += tegra/
drivers/firmware/riscv/Kconfig new file +15 -0
new file mode 100644
index 000000000000..ed5b663ac5f9
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menu "Risc-V Specific firmware drivers"
+depends on RISCV
+
+config RISCV_SBI_SSE
+       bool "Enable SBI Supervisor Software Events support"
+       depends on RISCV_SBI
+       default y
+       help
+         The Supervisor Software Events support allows the SBI to deliver
+         NMI-like notifications to the supervisor mode software. When enabled,
+         this option provides support to register callbacks on specific SSE
+         events.
+
+endmenu
drivers/firmware/riscv/Makefile new file +3 -0
new file mode 100644
index 000000000000..c8795d4bbb2e
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RISCV_SBI_SSE)            += riscv_sbi_sse.o
drivers/firmware/riscv/riscv_sbi_sse.c new file +48 -0
new file mode 100644
index 000000000000..57b6dad92482
@@ -0,0 +1,701 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/cpu_pm.h>
+#include <linux/hardirq.h>
+#include <linux/list.h>
+#include <linux/percpu-defs.h>
+#include <linux/reboot.h>
+#include <linux/riscv_sbi_sse.h>
+#include <linux/slab.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_event {
+       struct list_head list;
+       u32 evt_id;
+       u32 priority;
+       sse_event_handler_fn *handler;
+       void *handler_arg;
+       /* Only valid for global events */
+       unsigned int cpu;
+
+       union {
+               struct sse_registered_event *global;
+               struct sse_registered_event __percpu *local;
+       };
+};
+
+static int sse_hp_state;
+static bool sse_available __ro_after_init;
+static DEFINE_SPINLOCK(events_list_lock);
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+struct sse_registered_event {
+       struct sse_event_arch_data arch;
+       struct sse_event *event;
+       unsigned long attr;
+       bool is_enabled;
+};
+

<snip>

Quoted 11 lines

drivers/firmware/riscv/riscv_sbi_sse.c line 666

Quoted 7 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +7 -0
+
+static int __init sse_init(void)
+{
+       int ret;
+
+       if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+               pr_err("Missing SBI SSE extension\n");

We might have older firmware which implements an older SBI spec version so we should not throw errors over here. Please change this to pr_info().

Yeah indeed, that might confuse the end-user.

Thanks,

Clément

Quoted 3 lines

Regards,
Anup

#8 lore raw

Re: [PATCH v7 3/5] drivers: firmware: add riscv SSE support

In reply to Clément Léger

Hi Clement,

Quoted 758 lines · Clément Léger

Add driver level interface to use RISC-V SSE arch support. This interface allows registering SSE handlers, and receive them. This will be used by PMU and GHES driver.

---
 MAINTAINERS                            |  15 +
 drivers/firmware/Kconfig               |   1 +
 drivers/firmware/Makefile              |   1 +
 drivers/firmware/riscv/Kconfig         |  15 +
 drivers/firmware/riscv/Makefile        |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c | 701 +++++++++++++++++++++++++
 include/linux/riscv_sbi_sse.h          |  57 ++
 7 files changed, 793 insertions(+)
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h
MAINTAINERS +15 -0
index fe168477caa4..684d23f852c3 100644
@@ -21648,6 +21648,13 @@ T:     git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
 F:     Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
 F:     drivers/iommu/riscv/
 
+RISC-V FIRMWARE DRIVERS
+M:     Conor Dooley <[email protected]>
+L:     [email protected]
+S:     Maintained
+T:     git git://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git
+F:     drivers/firmware/riscv/*
+
 RISC-V MICROCHIP FPGA SUPPORT
 M:     Conor Dooley <[email protected]>
 M:     Daire McNamara <[email protected]>
@@ -21712,6 +21719,14 @@ F:     arch/riscv/boot/dts/spacemit/
 N:     spacemit
 K:     spacemit
 
+RISC-V SSE DRIVER
+M:     Clément Léger <[email protected]>
+R:     Himanshu Chauhan <[email protected]>
+L:     [email protected]
+S:     Maintained
+F:     drivers/firmware/riscv/riscv_sse.c
+F:     include/linux/riscv_sse.h
+
 RISC-V THEAD SoC SUPPORT
 M:     Drew Fustini <[email protected]>
 M:     Guo Ren <[email protected]>
drivers/firmware/Kconfig +1 -0
index bbd2155d8483..1894df87b08e 100644
@@ -294,6 +294,7 @@ source "drivers/firmware/meson/Kconfig"
 source "drivers/firmware/microchip/Kconfig"
 source "drivers/firmware/psci/Kconfig"
 source "drivers/firmware/qcom/Kconfig"
+source "drivers/firmware/riscv/Kconfig"
 source "drivers/firmware/samsung/Kconfig"
 source "drivers/firmware/smccc/Kconfig"
 source "drivers/firmware/tegra/Kconfig"
drivers/firmware/Makefile +1 -0
index 4ddec2820c96..6cdd84570ea7 100644
@@ -34,6 +34,7 @@ obj-y                         += efi/
 obj-y                          += imx/
 obj-y                          += psci/
 obj-y                          += qcom/
+obj-y                          += riscv/
 obj-y                          += samsung/
 obj-y                          += smccc/
 obj-y                          += tegra/
drivers/firmware/riscv/Kconfig new file +15 -0
new file mode 100644
index 000000000000..ed5b663ac5f9
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menu "Risc-V Specific firmware drivers"
+depends on RISCV
+
+config RISCV_SBI_SSE
+       bool "Enable SBI Supervisor Software Events support"
+       depends on RISCV_SBI
+       default y
+       help
+         The Supervisor Software Events support allows the SBI to deliver
+         NMI-like notifications to the supervisor mode software. When enabled,
+         this option provides support to register callbacks on specific SSE
+         events.
+
+endmenu
drivers/firmware/riscv/Makefile new file +3 -0
new file mode 100644
index 000000000000..c8795d4bbb2e
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RISCV_SBI_SSE)            += riscv_sbi_sse.o
drivers/firmware/riscv/riscv_sbi_sse.c new file +642 -0
new file mode 100644
index 000000000000..57b6dad92482
@@ -0,0 +1,701 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/cpu_pm.h>
+#include <linux/hardirq.h>
+#include <linux/list.h>
+#include <linux/percpu-defs.h>
+#include <linux/reboot.h>
+#include <linux/riscv_sbi_sse.h>
+#include <linux/slab.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_event {
+       struct list_head list;
+       u32 evt_id;
+       u32 priority;
+       sse_event_handler_fn *handler;
+       void *handler_arg;
+       /* Only valid for global events */
+       unsigned int cpu;
+
+       union {
+               struct sse_registered_event *global;
+               struct sse_registered_event __percpu *local;
+       };
+};
+
+static int sse_hp_state;
+static bool sse_available __ro_after_init;
+static DEFINE_SPINLOCK(events_list_lock);
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+struct sse_registered_event {
+       struct sse_event_arch_data arch;
+       struct sse_event *event;
+       unsigned long attr;
+       bool is_enabled;
+};
+
+void sse_handle_event(struct sse_event_arch_data *arch_event,
+                     struct pt_regs *regs)
+{
+       int ret;
+       struct sse_registered_event *reg_evt =
+               container_of(arch_event, struct sse_registered_event, arch);
+       struct sse_event *evt = reg_evt->event;
+
+       ret = evt->handler(evt->evt_id, evt->handler_arg, regs);
+       if (ret)
+               pr_warn("event %x handler failed with error %d\n", evt->evt_id, ret);
+}
+
+static struct sse_event *sse_event_get(u32 evt)
+{
+       struct sse_event *event = NULL;
+
+       scoped_guard(spinlock, &events_list_lock) {
+               list_for_each_entry(event, &events, list) {
+                       if (event->evt_id == evt)
+                               return event;
+               }
+       }
+
+       return NULL;
+}
+
+static phys_addr_t sse_event_get_attr_phys(struct sse_registered_event *reg_evt)
+{
+       phys_addr_t phys;
+       void *addr = &reg_evt->attr;
+
+       if (sse_event_is_global(reg_evt->event->evt_id))
+               phys = virt_to_phys(addr);
+       else
+               phys = per_cpu_ptr_to_phys(addr);
+
+       return phys;
+}
+
+static struct sse_registered_event *sse_get_reg_evt(struct sse_event *event)
+{
+       if (sse_event_is_global(event->evt_id))
+               return event->global;
+       else
+               return per_cpu_ptr(event->local, smp_processor_id());
+}
+
+static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
+{
+       struct sbiret ret;
+       u32 evt = event->evt_id;
+       struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+
+       ret = sbi_ecall(SBI_EXT_SSE, func, evt, 0, 0, 0, 0, 0);
+       if (ret.error) {
+               pr_warn("Failed to execute func %lx, event %x, error %ld\n",
+                       func, evt, ret.error);
+               return sbi_err_map_linux_errno(ret.error);
+       }
+
+       if (func == SBI_SSE_EVENT_DISABLE)
+               reg_evt->is_enabled = false;
+       else if (func == SBI_SSE_EVENT_ENABLE)
+               reg_evt->is_enabled = true;
+
+       return 0;
+}
+
+int sse_event_disable_local(struct sse_event *event)
+{
+       return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
+}
+EXPORT_SYMBOL_GPL(sse_event_disable_local);
+
+int sse_event_enable_local(struct sse_event *event)
+{
+       return sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
+}
+EXPORT_SYMBOL_GPL(sse_event_enable_local);
+
+static int sse_event_attr_get_no_lock(struct sse_registered_event *reg_evt,
+                                     unsigned long attr_id, unsigned long *val)
+{
+       struct sbiret sret;
+       u32 evt = reg_evt->event->evt_id;
+       unsigned long phys;
+
+       phys = sse_event_get_attr_phys(reg_evt);
+
+       sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, evt, attr_id, 1,
+                        phys, 0, 0);
+       if (sret.error) {
+               pr_debug("Failed to get event %x attr %lx, error %ld\n", evt,
+                        attr_id, sret.error);
+               return sbi_err_map_linux_errno(sret.error);
+       }
+
+       *val = reg_evt->attr;
+
+       return 0;
+}
+
+static int sse_event_attr_set_nolock(struct sse_registered_event *reg_evt,
+                                    unsigned long attr_id, unsigned long val)
+{
+       struct sbiret sret;
+       u32 evt = reg_evt->event->evt_id;
+       unsigned long phys;
+
+       reg_evt->attr = val;
+       phys = sse_event_get_attr_phys(reg_evt);
+
+       sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_WRITE, evt, attr_id, 1,
+                        phys, 0, 0);
+       if (sret.error)
+               pr_debug("Failed to set event %x attr %lx, error %ld\n", evt,
+                        attr_id, sret.error);
+
+       return sbi_err_map_linux_errno(sret.error);
+}
+
+static void sse_global_event_update_cpu(struct sse_event *event,
+                                       unsigned int cpu)
+{
+       struct sse_registered_event *reg_evt = event->global;
+
+       event->cpu = cpu;
+       arch_sse_event_update_cpu(&reg_evt->arch, cpu);
+}
+
+static int sse_event_set_target_cpu_nolock(struct sse_event *event,
+                                          unsigned int cpu)
+{
+       unsigned long hart_id = cpuid_to_hartid_map(cpu);
+       struct sse_registered_event *reg_evt = event->global;
+       u32 evt = event->evt_id;
+       bool was_enabled;
+       int ret;
+
+       if (!sse_event_is_global(evt))
+               return -EINVAL;
+
+       was_enabled = reg_evt->is_enabled;
+       if (was_enabled)
+               sse_event_disable_local(event);
+
+       ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PREFERRED_HART,
+                                       hart_id);
+       if (ret == 0)
+               sse_global_event_update_cpu(event, cpu);
+
+       if (was_enabled)
+               sse_event_enable_local(event);
+
+       return 0;
+}
+
+int sse_event_set_target_cpu(struct sse_event *event, unsigned int cpu)
+{
+       int ret;
+
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock) {
+                       if (!cpu_online(cpu))
+                               return -EINVAL;
+
+                       ret = sse_event_set_target_cpu_nolock(event, cpu);
+               }
+       }
+
+       return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_set_target_cpu);
+
+static int sse_event_init_registered(unsigned int cpu,
+                                    struct sse_registered_event *reg_evt,
+                                    struct sse_event *event)
+{
+       reg_evt->event = event;
+
+       return arch_sse_init_event(&reg_evt->arch, event->evt_id, cpu);
+}
+
+static void sse_event_free_registered(struct sse_registered_event *reg_evt)
+{
+       arch_sse_free_event(&reg_evt->arch);
+}
+
+static int sse_event_alloc_global(struct sse_event *event)
+{
+       int err;
+       struct sse_registered_event *reg_evt;
+
+       reg_evt = kzalloc(sizeof(*reg_evt), GFP_KERNEL);
+       if (!reg_evt)
+               return -ENOMEM;
+
+       event->global = reg_evt;
+       err = sse_event_init_registered(smp_processor_id(), reg_evt, event);
+       if (err)
+               kfree(reg_evt);
+
+       return err;
+}
+
+static int sse_event_alloc_local(struct sse_event *event)
+{
+       int err;
+       unsigned int cpu, err_cpu;
+       struct sse_registered_event *reg_evt;
+       struct sse_registered_event __percpu *reg_evts;
+
+       reg_evts = alloc_percpu(struct sse_registered_event);
+       if (!reg_evts)
+               return -ENOMEM;
+
+       event->local = reg_evts;
+
+       for_each_possible_cpu(cpu) {
+               reg_evt = per_cpu_ptr(reg_evts, cpu);
+               err = sse_event_init_registered(cpu, reg_evt, event);
+               if (err) {
+                       err_cpu = cpu;
+                       goto err_free_per_cpu;
+               }
+       }
+
+       return 0;
+
+err_free_per_cpu:
+       for_each_possible_cpu(cpu) {
+               if (cpu == err_cpu)
+                       break;
+               reg_evt = per_cpu_ptr(reg_evts, cpu);
+               sse_event_free_registered(reg_evt);
+       }
+
+       free_percpu(reg_evts);
+
+       return err;
+}
+
+static struct sse_event *sse_event_alloc(u32 evt, u32 priority,
+                                        sse_event_handler_fn *handler, void *arg)
+{
+       int err;
+       struct sse_event *event;
+
+       event = kzalloc(sizeof(*event), GFP_KERNEL);
+       if (!event)
+               return ERR_PTR(-ENOMEM);
+
+       event->evt_id = evt;
+       event->priority = priority;
+       event->handler_arg = arg;
+       event->handler = handler;
+
+       if (sse_event_is_global(evt))
+               err = sse_event_alloc_global(event);
+       else
+               err = sse_event_alloc_local(event);
+
+       if (err) {
+               kfree(event);
+               return ERR_PTR(err);
+       }
+
+       return event;
+}
+
+static int sse_sbi_register_event(struct sse_event *event,
+                                 struct sse_registered_event *reg_evt)
+{
+       int ret;
+
+       ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PRIO,
+                                       event->priority);
+       if (ret)
+               return ret;
+
+       return arch_sse_register_event(&reg_evt->arch);
+}
+
+static int sse_event_register_local(struct sse_event *event)
+{
+       int ret;
+       struct sse_registered_event *reg_evt;
+
+       reg_evt = per_cpu_ptr(event->local, smp_processor_id());
+       ret = sse_sbi_register_event(event, reg_evt);
+       if (ret)
+               pr_debug("Failed to register event %x: err %d\n", event->evt_id,
+                        ret);
+
+       return ret;
+}
+
+static int sse_sbi_unregister_event(struct sse_event *event)
+{
+       return sse_sbi_event_func(event, SBI_SSE_EVENT_UNREGISTER);
+}
+
+struct sse_per_cpu_evt {
+       struct sse_event *event;
+       unsigned long func;
+       cpumask_t error;
+};
+
+static void sse_event_per_cpu_func(void *info)
+{
+       int ret;
+       struct sse_per_cpu_evt *cpu_evt = info;
+
+       if (cpu_evt->func == SBI_SSE_EVENT_REGISTER)
+               ret = sse_event_register_local(cpu_evt->event);
+       else
+               ret = sse_sbi_event_func(cpu_evt->event, cpu_evt->func);
+
+       if (ret)
+               cpumask_set_cpu(smp_processor_id(), &cpu_evt->error);
+}
+
+static void sse_event_free(struct sse_event *event)
+{
+       unsigned int cpu;
+       struct sse_registered_event *reg_evt;
+
+       if (sse_event_is_global(event->evt_id)) {
+               sse_event_free_registered(event->global);
+               kfree(event->global);
+       } else {
+               for_each_possible_cpu(cpu) {
+                       reg_evt = per_cpu_ptr(event->local, cpu);
+                       sse_event_free_registered(reg_evt);
+               }
+               free_percpu(event->local);
+       }
+
+       kfree(event);
+}
+
+static int sse_on_each_cpu(struct sse_event *event, unsigned long func,
+                          unsigned long revert_func)
+{
+       struct sse_per_cpu_evt cpu_evt;
+
+       cpu_evt.event = event;
+       cpumask_clear(&cpu_evt.error);
+       cpu_evt.func = func;
+       on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+       /*
+        * If there are some error reported by CPUs, revert event state on the
+        * other ones
+        */
+       if (!cpumask_empty(&cpu_evt.error)) {
+               cpumask_t revert;
+
+               cpumask_andnot(&revert, cpu_online_mask, &cpu_evt.error);
+               cpu_evt.func = revert_func;
+               on_each_cpu_mask(&revert, sse_event_per_cpu_func, &cpu_evt, 1);
+
+               return -EIO;
+       }
+
+       return 0;
+}
+
+int sse_event_enable(struct sse_event *event)
+{
+       int ret = 0;
+
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock) {
+                       if (sse_event_is_global(event->evt_id)) {
+                               ret = sse_event_enable_local(event);
+                       } else {
+                               ret = sse_on_each_cpu(event,
+                                                     SBI_SSE_EVENT_ENABLE,
+                                                     SBI_SSE_EVENT_DISABLE);
+                       }
+               }
+       }
+       return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_enable);
+
+static int sse_events_mask(void)
+{
+       struct sbiret ret;
+
+       ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_MASK, 0, 0, 0, 0, 0, 0);
+
+       return sbi_err_map_linux_errno(ret.error);
+}
+
+static int sse_events_unmask(void)
+{
+       struct sbiret ret;
+
+       ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_UNMASK, 0, 0, 0, 0, 0, 0);
+
+       return sbi_err_map_linux_errno(ret.error);
+}
+
+static void sse_event_disable_nolock(struct sse_event *event)
+{
+       struct sse_per_cpu_evt cpu_evt;
+
+       if (sse_event_is_global(event->evt_id)) {
+               sse_event_disable_local(event);
+       } else {
+               cpu_evt.event = event;
+               cpu_evt.func = SBI_SSE_EVENT_DISABLE;
+               on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+       }
+}
+
+void sse_event_disable(struct sse_event *event)
+{
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock) {
+                       sse_event_disable_nolock(event);
+               }
+       }
+}
+EXPORT_SYMBOL_GPL(sse_event_disable);
+
+struct sse_event *sse_event_register(u32 evt, u32 priority,
+                                    sse_event_handler_fn *handler, void *arg)
+{
+       struct sse_event *event;
+       int cpu;
+       int ret = 0;
+
+       if (!sse_available)
+               return ERR_PTR(-EOPNOTSUPP);
+
+       guard(mutex)(&sse_mutex);
+       if (sse_event_get(evt))
+               return ERR_PTR(-EEXIST);
+
+       event = sse_event_alloc(evt, priority, handler, arg);
+       if (IS_ERR(event))
+               return event;
+
+       scoped_guard(cpus_read_lock) {
+               if (sse_event_is_global(evt)) {
+                       unsigned long preferred_hart;
+
+                       ret = sse_event_attr_get_no_lock(event->global,
+                                                       SBI_SSE_ATTR_PREFERRED_HART,
+                                                       &preferred_hart);
+                       if (ret)
+                               goto err_event_free;
+
+                       cpu = riscv_hartid_to_cpuid(preferred_hart);
+                       sse_global_event_update_cpu(event, cpu);
+
+                       ret = sse_sbi_register_event(event, event->global);
+                       if (ret)
+                               goto err_event_free;
+
+               } else {
+                       ret = sse_on_each_cpu(event, SBI_SSE_EVENT_REGISTER,
+                                             SBI_SSE_EVENT_DISABLE);
+                       if (ret)
+                               goto err_event_free;
+               }
+       }
+
+       scoped_guard(spinlock, &events_list_lock)
+               list_add(&event->list, &events);
+
+       return event;
+
+err_event_free:
+       sse_event_free(event);
+
+       return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(sse_event_register);
+
+static void sse_event_unregister_nolock(struct sse_event *event)
+{
+       struct sse_per_cpu_evt cpu_evt;
+
+       if (sse_event_is_global(event->evt_id)) {
+               sse_sbi_unregister_event(event);
+       } else {
+               cpu_evt.event = event;
+               cpu_evt.func = SBI_SSE_EVENT_UNREGISTER;
+               on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+       }
+}
+
+void sse_event_unregister(struct sse_event *event)
+{
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock)
+                       sse_event_unregister_nolock(event);
+
+               scoped_guard(spinlock, &events_list_lock)
+                       list_del(&event->list);
+
+               sse_event_free(event);
+       }
+}
+EXPORT_SYMBOL_GPL(sse_event_unregister);
+
+static int sse_cpu_online(unsigned int cpu)
+{
+       struct sse_event *event;
+
+       scoped_guard(spinlock, &events_list_lock) {
+               list_for_each_entry(event, &events, list) {
+                       if (sse_event_is_global(event->evt_id))
+                               continue;
+
+                       sse_event_register_local(event);
+                       if (sse_get_reg_evt(event))
+                               sse_event_enable_local(event);
+               }
+       }
+
+       /* Ready to handle events. Unmask SSE. */
+       return sse_events_unmask();
+}
+
+static int sse_cpu_teardown(unsigned int cpu)
+{
+       int ret = 0;
+       unsigned int next_cpu;
+       struct sse_event *event;
+
+       /* Mask the sse events */
+       ret = sse_events_mask();
+       if (ret)
+               return ret;
+
+       scoped_guard(spinlock, &events_list_lock) {
+               list_for_each_entry(event, &events, list) {
+                       if (!sse_event_is_global(event->evt_id)) {
+                               if (event->global->is_enabled)
+                                       sse_event_disable_local(event);
+
+                               sse_sbi_unregister_event(event);
+                               continue;
+                       }
+
+                       if (event->cpu != smp_processor_id())
+                               continue;
+
+                       /* Update destination hart for global event */
+                       next_cpu = cpumask_any_but(cpu_online_mask, cpu);
+                       ret = sse_event_set_target_cpu_nolock(event, next_cpu);
+               }
+       }
+
+       return ret;
+}
+
+static void sse_reset(void)
+{
+       struct sse_event *event;
+
+       list_for_each_entry(event, &events, list) {
+               sse_event_disable_nolock(event);
+               sse_event_unregister_nolock(event);
+       }
+}
+
+static int sse_pm_notifier(struct notifier_block *nb, unsigned long action,
+                          void *data)
+{
+       WARN_ON_ONCE(preemptible());
+
+       switch (action) {
+       case CPU_PM_ENTER:
+               sse_events_mask();
+               break;
+       case CPU_PM_EXIT:
+       case CPU_PM_ENTER_FAILED:
+               sse_events_unmask();
+               break;
+       default:
+               return NOTIFY_DONE;
+       }
+
+       return NOTIFY_OK;
+}
+
+static struct notifier_block sse_pm_nb = {
+       .notifier_call = sse_pm_notifier,

drivers/firmware/riscv/riscv_sbi_sse.c line 652

Quoted 10 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +10 -0
+};
+
+/*
+ * Mask all CPUs and unregister all events on panic, reboot or kexec.
+ */
+static int sse_reboot_notifier(struct notifier_block *nb, unsigned long action,
+                              void *data)
+{
+       cpuhp_remove_state(sse_hp_state);
+       sse_reset();

Is this sse_reset required?
I was running the poweroff command. The CPU teardown function is called and events are disabled & unregistered as part of it. when sse_reset is called, it again tries to call disable & unregister events for all cpus (nolock version). So I see errors from SBI that event is in invalid state. Commenting out sse_reset() works and shutdown happens without any issue.

Quoted 115 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +49 -0
+
+       return NOTIFY_OK;
+}
+
+static struct notifier_block sse_reboot_nb = {
+       .notifier_call = sse_reboot_notifier,
+};
+
+static int __init sse_init(void)
+{
+       int ret;
+
+       if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+               pr_err("Missing SBI SSE extension\n");
+               return -EOPNOTSUPP;
+       }
+       pr_info("SBI SSE extension detected\n");
+
+       ret = cpu_pm_register_notifier(&sse_pm_nb);
+       if (ret) {
+               pr_warn("Failed to register CPU PM notifier...\n");
+               return ret;
+       }
+
+       ret = register_reboot_notifier(&sse_reboot_nb);
+       if (ret) {
+               pr_warn("Failed to register reboot notifier...\n");
+               goto remove_cpupm;
+       }
+
+       ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "riscv/sse:online",
+                               sse_cpu_online, sse_cpu_teardown);
+       if (ret < 0)
+               goto remove_reboot;
+
+       sse_hp_state = ret;
+       sse_available = true;
+
+       return 0;
+
+remove_reboot:
+       unregister_reboot_notifier(&sse_reboot_nb);
+
+remove_cpupm:
+       cpu_pm_unregister_notifier(&sse_pm_nb);
+
+       return ret;
+}
+arch_initcall(sse_init);
include/linux/riscv_sbi_sse.h new file +57 -1
new file mode 100644
index 000000000000..a1b58e89dd19
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+
+#ifndef __LINUX_RISCV_SBI_SSE_H
+#define __LINUX_RISCV_SBI_SSE_H
+
+#include <linux/types.h>
+#include <linux/linkage.h>
+
+struct sse_event;
+struct pt_regs;
+
+typedef int (sse_event_handler_fn)(u32 event_num, void *arg,
+                                  struct pt_regs *regs);
+
+#ifdef CONFIG_RISCV_SBI_SSE
+
+struct sse_event *sse_event_register(u32 event_num, u32 priority,
+                                    sse_event_handler_fn *handler, void *arg);
+
+void sse_event_unregister(struct sse_event *evt);
+
+int sse_event_set_target_cpu(struct sse_event *sse_evt, unsigned int cpu);
+
+int sse_event_enable(struct sse_event *sse_evt);
+
+void sse_event_disable(struct sse_event *sse_evt);
+
+int sse_event_enable_local(struct sse_event *sse_evt);
+int sse_event_disable_local(struct sse_event *sse_evt);
+
+#else
+static inline struct sse_event *sse_event_register(u32 event_num, u32 priority,
+                                                  sse_event_handler_fn *handler,
+                                                  void *arg)
+{
+       return ERR_PTR(-EOPNOTSUPP);
+}
+
+static inline void sse_event_unregister(struct sse_event *evt) {}
+
+static inline int sse_event_set_target_cpu(struct sse_event *sse_evt,
+                                          unsigned int cpu)
+{
+       return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable(struct sse_event *sse_evt)
+{
+       return -EOPNOTSUPP;
+}
+
+static inline void sse_event_disable(struct sse_event *sse_evt) {}
+#endif
+#endif /* __LINUX_RISCV_SBI_SSE_H */
--

2.43.0

#9 lore raw

Re: [PATCH v7 3/5] drivers: firmware: add riscv SSE support

Quoted 30 lines · Himanshu Chauhan
Quoted 30 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +30 -0
+static int sse_pm_notifier(struct notifier_block *nb, unsigned long action,
+                          void *data)
+{
+       WARN_ON_ONCE(preemptible());
+
+       switch (action) {
+       case CPU_PM_ENTER:
+               sse_events_mask();
+               break;
+       case CPU_PM_EXIT:
+       case CPU_PM_ENTER_FAILED:
+               sse_events_unmask();
+               break;
+       default:
+               return NOTIFY_DONE;
+       }
+
+       return NOTIFY_OK;
+}
+
+static struct notifier_block sse_pm_nb = {
+       .notifier_call = sse_pm_notifier,
+};
+
+/*
+ * Mask all CPUs and unregister all events on panic, reboot or kexec.
+ */
+static int sse_reboot_notifier(struct notifier_block *nb, unsigned long action,
+                              void *data)
+{
Quoted 8 lines

drivers/firmware/riscv/riscv_sbi_sse.c line 652

Quoted 2 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +2 -0
+       cpuhp_remove_state(sse_hp_state);
+       sse_reset();

Is this sse_reset required?
I was running the poweroff command. The CPU teardown function is called and events are disabled & unregistered as part of it. when sse_reset is called, it again tries to call disable & unregister events for all cpus (nolock version). So I see errors from SBI that event is in invalid state. Commenting out sse_reset() wo

Hi Himanshu,

Yeah that is useless. I think it was there before you add PM support and wasn't removed then. I'll remove it.

Thanks,

Clément

#10 lore raw

Re: [PATCH v7 3/5] drivers: firmware: add riscv SSE support

In reply to Clément Léger
Quoted 698 lines · Clément Léger

Add driver level interface to use RISC-V SSE arch support. This interface allows registering SSE handlers, and receive them. This will be used by PMU and GHES driver.

---
 MAINTAINERS                            |  15 +
 drivers/firmware/Kconfig               |   1 +
 drivers/firmware/Makefile              |   1 +
 drivers/firmware/riscv/Kconfig         |  15 +
 drivers/firmware/riscv/Makefile        |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c | 701 +++++++++++++++++++++++++
 include/linux/riscv_sbi_sse.h          |  57 ++
 7 files changed, 793 insertions(+)
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h
MAINTAINERS +15 -0
index fe168477caa4..684d23f852c3 100644
@@ -21648,6 +21648,13 @@ T:     git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
 F:     Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
 F:     drivers/iommu/riscv/
 
+RISC-V FIRMWARE DRIVERS
+M:     Conor Dooley <[email protected]>
+L:     [email protected]
+S:     Maintained
+T:     git git://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git
+F:     drivers/firmware/riscv/*
+
 RISC-V MICROCHIP FPGA SUPPORT
 M:     Conor Dooley <[email protected]>
 M:     Daire McNamara <[email protected]>
@@ -21712,6 +21719,14 @@ F:     arch/riscv/boot/dts/spacemit/
 N:     spacemit
 K:     spacemit
 
+RISC-V SSE DRIVER
+M:     Clément Léger <[email protected]>
+R:     Himanshu Chauhan <[email protected]>
+L:     [email protected]
+S:     Maintained
+F:     drivers/firmware/riscv/riscv_sse.c
+F:     include/linux/riscv_sse.h
+
 RISC-V THEAD SoC SUPPORT
 M:     Drew Fustini <[email protected]>
 M:     Guo Ren <[email protected]>
drivers/firmware/Kconfig +1 -0
index bbd2155d8483..1894df87b08e 100644
@@ -294,6 +294,7 @@ source "drivers/firmware/meson/Kconfig"
 source "drivers/firmware/microchip/Kconfig"
 source "drivers/firmware/psci/Kconfig"
 source "drivers/firmware/qcom/Kconfig"
+source "drivers/firmware/riscv/Kconfig"
 source "drivers/firmware/samsung/Kconfig"
 source "drivers/firmware/smccc/Kconfig"
 source "drivers/firmware/tegra/Kconfig"
drivers/firmware/Makefile +1 -0
index 4ddec2820c96..6cdd84570ea7 100644
@@ -34,6 +34,7 @@ obj-y                         += efi/
 obj-y                          += imx/
 obj-y                          += psci/
 obj-y                          += qcom/
+obj-y                          += riscv/
 obj-y                          += samsung/
 obj-y                          += smccc/
 obj-y                          += tegra/
drivers/firmware/riscv/Kconfig new file +15 -0
new file mode 100644
index 000000000000..ed5b663ac5f9
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menu "Risc-V Specific firmware drivers"
+depends on RISCV
+
+config RISCV_SBI_SSE
+       bool "Enable SBI Supervisor Software Events support"
+       depends on RISCV_SBI
+       default y
+       help
+         The Supervisor Software Events support allows the SBI to deliver
+         NMI-like notifications to the supervisor mode software. When enabled,
+         this option provides support to register callbacks on specific SSE
+         events.
+
+endmenu
drivers/firmware/riscv/Makefile new file +3 -0
new file mode 100644
index 000000000000..c8795d4bbb2e
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RISCV_SBI_SSE)            += riscv_sbi_sse.o
drivers/firmware/riscv/riscv_sbi_sse.c new file +582 -0
new file mode 100644
index 000000000000..57b6dad92482
@@ -0,0 +1,701 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/cpu_pm.h>
+#include <linux/hardirq.h>
+#include <linux/list.h>
+#include <linux/percpu-defs.h>
+#include <linux/reboot.h>
+#include <linux/riscv_sbi_sse.h>
+#include <linux/slab.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_event {
+       struct list_head list;
+       u32 evt_id;
+       u32 priority;
+       sse_event_handler_fn *handler;
+       void *handler_arg;
+       /* Only valid for global events */
+       unsigned int cpu;
+
+       union {
+               struct sse_registered_event *global;
+               struct sse_registered_event __percpu *local;
+       };
+};
+
+static int sse_hp_state;
+static bool sse_available __ro_after_init;
+static DEFINE_SPINLOCK(events_list_lock);
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+struct sse_registered_event {
+       struct sse_event_arch_data arch;
+       struct sse_event *event;
+       unsigned long attr;
+       bool is_enabled;
+};
+
+void sse_handle_event(struct sse_event_arch_data *arch_event,
+                     struct pt_regs *regs)
+{
+       int ret;
+       struct sse_registered_event *reg_evt =
+               container_of(arch_event, struct sse_registered_event, arch);
+       struct sse_event *evt = reg_evt->event;
+
+       ret = evt->handler(evt->evt_id, evt->handler_arg, regs);
+       if (ret)
+               pr_warn("event %x handler failed with error %d\n", evt->evt_id, ret);
+}
+
+static struct sse_event *sse_event_get(u32 evt)
+{
+       struct sse_event *event = NULL;
+
+       scoped_guard(spinlock, &events_list_lock) {
+               list_for_each_entry(event, &events, list) {
+                       if (event->evt_id == evt)
+                               return event;
+               }
+       }
+
+       return NULL;
+}
+
+static phys_addr_t sse_event_get_attr_phys(struct sse_registered_event *reg_evt)
+{
+       phys_addr_t phys;
+       void *addr = &reg_evt->attr;
+
+       if (sse_event_is_global(reg_evt->event->evt_id))
+               phys = virt_to_phys(addr);
+       else
+               phys = per_cpu_ptr_to_phys(addr);
+
+       return phys;
+}
+
+static struct sse_registered_event *sse_get_reg_evt(struct sse_event *event)
+{
+       if (sse_event_is_global(event->evt_id))
+               return event->global;
+       else
+               return per_cpu_ptr(event->local, smp_processor_id());
+}
+
+static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
+{
+       struct sbiret ret;
+       u32 evt = event->evt_id;
+       struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+
+       ret = sbi_ecall(SBI_EXT_SSE, func, evt, 0, 0, 0, 0, 0);
+       if (ret.error) {
+               pr_warn("Failed to execute func %lx, event %x, error %ld\n",
+                       func, evt, ret.error);
+               return sbi_err_map_linux_errno(ret.error);
+       }
+
+       if (func == SBI_SSE_EVENT_DISABLE)
+               reg_evt->is_enabled = false;
+       else if (func == SBI_SSE_EVENT_ENABLE)
+               reg_evt->is_enabled = true;
+
+       return 0;
+}
+
+int sse_event_disable_local(struct sse_event *event)
+{
+       return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
+}
+EXPORT_SYMBOL_GPL(sse_event_disable_local);
+
+int sse_event_enable_local(struct sse_event *event)
+{
+       return sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
+}
+EXPORT_SYMBOL_GPL(sse_event_enable_local);
+
+static int sse_event_attr_get_no_lock(struct sse_registered_event *reg_evt,
+                                     unsigned long attr_id, unsigned long *val)
+{
+       struct sbiret sret;
+       u32 evt = reg_evt->event->evt_id;
+       unsigned long phys;
+
+       phys = sse_event_get_attr_phys(reg_evt);
+
+       sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, evt, attr_id, 1,
+                        phys, 0, 0);
+       if (sret.error) {
+               pr_debug("Failed to get event %x attr %lx, error %ld\n", evt,
+                        attr_id, sret.error);
+               return sbi_err_map_linux_errno(sret.error);
+       }
+
+       *val = reg_evt->attr;
+
+       return 0;
+}
+
+static int sse_event_attr_set_nolock(struct sse_registered_event *reg_evt,
+                                    unsigned long attr_id, unsigned long val)
+{
+       struct sbiret sret;
+       u32 evt = reg_evt->event->evt_id;
+       unsigned long phys;
+
+       reg_evt->attr = val;
+       phys = sse_event_get_attr_phys(reg_evt);
+
+       sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_WRITE, evt, attr_id, 1,
+                        phys, 0, 0);
+       if (sret.error)
+               pr_debug("Failed to set event %x attr %lx, error %ld\n", evt,
+                        attr_id, sret.error);
+
+       return sbi_err_map_linux_errno(sret.error);
+}
+
+static void sse_global_event_update_cpu(struct sse_event *event,
+                                       unsigned int cpu)
+{
+       struct sse_registered_event *reg_evt = event->global;
+
+       event->cpu = cpu;
+       arch_sse_event_update_cpu(&reg_evt->arch, cpu);
+}
+
+static int sse_event_set_target_cpu_nolock(struct sse_event *event,
+                                          unsigned int cpu)
+{
+       unsigned long hart_id = cpuid_to_hartid_map(cpu);
+       struct sse_registered_event *reg_evt = event->global;
+       u32 evt = event->evt_id;
+       bool was_enabled;
+       int ret;
+
+       if (!sse_event_is_global(evt))
+               return -EINVAL;
+
+       was_enabled = reg_evt->is_enabled;
+       if (was_enabled)
+               sse_event_disable_local(event);
+
+       ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PREFERRED_HART,
+                                       hart_id);
+       if (ret == 0)
+               sse_global_event_update_cpu(event, cpu);
+
+       if (was_enabled)
+               sse_event_enable_local(event);
+
+       return 0;
+}
+
+int sse_event_set_target_cpu(struct sse_event *event, unsigned int cpu)
+{
+       int ret;
+
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock) {
+                       if (!cpu_online(cpu))
+                               return -EINVAL;
+
+                       ret = sse_event_set_target_cpu_nolock(event, cpu);
+               }
+       }
+
+       return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_set_target_cpu);
+
+static int sse_event_init_registered(unsigned int cpu,
+                                    struct sse_registered_event *reg_evt,
+                                    struct sse_event *event)
+{
+       reg_evt->event = event;
+
+       return arch_sse_init_event(&reg_evt->arch, event->evt_id, cpu);
+}
+
+static void sse_event_free_registered(struct sse_registered_event *reg_evt)
+{
+       arch_sse_free_event(&reg_evt->arch);
+}
+
+static int sse_event_alloc_global(struct sse_event *event)
+{
+       int err;
+       struct sse_registered_event *reg_evt;
+
+       reg_evt = kzalloc(sizeof(*reg_evt), GFP_KERNEL);
+       if (!reg_evt)
+               return -ENOMEM;
+
+       event->global = reg_evt;
+       err = sse_event_init_registered(smp_processor_id(), reg_evt, event);
+       if (err)
+               kfree(reg_evt);
+
+       return err;
+}
+
+static int sse_event_alloc_local(struct sse_event *event)
+{
+       int err;
+       unsigned int cpu, err_cpu;
+       struct sse_registered_event *reg_evt;
+       struct sse_registered_event __percpu *reg_evts;
+
+       reg_evts = alloc_percpu(struct sse_registered_event);
+       if (!reg_evts)
+               return -ENOMEM;
+
+       event->local = reg_evts;
+
+       for_each_possible_cpu(cpu) {
+               reg_evt = per_cpu_ptr(reg_evts, cpu);
+               err = sse_event_init_registered(cpu, reg_evt, event);
+               if (err) {
+                       err_cpu = cpu;
+                       goto err_free_per_cpu;
+               }
+       }
+
+       return 0;
+
+err_free_per_cpu:
+       for_each_possible_cpu(cpu) {
+               if (cpu == err_cpu)
+                       break;
+               reg_evt = per_cpu_ptr(reg_evts, cpu);
+               sse_event_free_registered(reg_evt);
+       }
+
+       free_percpu(reg_evts);
+
+       return err;
+}
+
+static struct sse_event *sse_event_alloc(u32 evt, u32 priority,
+                                        sse_event_handler_fn *handler, void *arg)
+{
+       int err;
+       struct sse_event *event;
+
+       event = kzalloc(sizeof(*event), GFP_KERNEL);
+       if (!event)
+               return ERR_PTR(-ENOMEM);
+
+       event->evt_id = evt;
+       event->priority = priority;
+       event->handler_arg = arg;
+       event->handler = handler;
+
+       if (sse_event_is_global(evt))
+               err = sse_event_alloc_global(event);
+       else
+               err = sse_event_alloc_local(event);
+
+       if (err) {
+               kfree(event);
+               return ERR_PTR(err);
+       }
+
+       return event;
+}
+
+static int sse_sbi_register_event(struct sse_event *event,
+                                 struct sse_registered_event *reg_evt)
+{
+       int ret;
+
+       ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PRIO,
+                                       event->priority);
+       if (ret)
+               return ret;
+
+       return arch_sse_register_event(&reg_evt->arch);
+}
+
+static int sse_event_register_local(struct sse_event *event)
+{
+       int ret;
+       struct sse_registered_event *reg_evt;
+
+       reg_evt = per_cpu_ptr(event->local, smp_processor_id());
+       ret = sse_sbi_register_event(event, reg_evt);
+       if (ret)
+               pr_debug("Failed to register event %x: err %d\n", event->evt_id,
+                        ret);
+
+       return ret;
+}
+
+static int sse_sbi_unregister_event(struct sse_event *event)
+{
+       return sse_sbi_event_func(event, SBI_SSE_EVENT_UNREGISTER);
+}
+
+struct sse_per_cpu_evt {
+       struct sse_event *event;
+       unsigned long func;
+       cpumask_t error;
+};
+
+static void sse_event_per_cpu_func(void *info)
+{
+       int ret;
+       struct sse_per_cpu_evt *cpu_evt = info;
+
+       if (cpu_evt->func == SBI_SSE_EVENT_REGISTER)
+               ret = sse_event_register_local(cpu_evt->event);
+       else
+               ret = sse_sbi_event_func(cpu_evt->event, cpu_evt->func);
+
+       if (ret)
+               cpumask_set_cpu(smp_processor_id(), &cpu_evt->error);
+}
+
+static void sse_event_free(struct sse_event *event)
+{
+       unsigned int cpu;
+       struct sse_registered_event *reg_evt;
+
+       if (sse_event_is_global(event->evt_id)) {
+               sse_event_free_registered(event->global);
+               kfree(event->global);
+       } else {
+               for_each_possible_cpu(cpu) {
+                       reg_evt = per_cpu_ptr(event->local, cpu);
+                       sse_event_free_registered(reg_evt);
+               }
+               free_percpu(event->local);
+       }
+
+       kfree(event);
+}
+
+static int sse_on_each_cpu(struct sse_event *event, unsigned long func,
+                          unsigned long revert_func)
+{
+       struct sse_per_cpu_evt cpu_evt;
+
+       cpu_evt.event = event;
+       cpumask_clear(&cpu_evt.error);
+       cpu_evt.func = func;
+       on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+       /*
+        * If there are some error reported by CPUs, revert event state on the
+        * other ones
+        */
+       if (!cpumask_empty(&cpu_evt.error)) {
+               cpumask_t revert;
+
+               cpumask_andnot(&revert, cpu_online_mask, &cpu_evt.error);
+               cpu_evt.func = revert_func;
+               on_each_cpu_mask(&revert, sse_event_per_cpu_func, &cpu_evt, 1);
+
+               return -EIO;
+       }
+
+       return 0;
+}
+
+int sse_event_enable(struct sse_event *event)
+{
+       int ret = 0;
+
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock) {
+                       if (sse_event_is_global(event->evt_id)) {
+                               ret = sse_event_enable_local(event);
+                       } else {
+                               ret = sse_on_each_cpu(event,
+                                                     SBI_SSE_EVENT_ENABLE,
+                                                     SBI_SSE_EVENT_DISABLE);
+                       }
+               }
+       }
+       return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_enable);
+
+static int sse_events_mask(void)
+{
+       struct sbiret ret;
+
+       ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_MASK, 0, 0, 0, 0, 0, 0);
+
+       return sbi_err_map_linux_errno(ret.error);
+}
+
+static int sse_events_unmask(void)
+{
+       struct sbiret ret;
+
+       ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_UNMASK, 0, 0, 0, 0, 0, 0);
+
+       return sbi_err_map_linux_errno(ret.error);
+}
+
+static void sse_event_disable_nolock(struct sse_event *event)
+{
+       struct sse_per_cpu_evt cpu_evt;
+
+       if (sse_event_is_global(event->evt_id)) {
+               sse_event_disable_local(event);
+       } else {
+               cpu_evt.event = event;
+               cpu_evt.func = SBI_SSE_EVENT_DISABLE;
+               on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+       }
+}
+
+void sse_event_disable(struct sse_event *event)
+{
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock) {
+                       sse_event_disable_nolock(event);
+               }
+       }
+}
+EXPORT_SYMBOL_GPL(sse_event_disable);
+
+struct sse_event *sse_event_register(u32 evt, u32 priority,
+                                    sse_event_handler_fn *handler, void *arg)
+{
+       struct sse_event *event;
+       int cpu;
+       int ret = 0;
+
+       if (!sse_available)
+               return ERR_PTR(-EOPNOTSUPP);
+
+       guard(mutex)(&sse_mutex);
+       if (sse_event_get(evt))
+               return ERR_PTR(-EEXIST);
+
+       event = sse_event_alloc(evt, priority, handler, arg);
+       if (IS_ERR(event))
+               return event;
+
+       scoped_guard(cpus_read_lock) {
+               if (sse_event_is_global(evt)) {
+                       unsigned long preferred_hart;
+
+                       ret = sse_event_attr_get_no_lock(event->global,
+                                                       SBI_SSE_ATTR_PREFERRED_HART,
+                                                       &preferred_hart);
+                       if (ret)
+                               goto err_event_free;
+
+                       cpu = riscv_hartid_to_cpuid(preferred_hart);
+                       sse_global_event_update_cpu(event, cpu);
+
+                       ret = sse_sbi_register_event(event, event->global);
+                       if (ret)
+                               goto err_event_free;
+
+               } else {
+                       ret = sse_on_each_cpu(event, SBI_SSE_EVENT_REGISTER,
+                                             SBI_SSE_EVENT_DISABLE);
+                       if (ret)
+                               goto err_event_free;
+               }
+       }
+
+       scoped_guard(spinlock, &events_list_lock)
+               list_add(&event->list, &events);
+
+       return event;
+
+err_event_free:
+       sse_event_free(event);
+
+       return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(sse_event_register);
+
+static void sse_event_unregister_nolock(struct sse_event *event)
+{
+       struct sse_per_cpu_evt cpu_evt;
+
+       if (sse_event_is_global(event->evt_id)) {
+               sse_sbi_unregister_event(event);
+       } else {
+               cpu_evt.event = event;
+               cpu_evt.func = SBI_SSE_EVENT_UNREGISTER;
+               on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+       }
+}
+
+void sse_event_unregister(struct sse_event *event)
+{
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock)
+                       sse_event_unregister_nolock(event);
+
+               scoped_guard(spinlock, &events_list_lock)
+                       list_del(&event->list);
+
+               sse_event_free(event);
+       }
+}
+EXPORT_SYMBOL_GPL(sse_event_unregister);
+
+static int sse_cpu_online(unsigned int cpu)
+{
+       struct sse_event *event;
+
+       scoped_guard(spinlock, &events_list_lock) {
+               list_for_each_entry(event, &events, list) {
+                       if (sse_event_is_global(event->evt_id))
+                               continue;
+
+                       sse_event_register_local(event);
+                       if (sse_get_reg_evt(event))
+                               sse_event_enable_local(event);
+               }
+       }
+
+       /* Ready to handle events. Unmask SSE. */
+       return sse_events_unmask();
+}
+
+static int sse_cpu_teardown(unsigned int cpu)
+{
+       int ret = 0;
+       unsigned int next_cpu;
+       struct sse_event *event;

drivers/firmware/riscv/riscv_sbi_sse.c line 592

Quoted 10 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +10 -0
+
+       /* Mask the sse events */
+       ret = sse_events_mask();
+       if (ret)
+               return ret;
+
+       scoped_guard(spinlock, &events_list_lock) {
+               list_for_each_entry(event, &events, list) {
+                       if (!sse_event_is_global(event->evt_id)) {
+                               if (event->global->is_enabled)

For consistency, it would be better to get the registered event for the event using the API sse_get_reg_evt(event).

Quoted 175 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +109 -0
+                                       sse_event_disable_local(event);
+
+                               sse_sbi_unregister_event(event);
+                               continue;
+                       }
+
+                       if (event->cpu != smp_processor_id())
+                               continue;
+
+                       /* Update destination hart for global event */
+                       next_cpu = cpumask_any_but(cpu_online_mask, cpu);
+                       ret = sse_event_set_target_cpu_nolock(event, next_cpu);
+               }
+       }
+
+       return ret;
+}
+
+static void sse_reset(void)
+{
+       struct sse_event *event;
+
+       list_for_each_entry(event, &events, list) {
+               sse_event_disable_nolock(event);
+               sse_event_unregister_nolock(event);
+       }
+}
+
+static int sse_pm_notifier(struct notifier_block *nb, unsigned long action,
+                          void *data)
+{
+       WARN_ON_ONCE(preemptible());
+
+       switch (action) {
+       case CPU_PM_ENTER:
+               sse_events_mask();
+               break;
+       case CPU_PM_EXIT:
+       case CPU_PM_ENTER_FAILED:
+               sse_events_unmask();
+               break;
+       default:
+               return NOTIFY_DONE;
+       }
+
+       return NOTIFY_OK;
+}
+
+static struct notifier_block sse_pm_nb = {
+       .notifier_call = sse_pm_notifier,
+};
+
+/*
+ * Mask all CPUs and unregister all events on panic, reboot or kexec.
+ */
+static int sse_reboot_notifier(struct notifier_block *nb, unsigned long action,
+                              void *data)
+{
+       cpuhp_remove_state(sse_hp_state);
+       sse_reset();
+
+       return NOTIFY_OK;
+}
+
+static struct notifier_block sse_reboot_nb = {
+       .notifier_call = sse_reboot_notifier,
+};
+
+static int __init sse_init(void)
+{
+       int ret;
+
+       if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+               pr_err("Missing SBI SSE extension\n");
+               return -EOPNOTSUPP;
+       }
+       pr_info("SBI SSE extension detected\n");
+
+       ret = cpu_pm_register_notifier(&sse_pm_nb);
+       if (ret) {
+               pr_warn("Failed to register CPU PM notifier...\n");
+               return ret;
+       }
+
+       ret = register_reboot_notifier(&sse_reboot_nb);
+       if (ret) {
+               pr_warn("Failed to register reboot notifier...\n");
+               goto remove_cpupm;
+       }
+
+       ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "riscv/sse:online",
+                               sse_cpu_online, sse_cpu_teardown);
+       if (ret < 0)
+               goto remove_reboot;
+
+       sse_hp_state = ret;
+       sse_available = true;
+
+       return 0;
+
+remove_reboot:
+       unregister_reboot_notifier(&sse_reboot_nb);
+
+remove_cpupm:
+       cpu_pm_unregister_notifier(&sse_pm_nb);
+
+       return ret;
+}
+arch_initcall(sse_init);
include/linux/riscv_sbi_sse.h new file +57 -1
new file mode 100644
index 000000000000..a1b58e89dd19
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+
+#ifndef __LINUX_RISCV_SBI_SSE_H
+#define __LINUX_RISCV_SBI_SSE_H
+
+#include <linux/types.h>
+#include <linux/linkage.h>
+
+struct sse_event;
+struct pt_regs;
+
+typedef int (sse_event_handler_fn)(u32 event_num, void *arg,
+                                  struct pt_regs *regs);
+
+#ifdef CONFIG_RISCV_SBI_SSE
+
+struct sse_event *sse_event_register(u32 event_num, u32 priority,
+                                    sse_event_handler_fn *handler, void *arg);
+
+void sse_event_unregister(struct sse_event *evt);
+
+int sse_event_set_target_cpu(struct sse_event *sse_evt, unsigned int cpu);
+
+int sse_event_enable(struct sse_event *sse_evt);
+
+void sse_event_disable(struct sse_event *sse_evt);
+
+int sse_event_enable_local(struct sse_event *sse_evt);
+int sse_event_disable_local(struct sse_event *sse_evt);
+
+#else
+static inline struct sse_event *sse_event_register(u32 event_num, u32 priority,
+                                                  sse_event_handler_fn *handler,
+                                                  void *arg)
+{
+       return ERR_PTR(-EOPNOTSUPP);
+}
+
+static inline void sse_event_unregister(struct sse_event *evt) {}
+
+static inline int sse_event_set_target_cpu(struct sse_event *sse_evt,
+                                          unsigned int cpu)
+{
+       return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable(struct sse_event *sse_evt)
+{
+       return -EOPNOTSUPP;
+}
+
+static inline void sse_event_disable(struct sse_event *sse_evt) {}
+#endif
+#endif /* __LINUX_RISCV_SBI_SSE_H */
--

2.43.0

#11 lore raw

Re: [PATCH v7 3/5] drivers: firmware: add riscv SSE support

Hi Clement,

Quoted 705 lines · Himanshu Chauhan
Quoted 703 lines · Clément Léger

Add driver level interface to use RISC-V SSE arch support. This interface allows registering SSE handlers, and receive them. This will be used by PMU and GHES driver.

---
 MAINTAINERS                            |  15 +
 drivers/firmware/Kconfig               |   1 +
 drivers/firmware/Makefile              |   1 +
 drivers/firmware/riscv/Kconfig         |  15 +
 drivers/firmware/riscv/Makefile        |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c | 701 +++++++++++++++++++++++++
 include/linux/riscv_sbi_sse.h          |  57 ++
 7 files changed, 793 insertions(+)
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h
MAINTAINERS +15 -0
index fe168477caa4..684d23f852c3 100644
@@ -21648,6 +21648,13 @@ T:     git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
 F:     Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
 F:     drivers/iommu/riscv/
 
+RISC-V FIRMWARE DRIVERS
+M:     Conor Dooley <[email protected]>
+L:     [email protected]
+S:     Maintained
+T:     git git://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git
+F:     drivers/firmware/riscv/*
+
 RISC-V MICROCHIP FPGA SUPPORT
 M:     Conor Dooley <[email protected]>
 M:     Daire McNamara <[email protected]>
@@ -21712,6 +21719,14 @@ F:     arch/riscv/boot/dts/spacemit/
 N:     spacemit
 K:     spacemit
 
+RISC-V SSE DRIVER
+M:     Clément Léger <[email protected]>
+R:     Himanshu Chauhan <[email protected]>
+L:     [email protected]
+S:     Maintained
+F:     drivers/firmware/riscv/riscv_sse.c
+F:     include/linux/riscv_sse.h
+
 RISC-V THEAD SoC SUPPORT
 M:     Drew Fustini <[email protected]>
 M:     Guo Ren <[email protected]>
drivers/firmware/Kconfig +1 -0
index bbd2155d8483..1894df87b08e 100644
@@ -294,6 +294,7 @@ source "drivers/firmware/meson/Kconfig"
 source "drivers/firmware/microchip/Kconfig"
 source "drivers/firmware/psci/Kconfig"
 source "drivers/firmware/qcom/Kconfig"
+source "drivers/firmware/riscv/Kconfig"
 source "drivers/firmware/samsung/Kconfig"
 source "drivers/firmware/smccc/Kconfig"
 source "drivers/firmware/tegra/Kconfig"
drivers/firmware/Makefile +1 -0
index 4ddec2820c96..6cdd84570ea7 100644
@@ -34,6 +34,7 @@ obj-y                         += efi/
 obj-y                          += imx/
 obj-y                          += psci/
 obj-y                          += qcom/
+obj-y                          += riscv/
 obj-y                          += samsung/
 obj-y                          += smccc/
 obj-y                          += tegra/
drivers/firmware/riscv/Kconfig new file +15 -0
new file mode 100644
index 000000000000..ed5b663ac5f9
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menu "Risc-V Specific firmware drivers"
+depends on RISCV
+
+config RISCV_SBI_SSE
+       bool "Enable SBI Supervisor Software Events support"
+       depends on RISCV_SBI
+       default y
+       help
+         The Supervisor Software Events support allows the SBI to deliver
+         NMI-like notifications to the supervisor mode software. When enabled,
+         this option provides support to register callbacks on specific SSE
+         events.
+
+endmenu
drivers/firmware/riscv/Makefile new file +3 -0
new file mode 100644
index 000000000000..c8795d4bbb2e
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RISCV_SBI_SSE)            += riscv_sbi_sse.o
drivers/firmware/riscv/riscv_sbi_sse.c new file +587 -0
new file mode 100644
index 000000000000..57b6dad92482
@@ -0,0 +1,701 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/cpu_pm.h>
+#include <linux/hardirq.h>
+#include <linux/list.h>
+#include <linux/percpu-defs.h>
+#include <linux/reboot.h>
+#include <linux/riscv_sbi_sse.h>
+#include <linux/slab.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_event {
+       struct list_head list;
+       u32 evt_id;
+       u32 priority;
+       sse_event_handler_fn *handler;
+       void *handler_arg;
+       /* Only valid for global events */
+       unsigned int cpu;
+
+       union {
+               struct sse_registered_event *global;
+               struct sse_registered_event __percpu *local;
+       };
+};
+
+static int sse_hp_state;
+static bool sse_available __ro_after_init;
+static DEFINE_SPINLOCK(events_list_lock);
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+struct sse_registered_event {
+       struct sse_event_arch_data arch;
+       struct sse_event *event;
+       unsigned long attr;
+       bool is_enabled;
+};
+
+void sse_handle_event(struct sse_event_arch_data *arch_event,
+                     struct pt_regs *regs)
+{
+       int ret;
+       struct sse_registered_event *reg_evt =
+               container_of(arch_event, struct sse_registered_event, arch);
+       struct sse_event *evt = reg_evt->event;
+
+       ret = evt->handler(evt->evt_id, evt->handler_arg, regs);
+       if (ret)
+               pr_warn("event %x handler failed with error %d\n", evt->evt_id, ret);
+}
+
+static struct sse_event *sse_event_get(u32 evt)
+{
+       struct sse_event *event = NULL;
+
+       scoped_guard(spinlock, &events_list_lock) {
+               list_for_each_entry(event, &events, list) {
+                       if (event->evt_id == evt)
+                               return event;
+               }
+       }
+
+       return NULL;
+}
+
+static phys_addr_t sse_event_get_attr_phys(struct sse_registered_event *reg_evt)
+{
+       phys_addr_t phys;
+       void *addr = &reg_evt->attr;
+
+       if (sse_event_is_global(reg_evt->event->evt_id))
+               phys = virt_to_phys(addr);
+       else
+               phys = per_cpu_ptr_to_phys(addr);
+
+       return phys;
+}
+
+static struct sse_registered_event *sse_get_reg_evt(struct sse_event *event)
+{
+       if (sse_event_is_global(event->evt_id))
+               return event->global;
+       else
+               return per_cpu_ptr(event->local, smp_processor_id());
+}
+
+static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
+{
+       struct sbiret ret;
+       u32 evt = event->evt_id;
+       struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+
+       ret = sbi_ecall(SBI_EXT_SSE, func, evt, 0, 0, 0, 0, 0);
+       if (ret.error) {
+               pr_warn("Failed to execute func %lx, event %x, error %ld\n",
+                       func, evt, ret.error);
+               return sbi_err_map_linux_errno(ret.error);
+       }
+
+       if (func == SBI_SSE_EVENT_DISABLE)
+               reg_evt->is_enabled = false;
+       else if (func == SBI_SSE_EVENT_ENABLE)
+               reg_evt->is_enabled = true;
+
+       return 0;
+}
+
+int sse_event_disable_local(struct sse_event *event)
+{
+       return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
+}
+EXPORT_SYMBOL_GPL(sse_event_disable_local);
+
+int sse_event_enable_local(struct sse_event *event)
+{
+       return sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
+}
+EXPORT_SYMBOL_GPL(sse_event_enable_local);
+
+static int sse_event_attr_get_no_lock(struct sse_registered_event *reg_evt,
+                                     unsigned long attr_id, unsigned long *val)
+{
+       struct sbiret sret;
+       u32 evt = reg_evt->event->evt_id;
+       unsigned long phys;
+
+       phys = sse_event_get_attr_phys(reg_evt);
+
+       sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, evt, attr_id, 1,
+                        phys, 0, 0);
+       if (sret.error) {
+               pr_debug("Failed to get event %x attr %lx, error %ld\n", evt,
+                        attr_id, sret.error);
+               return sbi_err_map_linux_errno(sret.error);
+       }
+
+       *val = reg_evt->attr;
+
+       return 0;
+}
+
+static int sse_event_attr_set_nolock(struct sse_registered_event *reg_evt,
+                                    unsigned long attr_id, unsigned long val)
+{
+       struct sbiret sret;
+       u32 evt = reg_evt->event->evt_id;
+       unsigned long phys;
+
+       reg_evt->attr = val;
+       phys = sse_event_get_attr_phys(reg_evt);
+
+       sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_WRITE, evt, attr_id, 1,
+                        phys, 0, 0);
+       if (sret.error)
+               pr_debug("Failed to set event %x attr %lx, error %ld\n", evt,
+                        attr_id, sret.error);
+
+       return sbi_err_map_linux_errno(sret.error);
+}
+
+static void sse_global_event_update_cpu(struct sse_event *event,
+                                       unsigned int cpu)
+{
+       struct sse_registered_event *reg_evt = event->global;
+
+       event->cpu = cpu;
+       arch_sse_event_update_cpu(&reg_evt->arch, cpu);
+}
+
+static int sse_event_set_target_cpu_nolock(struct sse_event *event,
+                                          unsigned int cpu)
+{
+       unsigned long hart_id = cpuid_to_hartid_map(cpu);
+       struct sse_registered_event *reg_evt = event->global;
+       u32 evt = event->evt_id;
+       bool was_enabled;
+       int ret;
+
+       if (!sse_event_is_global(evt))
+               return -EINVAL;
+
+       was_enabled = reg_evt->is_enabled;
+       if (was_enabled)
+               sse_event_disable_local(event);
+
+       ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PREFERRED_HART,
+                                       hart_id);
+       if (ret == 0)
+               sse_global_event_update_cpu(event, cpu);
+
+       if (was_enabled)
+               sse_event_enable_local(event);
+
+       return 0;
+}
+
+int sse_event_set_target_cpu(struct sse_event *event, unsigned int cpu)
+{
+       int ret;
+
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock) {
+                       if (!cpu_online(cpu))
+                               return -EINVAL;
+
+                       ret = sse_event_set_target_cpu_nolock(event, cpu);
+               }
+       }
+
+       return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_set_target_cpu);
+
+static int sse_event_init_registered(unsigned int cpu,
+                                    struct sse_registered_event *reg_evt,
+                                    struct sse_event *event)
+{
+       reg_evt->event = event;
+
+       return arch_sse_init_event(&reg_evt->arch, event->evt_id, cpu);
+}
+
+static void sse_event_free_registered(struct sse_registered_event *reg_evt)
+{
+       arch_sse_free_event(&reg_evt->arch);
+}
+
+static int sse_event_alloc_global(struct sse_event *event)
+{
+       int err;
+       struct sse_registered_event *reg_evt;
+
+       reg_evt = kzalloc(sizeof(*reg_evt), GFP_KERNEL);
+       if (!reg_evt)
+               return -ENOMEM;
+
+       event->global = reg_evt;
+       err = sse_event_init_registered(smp_processor_id(), reg_evt, event);
+       if (err)
+               kfree(reg_evt);
+
+       return err;
+}
+
+static int sse_event_alloc_local(struct sse_event *event)
+{
+       int err;
+       unsigned int cpu, err_cpu;
+       struct sse_registered_event *reg_evt;
+       struct sse_registered_event __percpu *reg_evts;
+
+       reg_evts = alloc_percpu(struct sse_registered_event);
+       if (!reg_evts)
+               return -ENOMEM;
+
+       event->local = reg_evts;
+
+       for_each_possible_cpu(cpu) {
+               reg_evt = per_cpu_ptr(reg_evts, cpu);
+               err = sse_event_init_registered(cpu, reg_evt, event);
+               if (err) {
+                       err_cpu = cpu;
+                       goto err_free_per_cpu;
+               }
+       }
+
+       return 0;
+
+err_free_per_cpu:
+       for_each_possible_cpu(cpu) {
+               if (cpu == err_cpu)
+                       break;
+               reg_evt = per_cpu_ptr(reg_evts, cpu);
+               sse_event_free_registered(reg_evt);
+       }
+
+       free_percpu(reg_evts);
+
+       return err;
+}
+
+static struct sse_event *sse_event_alloc(u32 evt, u32 priority,
+                                        sse_event_handler_fn *handler, void *arg)
+{
+       int err;
+       struct sse_event *event;
+
+       event = kzalloc(sizeof(*event), GFP_KERNEL);
+       if (!event)
+               return ERR_PTR(-ENOMEM);
+
+       event->evt_id = evt;
+       event->priority = priority;
+       event->handler_arg = arg;
+       event->handler = handler;
+
+       if (sse_event_is_global(evt))
+               err = sse_event_alloc_global(event);
+       else
+               err = sse_event_alloc_local(event);
+
+       if (err) {
+               kfree(event);
+               return ERR_PTR(err);
+       }
+
+       return event;
+}
+
+static int sse_sbi_register_event(struct sse_event *event,
+                                 struct sse_registered_event *reg_evt)
+{
+       int ret;
+
+       ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PRIO,
+                                       event->priority);
+       if (ret)
+               return ret;
+
+       return arch_sse_register_event(&reg_evt->arch);
+}
+
+static int sse_event_register_local(struct sse_event *event)
+{
+       int ret;
+       struct sse_registered_event *reg_evt;
+
+       reg_evt = per_cpu_ptr(event->local, smp_processor_id());
+       ret = sse_sbi_register_event(event, reg_evt);
+       if (ret)
+               pr_debug("Failed to register event %x: err %d\n", event->evt_id,
+                        ret);
+
+       return ret;
+}
+
+static int sse_sbi_unregister_event(struct sse_event *event)
+{
+       return sse_sbi_event_func(event, SBI_SSE_EVENT_UNREGISTER);
+}
+
+struct sse_per_cpu_evt {
+       struct sse_event *event;
+       unsigned long func;
+       cpumask_t error;
+};
+
+static void sse_event_per_cpu_func(void *info)
+{
+       int ret;
+       struct sse_per_cpu_evt *cpu_evt = info;
+
+       if (cpu_evt->func == SBI_SSE_EVENT_REGISTER)
+               ret = sse_event_register_local(cpu_evt->event);
+       else
+               ret = sse_sbi_event_func(cpu_evt->event, cpu_evt->func);
+
+       if (ret)
+               cpumask_set_cpu(smp_processor_id(), &cpu_evt->error);
+}
+
+static void sse_event_free(struct sse_event *event)
+{
+       unsigned int cpu;
+       struct sse_registered_event *reg_evt;
+
+       if (sse_event_is_global(event->evt_id)) {
+               sse_event_free_registered(event->global);
+               kfree(event->global);
+       } else {
+               for_each_possible_cpu(cpu) {
+                       reg_evt = per_cpu_ptr(event->local, cpu);
+                       sse_event_free_registered(reg_evt);
+               }
+               free_percpu(event->local);
+       }
+
+       kfree(event);
+}
+
+static int sse_on_each_cpu(struct sse_event *event, unsigned long func,
+                          unsigned long revert_func)
+{
+       struct sse_per_cpu_evt cpu_evt;
+
+       cpu_evt.event = event;
+       cpumask_clear(&cpu_evt.error);
+       cpu_evt.func = func;
+       on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+       /*
+        * If there are some error reported by CPUs, revert event state on the
+        * other ones
+        */
+       if (!cpumask_empty(&cpu_evt.error)) {
+               cpumask_t revert;
+
+               cpumask_andnot(&revert, cpu_online_mask, &cpu_evt.error);
+               cpu_evt.func = revert_func;
+               on_each_cpu_mask(&revert, sse_event_per_cpu_func, &cpu_evt, 1);
+
+               return -EIO;
+       }
+
+       return 0;
+}
+
+int sse_event_enable(struct sse_event *event)
+{
+       int ret = 0;
+
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock) {
+                       if (sse_event_is_global(event->evt_id)) {
+                               ret = sse_event_enable_local(event);
+                       } else {
+                               ret = sse_on_each_cpu(event,
+                                                     SBI_SSE_EVENT_ENABLE,
+                                                     SBI_SSE_EVENT_DISABLE);
+                       }
+               }
+       }
+       return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_enable);
+
+static int sse_events_mask(void)
+{
+       struct sbiret ret;
+
+       ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_MASK, 0, 0, 0, 0, 0, 0);
+
+       return sbi_err_map_linux_errno(ret.error);
+}
+
+static int sse_events_unmask(void)
+{
+       struct sbiret ret;
+
+       ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_UNMASK, 0, 0, 0, 0, 0, 0);
+
+       return sbi_err_map_linux_errno(ret.error);
+}
+
+static void sse_event_disable_nolock(struct sse_event *event)
+{
+       struct sse_per_cpu_evt cpu_evt;
+
+       if (sse_event_is_global(event->evt_id)) {
+               sse_event_disable_local(event);
+       } else {
+               cpu_evt.event = event;
+               cpu_evt.func = SBI_SSE_EVENT_DISABLE;
+               on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+       }
+}
+
+void sse_event_disable(struct sse_event *event)
+{
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock) {
+                       sse_event_disable_nolock(event);
+               }
+       }
+}
+EXPORT_SYMBOL_GPL(sse_event_disable);
+
+struct sse_event *sse_event_register(u32 evt, u32 priority,
+                                    sse_event_handler_fn *handler, void *arg)
+{
+       struct sse_event *event;
+       int cpu;
+       int ret = 0;
+
+       if (!sse_available)
+               return ERR_PTR(-EOPNOTSUPP);
+
+       guard(mutex)(&sse_mutex);
+       if (sse_event_get(evt))
+               return ERR_PTR(-EEXIST);
+
+       event = sse_event_alloc(evt, priority, handler, arg);
+       if (IS_ERR(event))
+               return event;
+
+       scoped_guard(cpus_read_lock) {
+               if (sse_event_is_global(evt)) {
+                       unsigned long preferred_hart;
+
+                       ret = sse_event_attr_get_no_lock(event->global,
+                                                       SBI_SSE_ATTR_PREFERRED_HART,
+                                                       &preferred_hart);
+                       if (ret)
+                               goto err_event_free;
+
+                       cpu = riscv_hartid_to_cpuid(preferred_hart);
+                       sse_global_event_update_cpu(event, cpu);
+
+                       ret = sse_sbi_register_event(event, event->global);
+                       if (ret)
+                               goto err_event_free;
+
+               } else {
+                       ret = sse_on_each_cpu(event, SBI_SSE_EVENT_REGISTER,
+                                             SBI_SSE_EVENT_DISABLE);
+                       if (ret)
+                               goto err_event_free;
+               }
+       }
+
+       scoped_guard(spinlock, &events_list_lock)
+               list_add(&event->list, &events);
+
+       return event;
+
+err_event_free:
+       sse_event_free(event);
+
+       return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(sse_event_register);
+
+static void sse_event_unregister_nolock(struct sse_event *event)
+{
+       struct sse_per_cpu_evt cpu_evt;
+
+       if (sse_event_is_global(event->evt_id)) {
+               sse_sbi_unregister_event(event);
+       } else {
+               cpu_evt.event = event;
+               cpu_evt.func = SBI_SSE_EVENT_UNREGISTER;
+               on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+       }
+}
+
+void sse_event_unregister(struct sse_event *event)
+{
+       scoped_guard(mutex, &sse_mutex) {
+               scoped_guard(cpus_read_lock)
+                       sse_event_unregister_nolock(event);
+
+               scoped_guard(spinlock, &events_list_lock)
+                       list_del(&event->list);
+
+               sse_event_free(event);
+       }
+}
+EXPORT_SYMBOL_GPL(sse_event_unregister);
+
+static int sse_cpu_online(unsigned int cpu)
+{
+       struct sse_event *event;
+
+       scoped_guard(spinlock, &events_list_lock) {
+               list_for_each_entry(event, &events, list) {
+                       if (sse_event_is_global(event->evt_id))
+                               continue;
+
+                       sse_event_register_local(event);
+                       if (sse_get_reg_evt(event))
+                               sse_event_enable_local(event);
+               }
+       }
+
+       /* Ready to handle events. Unmask SSE. */
+       return sse_events_unmask();
+}
+
+static int sse_cpu_teardown(unsigned int cpu)
+{
+       int ret = 0;
+       unsigned int next_cpu;
+       struct sse_event *event;
+
+       /* Mask the sse events */
+       ret = sse_events_mask();
+       if (ret)
+               return ret;
Quoted 8 lines

drivers/firmware/riscv/riscv_sbi_sse.c line 592

Quoted 5 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +5 -0
+
+       scoped_guard(spinlock, &events_list_lock) {
+               list_for_each_entry(event, &events, list) {
+                       if (!sse_event_is_global(event->evt_id)) {
+                               if (event->global->is_enabled)

For consistency, it would be better to get the registered event for the event using the API sse_get_reg_evt(event).

Actually, it needed not just for consistency. The check above is !sse_event_is_global() and then event->global is accessed. This causes additional errors. Using sse_get_reg_evt() fixes these errors. Please make this change.

Thank you.

Quoted 176 lines
Quoted 176 lines
drivers/firmware/riscv/riscv_sbi_sse.c new file +109 -0
+                                       sse_event_disable_local(event);
+
+                               sse_sbi_unregister_event(event);
+                               continue;
+                       }
+
+                       if (event->cpu != smp_processor_id())
+                               continue;
+
+                       /* Update destination hart for global event */
+                       next_cpu = cpumask_any_but(cpu_online_mask, cpu);
+                       ret = sse_event_set_target_cpu_nolock(event, next_cpu);
+               }
+       }
+
+       return ret;
+}
+
+static void sse_reset(void)
+{
+       struct sse_event *event;
+
+       list_for_each_entry(event, &events, list) {
+               sse_event_disable_nolock(event);
+               sse_event_unregister_nolock(event);
+       }
+}
+
+static int sse_pm_notifier(struct notifier_block *nb, unsigned long action,
+                          void *data)
+{
+       WARN_ON_ONCE(preemptible());
+
+       switch (action) {
+       case CPU_PM_ENTER:
+               sse_events_mask();
+               break;
+       case CPU_PM_EXIT:
+       case CPU_PM_ENTER_FAILED:
+               sse_events_unmask();
+               break;
+       default:
+               return NOTIFY_DONE;
+       }
+
+       return NOTIFY_OK;
+}
+
+static struct notifier_block sse_pm_nb = {
+       .notifier_call = sse_pm_notifier,
+};
+
+/*
+ * Mask all CPUs and unregister all events on panic, reboot or kexec.
+ */
+static int sse_reboot_notifier(struct notifier_block *nb, unsigned long action,
+                              void *data)
+{
+       cpuhp_remove_state(sse_hp_state);
+       sse_reset();
+
+       return NOTIFY_OK;
+}
+
+static struct notifier_block sse_reboot_nb = {
+       .notifier_call = sse_reboot_notifier,
+};
+
+static int __init sse_init(void)
+{
+       int ret;
+
+       if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+               pr_err("Missing SBI SSE extension\n");
+               return -EOPNOTSUPP;
+       }
+       pr_info("SBI SSE extension detected\n");
+
+       ret = cpu_pm_register_notifier(&sse_pm_nb);
+       if (ret) {
+               pr_warn("Failed to register CPU PM notifier...\n");
+               return ret;
+       }
+
+       ret = register_reboot_notifier(&sse_reboot_nb);
+       if (ret) {
+               pr_warn("Failed to register reboot notifier...\n");
+               goto remove_cpupm;
+       }
+
+       ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "riscv/sse:online",
+                               sse_cpu_online, sse_cpu_teardown);
+       if (ret < 0)
+               goto remove_reboot;
+
+       sse_hp_state = ret;
+       sse_available = true;
+
+       return 0;
+
+remove_reboot:
+       unregister_reboot_notifier(&sse_reboot_nb);
+
+remove_cpupm:
+       cpu_pm_unregister_notifier(&sse_pm_nb);
+
+       return ret;
+}
+arch_initcall(sse_init);
include/linux/riscv_sbi_sse.h new file +57 -1
new file mode 100644
index 000000000000..a1b58e89dd19
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+
+#ifndef __LINUX_RISCV_SBI_SSE_H
+#define __LINUX_RISCV_SBI_SSE_H
+
+#include <linux/types.h>
+#include <linux/linkage.h>
+
+struct sse_event;
+struct pt_regs;
+
+typedef int (sse_event_handler_fn)(u32 event_num, void *arg,
+                                  struct pt_regs *regs);
+
+#ifdef CONFIG_RISCV_SBI_SSE
+
+struct sse_event *sse_event_register(u32 event_num, u32 priority,
+                                    sse_event_handler_fn *handler, void *arg);
+
+void sse_event_unregister(struct sse_event *evt);
+
+int sse_event_set_target_cpu(struct sse_event *sse_evt, unsigned int cpu);
+
+int sse_event_enable(struct sse_event *sse_evt);
+
+void sse_event_disable(struct sse_event *sse_evt);
+
+int sse_event_enable_local(struct sse_event *sse_evt);
+int sse_event_disable_local(struct sse_event *sse_evt);
+
+#else
+static inline struct sse_event *sse_event_register(u32 event_num, u32 priority,
+                                                  sse_event_handler_fn *handler,
+                                                  void *arg)
+{
+       return ERR_PTR(-EOPNOTSUPP);
+}
+
+static inline void sse_event_unregister(struct sse_event *evt) {}
+
+static inline int sse_event_set_target_cpu(struct sse_event *sse_evt,
+                                          unsigned int cpu)
+{
+       return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable(struct sse_event *sse_evt)
+{
+       return -EOPNOTSUPP;
+}
+
+static inline void sse_event_disable(struct sse_event *sse_evt) {}
+#endif
+#endif /* __LINUX_RISCV_SBI_SSE_H */
--

2.43.0

#12 lore raw

PATCH v7 4/5 perf: RISC-V: add support for SSE event

In reply to Clément Léger

In order to use SSE within PMU drivers, register a SSE handler for the local PMU event. Reuse the existing overflow IRQ handler and pass appropriate pt_regs. Add a config option RISCV_PMU_SSE to select event delivery via SSE events.

---
 drivers/perf/Kconfig           | 10 +++++
 drivers/perf/riscv_pmu.c       | 23 +++++++++++
 drivers/perf/riscv_pmu_sbi.c   | 71 +++++++++++++++++++++++++++++-----
 include/linux/perf/riscv_pmu.h |  5 +++
 4 files changed, 99 insertions(+), 10 deletions(-)
drivers/perf/Kconfig +10 -0
index a9188dec36fe..bea08d4689b1 100644
@@ -105,6 +105,16 @@ config RISCV_PMU_SBI
 	  full perf feature support i.e. counter overflow, privilege mode
 	  filtering, counter configuration.
 
+config RISCV_PMU_SBI_SSE
+	depends on RISCV_PMU && RISCV_SBI_SSE
+	bool "RISC-V PMU SSE events"
+	default n
+	help
+	  Say y if you want to use SSE events to deliver PMU interrupts. This
+	  provides a way to profile the kernel at any level by using NMI-like
+	  SSE events. SSE events being really intrusive, this option allows
+	  to select it only if needed.
+
 config STARFIVE_STARLINK_PMU
 	depends on ARCH_STARFIVE || COMPILE_TEST
 	depends on 64BIT
drivers/perf/riscv_pmu.c +23 -0
index 7644147d50b4..dda2814801c0 100644
@@ -13,6 +13,7 @@
 #include <linux/irqdesc.h>
 #include <linux/perf/riscv_pmu.h>
 #include <linux/printk.h>
+#include <linux/riscv_sbi_sse.h>
 #include <linux/smp.h>
 #include <linux/sched_clock.h>
 
@@ -254,6 +255,24 @@ void riscv_pmu_start(struct perf_event *event, int flags)
 	perf_event_update_userpage(event);
 }
 
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+static void riscv_pmu_disable(struct pmu *pmu)
+{
+	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
+
+	if (rvpmu->sse_evt)
+		sse_event_disable_local(rvpmu->sse_evt);
+}
+
+static void riscv_pmu_enable(struct pmu *pmu)
+{
+	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
+
+	if (rvpmu->sse_evt)
+		sse_event_enable_local(rvpmu->sse_evt);
+}
+#endif
+
 static int riscv_pmu_add(struct perf_event *event, int flags)
 {
 	struct riscv_pmu *rvpmu = to_riscv_pmu(event->pmu);
@@ -411,6 +430,10 @@ struct riscv_pmu *riscv_pmu_alloc(void)
 		.event_mapped	= riscv_pmu_event_mapped,
 		.event_unmapped	= riscv_pmu_event_unmapped,
 		.event_idx	= riscv_pmu_event_idx,
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+		.pmu_enable	= riscv_pmu_enable,
+		.pmu_disable	= riscv_pmu_disable,
+#endif
 		.add		= riscv_pmu_add,
 		.del		= riscv_pmu_del,
 		.start		= riscv_pmu_start,
drivers/perf/riscv_pmu_sbi.c +61 -10

132 lines not on this page · raw message

include/linux/perf/riscv_pmu.h +5 -0
index 701974639ff2..cd493fcab9b3 100644
@@ -28,6 +28,8 @@
 
 #define RISCV_PMU_CONFIG1_GUEST_EVENTS 0x1
 
+struct sse_event;
+
 struct cpu_hw_events {
 	/* currently enabled events */
 	int			n_events;
@@ -54,6 +56,9 @@ struct riscv_pmu {
 	char		*name;
 
 	irqreturn_t	(*handle_irq)(int irq_num, void *dev);
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	struct sse_event *sse_evt;
+#endif
 
 	unsigned long	cmask;
 	u64		(*ctr_read)(struct perf_event *event);
Signature · 1 line
2.43.0
#13 lore raw

PATCH v7 5/5 selftests/riscv: add SSE test module

In reply to Clément Léger

This module, once loaded, will execute a series of tests using the SSE framework. The provided script will check for any error reported by the test module.

---
 tools/testing/selftests/riscv/Makefile        |   2 +-
 tools/testing/selftests/riscv/sse/Makefile    |   5 +
 .../selftests/riscv/sse/module/Makefile       |  16 +
 .../riscv/sse/module/riscv_sse_test.c         | 513 ++++++++++++++++++
 .../selftests/riscv/sse/run_sse_test.sh       |  44 ++
 5 files changed, 579 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/riscv/sse/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
 create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh
tools/testing/selftests/riscv/Makefile +1 -1
index 099b8c1f46f8..c62f58414b29 100644
@@ -5,7 +5,7 @@
 ARCH ?= $(shell uname -m 2>/dev/null || echo not)
 
 ifneq (,$(filter $(ARCH),riscv))
-RISCV_SUBTARGETS ?= abi hwprobe mm sigreturn vector
+RISCV_SUBTARGETS ?= abi hwprobe mm sigreturn vector sse
 else
 RISCV_SUBTARGETS :=
 endif
tools/testing/selftests/riscv/sse/Makefile new file +5 -0
new file mode 100644
index 000000000000..67eaee06f213
@@ -0,0 +1,5 @@
+TEST_GEN_MODS_DIR := module
+
+TEST_FILES := run_sse_test.sh
+
+include ../../lib.mk
tools/testing/selftests/riscv/sse/module/Makefile new file +16 -0
new file mode 100644
index 000000000000..02018f083456
@@ -0,0 +1,16 @@
+TESTMODS_DIR := $(realpath $(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
+KDIR ?= /lib/modules/$(shell uname -r)/build
+
+obj-m += riscv_sse_test.o
+
+# Ensure that KDIR exists, otherwise skip the compilation
+modules:
+ifneq ("$(wildcard $(KDIR))", "")
+	$(Q)$(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=$(TESTMODS_DIR)
+endif
+
+# Ensure that KDIR exists, otherwise skip the clean target
+clean:
+ifneq ("$(wildcard $(KDIR))", "")
+	$(Q)$(MAKE) -C $(KDIR) clean KBUILD_EXTMOD=$(TESTMODS_DIR)
+endif
tools/testing/selftests/riscv/sse/module/riscv_sse_test.c new file +513 -0

513 lines not on this page · raw message

tools/testing/selftests/riscv/sse/run_sse_test.sh new file +44 -0

44 lines not on this page · raw message

Signature · 1 line
2.43.0
maintainer
#14 lore raw
In reply to Clément Léger

Hi Clément,

Quoted 44 lines · Clément Léger

The SBI Supervisor Software Events (SSE) extensions provides a mechanism to inject software events from an SBI implementation to supervisor software such that it preempts all other supervisor level traps and interrupts. This extension is introduced by the SBI v3.0 specification[1].

Various events are defined and can be send asynchronously to supervisor software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well as platform specific events. Events can be either local (per-hart) or global. Events can be nested on top of each other based on priority and can interrupt the kernel at any time.

First patch adds the SSE definitions. Second one adds support for SSE at arch level (entry code and stack allocations) and third one at driver level. Finally, the last patch add support for SSE events in the SBI PMU driver. Additional testing for that part is highly welcomed since there are a lot of possible path that needs to be exercised.

Amongst the specific points that needs to be handle is the interruption at any point of the kernel execution and more specifically at the beginning of exception handling. Due to the fact that the exception entry implementation uses the SCRATCH CSR as both the current task struct and as the temporary register to switch the stack and save register, it is difficult to reliably get the current task struct if we get interrupted at this specific moment (ie, it might contain 0, the task pointer or tp). A fixup-like mechanism is not possible due to the nested nature of SSE which makes it really hard to obtain the original interruption site. In order to retrieve the task in a reliable manner, add an additional __sse_entry_task per_cpu array which stores the current task. Ideally, we would need to modify the way we retrieve/store the current task in exception handling so that it does not depend on the place where it's interrupted.

Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are handled and does not adds any overhead to existing code. Moreover, it provides "true" NMI-like interrupts which can interrupt the kernel at any time (even in exception handling). This is particularly crucial for RAS errors which needs to be handled as fast as possible to avoid any fault propagation.

A test suite is available as a separate kselftest module. In order to build it, you can use the following command:

$ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests
Quoted 8 lines

Then load the module using:

$ sh run_sse_test.sh

A KVM SBI SSE extension implementation is available at [2].

I updated these to apply on Linus' current master, commit 5472d60c129f, cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() suggestion, and pushed them up here to make it convenient for folks to integrate and test:

https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18

Am assuming you didn't have other changes that you wanted to make; let me know if that's not the case.

I noticed that you asked for folks to do additional testing, particularly of the SBI PMU driver integration, but didn't notice any additional Tested-by:s. It would be great if other folks on the list could do some focused testing now, particularly since we're on v7 of this series, and I'm sure others care about this.

thanks,

- Paul

maintainer
#15 lore raw
Quoted 6 lines · Paul Walmsley

I updated these to apply on Linus' current master, commit 5472d60c129f, cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() suggestion, and pushed them up here to make it convenient for folks to integrate and test:

https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18

Sorry, wrong link - here's the correct one:

https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/log/?h=experimental/sse-v7

- Paul

#16 lore raw
In reply to Paul Walmsley

Hi Paul,

Thanks for caring about this series ;)

Quoted 62 lines · Paul Walmsley

Hi Clément,

Quoted 44 lines · Clément Léger

The SBI Supervisor Software Events (SSE) extensions provides a mechanism to inject software events from an SBI implementation to supervisor software such that it preempts all other supervisor level traps and interrupts. This extension is introduced by the SBI v3.0 specification[1].

Various events are defined and can be send asynchronously to supervisor software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well as platform specific events. Events can be either local (per-hart) or global. Events can be nested on top of each other based on priority and can interrupt the kernel at any time.

First patch adds the SSE definitions. Second one adds support for SSE at arch level (entry code and stack allocations) and third one at driver level. Finally, the last patch add support for SSE events in the SBI PMU driver. Additional testing for that part is highly welcomed since there are a lot of possible path that needs to be exercised.

Amongst the specific points that needs to be handle is the interruption at any point of the kernel execution and more specifically at the beginning of exception handling. Due to the fact that the exception entry implementation uses the SCRATCH CSR as both the current task struct and as the temporary register to switch the stack and save register, it is difficult to reliably get the current task struct if we get interrupted at this specific moment (ie, it might contain 0, the task pointer or tp). A fixup-like mechanism is not possible due to the nested nature of SSE which makes it really hard to obtain the original interruption site. In order to retrieve the task in a reliable manner, add an additional __sse_entry_task per_cpu array which stores the current task. Ideally, we would need to modify the way we retrieve/store the current task in exception handling so that it does not depend on the place where it's interrupted.

Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are handled and does not adds any overhead to existing code. Moreover, it provides "true" NMI-like interrupts which can interrupt the kernel at any time (even in exception handling). This is particularly crucial for RAS errors which needs to be handled as fast as possible to avoid any fault propagation.

A test suite is available as a separate kselftest module. In order to build it, you can use the following command:

$ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests
Quoted 8 lines

Then load the module using:

$ sh run_sse_test.sh

A KVM SBI SSE extension implementation is available at [2].

I updated these to apply on Linus' current master, commit 5472d60c129f, cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() suggestion, and pushed them up here to make it convenient for folks to integrate and test:

Quoted 4 lines

https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18

Am assuming you didn't have other changes that you wanted to make; let me know if that's not the case.

Indeed, I do not have pending modification for this series. The KVM SBI SSE support will be submitted later that year.

Quoted 6 lines

I noticed that you asked for folks to do additional testing, particularly of the SBI PMU driver integration, but didn't notice any additional Tested-by:s. It would be great if other folks on the list could do some focused testing now, particularly since we're on v7 of this series, and I'm sure others care about this.

I would have prefer a bit more reviews and testing before going through
since SSE can be quite intrusive at execution time (even though the classic IRQ path should not be impacted, I expect mostly feedback/bugs from the SSE handling path itself). We'll see in a few days if people find such problems.

Thanks,

Clément

Quoted 6 lines

thanks,

- Paul

#17 lore raw
Quoted 68 lines · Clément Léger

Hi Paul,

Thanks for caring about this series ;)

Quoted 62 lines · Paul Walmsley

Hi Clément,

Quoted 44 lines · Clément Léger

The SBI Supervisor Software Events (SSE) extensions provides a mechanism to inject software events from an SBI implementation to supervisor software such that it preempts all other supervisor level traps and interrupts. This extension is introduced by the SBI v3.0 specification[1].

Various events are defined and can be send asynchronously to supervisor software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well as platform specific events. Events can be either local (per-hart) or global. Events can be nested on top of each other based on priority and can interrupt the kernel at any time.

First patch adds the SSE definitions. Second one adds support for SSE at arch level (entry code and stack allocations) and third one at driver level. Finally, the last patch add support for SSE events in the SBI PMU driver. Additional testing for that part is highly welcomed since there are a lot of possible path that needs to be exercised.

Amongst the specific points that needs to be handle is the interruption at any point of the kernel execution and more specifically at the beginning of exception handling. Due to the fact that the exception entry implementation uses the SCRATCH CSR as both the current task struct and as the temporary register to switch the stack and save register, it is difficult to reliably get the current task struct if we get interrupted at this specific moment (ie, it might contain 0, the task pointer or tp). A fixup-like mechanism is not possible due to the nested nature of SSE which makes it really hard to obtain the original interruption site. In order to retrieve the task in a reliable manner, add an additional __sse_entry_task per_cpu array which stores the current task. Ideally, we would need to modify the way we retrieve/store the current task in exception handling so that it does not depend on the place where it's interrupted.

Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are handled and does not adds any overhead to existing code. Moreover, it provides "true" NMI-like interrupts which can interrupt the kernel at any time (even in exception handling). This is particularly crucial for RAS errors which needs to be handled as fast as possible to avoid any fault propagation.

A test suite is available as a separate kselftest module. In order to build it, you can use the following command:

$ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests
Quoted 8 lines

Then load the module using:

$ sh run_sse_test.sh

A KVM SBI SSE extension implementation is available at [2].

I updated these to apply on Linus' current master, commit 5472d60c129f, cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() suggestion, and pushed them up here to make it convenient for folks to integrate and test:

Quoted 7 lines
Quoted 4 lines

https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18

Am assuming you didn't have other changes that you wanted to make; let me know if that's not the case.

Indeed, I do not have pending modification for this series. The KVM SBI SSE support will be submitted later that year.

I am testing this series with my RAS patch set. Will update on this.

Thanks
Regards
Himanshu

Quoted 26 lines
Quoted 7 lines

I noticed that you asked for folks to do additional testing, particularly of the SBI PMU driver integration, but didn't notice any additional Tested-by:s. It would be great if other folks on the list could do some focused testing now, particularly since we're on v7 of this series, and I'm sure others care about this.

I would have prefer a bit more reviews and testing before going through
since SSE can be quite intrusive at execution time (even though the classic IRQ path should not be impacted, I expect mostly feedback/bugs from the SSE handling path itself). We'll see in a few days if people find such problems.

Thanks,

Clément

Quoted 6 lines

thanks,

- Paul

#18 lore raw
In reply to Clément Léger

On Mon, Oct 13, 2025 at 08:53:01AM +0200, Clément Léger wrote:
...

Quoted 5 lines

I would have prefer a bit more reviews and testing before going through
since SSE can be quite intrusive at execution time (even though the classic IRQ path should not be impacted, I expect mostly feedback/bugs from the SSE handling path itself). We'll see in a few days if people find such problems.

I've been planning to compare this series with the code we merged into kvm-unit-tests since the kvm-unit-tests series is where I put the effort into reviewing wrt the spec, etc. On a skim of this series I see several similarities with the kvm-unit-tests code so I should try to actually do that comparison exercise, but I'm not sure when I'll get the time...

Thanks,
drew

#19 lore raw
In reply to Clément Léger

Hi Clément,

Quoted 170 lines · Clément Léger

The SBI Supervisor Software Events (SSE) extensions provides a mechanism to inject software events from an SBI implementation to supervisor software such that it preempts all other supervisor level traps and interrupts. This extension is introduced by the SBI v3.0 specification[1].

Various events are defined and can be send asynchronously to supervisor software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well as platform specific events. Events can be either local (per-hart) or global. Events can be nested on top of each other based on priority and can interrupt the kernel at any time.

First patch adds the SSE definitions. Second one adds support for SSE at arch level (entry code and stack allocations) and third one at driver level. Finally, the last patch add support for SSE events in the SBI PMU driver. Additional testing for that part is highly welcomed since there are a lot of possible path that needs to be exercised.

Amongst the specific points that needs to be handle is the interruption at any point of the kernel execution and more specifically at the beginning of exception handling. Due to the fact that the exception entry implementation uses the SCRATCH CSR as both the current task struct and as the temporary register to switch the stack and save register, it is difficult to reliably get the current task struct if we get interrupted at this specific moment (ie, it might contain 0, the task pointer or tp). A fixup-like mechanism is not possible due to the nested nature of SSE which makes it really hard to obtain the original interruption site. In order to retrieve the task in a reliable manner, add an additional __sse_entry_task per_cpu array which stores the current task. Ideally, we would need to modify the way we retrieve/store the current task in exception handling so that it does not depend on the place where it's interrupted.

Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are handled and does not adds any overhead to existing code. Moreover, it provides "true" NMI-like interrupts which can interrupt the kernel at any time (even in exception handling). This is particularly crucial for RAS errors which needs to be handled as fast as possible to avoid any fault propagation.

A test suite is available as a separate kselftest module. In order to build it, you can use the following command:

$ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests

Then load the module using:

$ sh run_sse_test.sh

A KVM SBI SSE extension implementation is available at [2].

---

Changes in v7:
- Check return values of sse_on_each_cpu()
- Fix typos in commit
- Rename SBI_SSE_EVENT_SIGNAL to SBI_SSE_EVENT_INJECT
- Rename SBI_SSE_EVENT_HART_UNMASK/MASK to SBI_SSE_HART_UNMASK/MASK
- Add tlb flush for vmap stack to avoid taking exception during sse handler upon stack access. (Alex)
- Move some assembly instruction to slow path
- Renamed sse.c to sbi_sse.c, ditto for other files
- Renamed RISCV_SSE to RISCV_SBI_SSE
- Renamed sse_event_handler to sse_event_handler_fn
- Put ifdef around sse_evt in PMU SBI driver

Changes in v6:
- Fix comment in assembly argument
- Check hart id to be the expected one in order to skip CPU id matching in sse assembly.

Changes in v5:
- Added a SSE test module in kselftests
- Removed an unused variable
- Applied checkpatch.pl --strict and fix all errors
- Use scope_guard(cpus_read_lock) instead of manual cpus_read_lock()
- Fix wrong variable returned in sse_get_event
- Remove useless init of events list
- Remove useless empty for loop on cpus
- Set sse_available as __ro_after_init
- Changed a few pr_debug to pr_warn
- Fix event enabled stated updated in case of failure
- Change no_lock to nolock
- Rename attr_buf to attr
- renamed sse_get_event_phys() to sse_event_get_attr_phys() and removed the second argument
- Simplify return value in sse_event_attr_set_nolock()
- Remove while loop(-EINVAL) for event cpu set call
- Renamed interrupted_state_phys to interrupted_phys
- Use scoped_guards/guard for sse_mutex
- Remove useless struct forward declaration in sse.h
- Add more explanations as to why we set SIE bit in IP
- Unconditionnally set SIE in SIP
- Move SSE_STACK_SIZE adjustement in sse_stack_alloc/free()
- Replace move instructions with mv
- Rename NR_CPUS asm symbol to ASM_NR_CPUS
- Restore SSTATUS first in sse_entry return path so that it works for double trap without any modification later.
- Implement proper per cpu revert if enable/register fails

Changes in v4:
- Fix a bug when using per_cpu ptr for local event (Andrew)
- Add sse_event_disable/enable_local()
- Add pmu_disable/pmu_enable() to disable/enable SSE event
- Update event ID description according to the latest spec
- Fix comment about arguments in handle_sse()
- Added Himanchu as a SSE reviewer
- Used SYM_DATA_*() macros instead of hardcoded labels
- Invoke softirqs only if not returning to kernel with irqs disabled
- Remove invalid state check for write attribute function.
- Remove useless bneq statement in sse_entry.S

Changes in v3:
- Split arch/driver support
- Fix potential register failure reporting
- Set a few pr_err as pr_debug
- Allow CONFIG_RISCV_SSE to be disabled
- Fix build without CONFIG_RISCV_SSE
- Remove fixup-like mechanism and use a per-cpu array
- Fixed SSCRATCH being corrupted when interrupting the kernel in early exception path.
- Split SSE assembly from entry.S
- Add Himanchu SSE mask/unmask and runtime PM support.
- Disable user memory access/floating point/vector in SSE handler
- Rebased on master

v2: https://lore.kernel.org/linux-riscv/[email protected]/

Changes in v2:
- Implemented specification v2
- Fix various error handling cases
- Added shadow stack support

v1: https://lore.kernel.org/linux-riscv/[email protected]/

Clément Léger (5):
riscv: add SBI SSE extension definitions
riscv: add support for SBI Supervisor Software Events extension
drivers: firmware: add riscv SSE support
perf: RISC-V: add support for SSE event selftests/riscv: add SSE test module

 MAINTAINERS                                   |  15 +
 arch/riscv/include/asm/asm.h                  |  14 +-
 arch/riscv/include/asm/sbi.h                  |  61 ++
 arch/riscv/include/asm/scs.h                  |   7 +
 arch/riscv/include/asm/sse.h                  |  47 ++
 arch/riscv/include/asm/switch_to.h            |  14 +
 arch/riscv/include/asm/thread_info.h          |   1 +
 arch/riscv/kernel/Makefile                    |   1 +
 arch/riscv/kernel/asm-offsets.c               |  14 +
 arch/riscv/kernel/sbi_sse.c                   | 174 +++++
 arch/riscv/kernel/sbi_sse_entry.S             | 178 +++++
 drivers/firmware/Kconfig                      |   1 +
 drivers/firmware/Makefile                     |   1 +
 drivers/firmware/riscv/Kconfig                |  15 +
 drivers/firmware/riscv/Makefile               |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c        | 701 ++++++++++++++++++
 drivers/perf/Kconfig                          |  10 +
 drivers/perf/riscv_pmu.c                      |  23 +
 drivers/perf/riscv_pmu_sbi.c                  |  71 +-
 include/linux/perf/riscv_pmu.h                |   5 +
 include/linux/riscv_sbi_sse.h                 |  57 ++
 tools/testing/selftests/riscv/Makefile        |   2 +-
 tools/testing/selftests/riscv/sse/Makefile    |   5 +
 .../selftests/riscv/sse/module/Makefile       |  16 +
 .../riscv/sse/module/riscv_sse_test.c         | 513 +++++++++++++
Quoted 8 lines
 .../selftests/riscv/sse/run_sse_test.sh       |  44 ++
 26 files changed, 1979 insertions(+), 14 deletions(-)
 create mode 100644 arch/riscv/include/asm/sse.h
 create mode 100644 arch/riscv/kernel/sbi_sse.c
 create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c

These newly added files use different GPL-xxx licenses respectively:
arch/riscv/kernel/sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-or-later drivers/firmware/riscv/riscv_sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-only arch/riscv/include/asm/sse.h: /* SPDX-License-Identifier: GPL-2.0-only */ Should they be consistent, or is this by design?

Quoted 9 lines
 create mode 100644 include/linux/riscv_sbi_sse.h
 create mode 100644 tools/testing/selftests/riscv/sse/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
 create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh

--
2.43.0

Thanks,
Yunhui

#20 lore raw
Quoted 182 lines · yunhui cui

Hi Clément,

Quoted 170 lines · Clément Léger

The SBI Supervisor Software Events (SSE) extensions provides a mechanism to inject software events from an SBI implementation to supervisor software such that it preempts all other supervisor level traps and interrupts. This extension is introduced by the SBI v3.0 specification[1].

Various events are defined and can be send asynchronously to supervisor software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well as platform specific events. Events can be either local (per-hart) or global. Events can be nested on top of each other based on priority and can interrupt the kernel at any time.

First patch adds the SSE definitions. Second one adds support for SSE at arch level (entry code and stack allocations) and third one at driver level. Finally, the last patch add support for SSE events in the SBI PMU driver. Additional testing for that part is highly welcomed since there are a lot of possible path that needs to be exercised.

Amongst the specific points that needs to be handle is the interruption at any point of the kernel execution and more specifically at the beginning of exception handling. Due to the fact that the exception entry implementation uses the SCRATCH CSR as both the current task struct and as the temporary register to switch the stack and save register, it is difficult to reliably get the current task struct if we get interrupted at this specific moment (ie, it might contain 0, the task pointer or tp). A fixup-like mechanism is not possible due to the nested nature of SSE which makes it really hard to obtain the original interruption site. In order to retrieve the task in a reliable manner, add an additional __sse_entry_task per_cpu array which stores the current task. Ideally, we would need to modify the way we retrieve/store the current task in exception handling so that it does not depend on the place where it's interrupted.

Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are handled and does not adds any overhead to existing code. Moreover, it provides "true" NMI-like interrupts which can interrupt the kernel at any time (even in exception handling). This is particularly crucial for RAS errors which needs to be handled as fast as possible to avoid any fault propagation.

A test suite is available as a separate kselftest module. In order to build it, you can use the following command:

$ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests

Then load the module using:

$ sh run_sse_test.sh

A KVM SBI SSE extension implementation is available at [2].

---

Changes in v7:
- Check return values of sse_on_each_cpu()
- Fix typos in commit
- Rename SBI_SSE_EVENT_SIGNAL to SBI_SSE_EVENT_INJECT
- Rename SBI_SSE_EVENT_HART_UNMASK/MASK to SBI_SSE_HART_UNMASK/MASK
- Add tlb flush for vmap stack to avoid taking exception during sse handler upon stack access. (Alex)
- Move some assembly instruction to slow path
- Renamed sse.c to sbi_sse.c, ditto for other files
- Renamed RISCV_SSE to RISCV_SBI_SSE
- Renamed sse_event_handler to sse_event_handler_fn
- Put ifdef around sse_evt in PMU SBI driver

Changes in v6:
- Fix comment in assembly argument
- Check hart id to be the expected one in order to skip CPU id matching in sse assembly.

Changes in v5:
- Added a SSE test module in kselftests
- Removed an unused variable
- Applied checkpatch.pl --strict and fix all errors
- Use scope_guard(cpus_read_lock) instead of manual cpus_read_lock()
- Fix wrong variable returned in sse_get_event
- Remove useless init of events list
- Remove useless empty for loop on cpus
- Set sse_available as __ro_after_init
- Changed a few pr_debug to pr_warn
- Fix event enabled stated updated in case of failure
- Change no_lock to nolock
- Rename attr_buf to attr
- renamed sse_get_event_phys() to sse_event_get_attr_phys() and removed the second argument
- Simplify return value in sse_event_attr_set_nolock()
- Remove while loop(-EINVAL) for event cpu set call
- Renamed interrupted_state_phys to interrupted_phys
- Use scoped_guards/guard for sse_mutex
- Remove useless struct forward declaration in sse.h
- Add more explanations as to why we set SIE bit in IP
- Unconditionnally set SIE in SIP
- Move SSE_STACK_SIZE adjustement in sse_stack_alloc/free()
- Replace move instructions with mv
- Rename NR_CPUS asm symbol to ASM_NR_CPUS
- Restore SSTATUS first in sse_entry return path so that it works for double trap without any modification later.
- Implement proper per cpu revert if enable/register fails

Changes in v4:
- Fix a bug when using per_cpu ptr for local event (Andrew)
- Add sse_event_disable/enable_local()
- Add pmu_disable/pmu_enable() to disable/enable SSE event
- Update event ID description according to the latest spec
- Fix comment about arguments in handle_sse()
- Added Himanchu as a SSE reviewer
- Used SYM_DATA_*() macros instead of hardcoded labels
- Invoke softirqs only if not returning to kernel with irqs disabled
- Remove invalid state check for write attribute function.
- Remove useless bneq statement in sse_entry.S

Changes in v3:
- Split arch/driver support
- Fix potential register failure reporting
- Set a few pr_err as pr_debug
- Allow CONFIG_RISCV_SSE to be disabled
- Fix build without CONFIG_RISCV_SSE
- Remove fixup-like mechanism and use a per-cpu array
- Fixed SSCRATCH being corrupted when interrupting the kernel in early exception path.
- Split SSE assembly from entry.S
- Add Himanchu SSE mask/unmask and runtime PM support.
- Disable user memory access/floating point/vector in SSE handler
- Rebased on master

v2: https://lore.kernel.org/linux-riscv/[email protected]/

Changes in v2:
- Implemented specification v2
- Fix various error handling cases
- Added shadow stack support

v1: https://lore.kernel.org/linux-riscv/[email protected]/

Clément Léger (5):
riscv: add SBI SSE extension definitions
riscv: add support for SBI Supervisor Software Events extension
drivers: firmware: add riscv SSE support
perf: RISC-V: add support for SSE event selftests/riscv: add SSE test module

 MAINTAINERS                                   |  15 +
 arch/riscv/include/asm/asm.h                  |  14 +-
 arch/riscv/include/asm/sbi.h                  |  61 ++
 arch/riscv/include/asm/scs.h                  |   7 +
 arch/riscv/include/asm/sse.h                  |  47 ++
 arch/riscv/include/asm/switch_to.h            |  14 +
 arch/riscv/include/asm/thread_info.h          |   1 +
 arch/riscv/kernel/Makefile                    |   1 +
 arch/riscv/kernel/asm-offsets.c               |  14 +
 arch/riscv/kernel/sbi_sse.c                   | 174 +++++
 arch/riscv/kernel/sbi_sse_entry.S             | 178 +++++
 drivers/firmware/Kconfig                      |   1 +
 drivers/firmware/Makefile                     |   1 +
 drivers/firmware/riscv/Kconfig                |  15 +
 drivers/firmware/riscv/Makefile               |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c        | 701 ++++++++++++++++++
 drivers/perf/Kconfig                          |  10 +
 drivers/perf/riscv_pmu.c                      |  23 +
 drivers/perf/riscv_pmu_sbi.c                  |  71 +-
 include/linux/perf/riscv_pmu.h                |   5 +
 include/linux/riscv_sbi_sse.h                 |  57 ++
 tools/testing/selftests/riscv/Makefile        |   2 +-
 tools/testing/selftests/riscv/sse/Makefile    |   5 +
 .../selftests/riscv/sse/module/Makefile       |  16 +
 .../riscv/sse/module/riscv_sse_test.c         | 513 +++++++++++++
Quoted 8 lines
 .../selftests/riscv/sse/run_sse_test.sh       |  44 ++
 26 files changed, 1979 insertions(+), 14 deletions(-)
 create mode 100644 arch/riscv/include/asm/sse.h
 create mode 100644 arch/riscv/kernel/sbi_sse.c
 create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
Quoted 5 lines

These newly added files use different GPL-xxx licenses respectively:
arch/riscv/kernel/sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-or-later drivers/firmware/riscv/riscv_sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-only arch/riscv/include/asm/sse.h: /* SPDX-License-Identifier: GPL-2.0-only */ Should they be consistent, or is this by design?

Hi Yunhui,

That is not intended, I'll fix it.

Thanks,

Clément

Quoted 14 lines
Quoted 11 lines

 create mode 100644 include/linux/riscv_sbi_sse.h
 create mode 100644 tools/testing/selftests/riscv/sse/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
 create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh

--
2.43.0

Thanks,
Yunhui