Skip to contentSkip to conversation

riscv: introduce basic UART support and interrupts for hypervisor mode

PATCH Superseded Oleksii Kurochko 2 participants 18 messages Jun 5, 2025 – Jun 11, 2025 RISC-V

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#5 lore raw

PATCH v4 2/9 xen/riscv: introduce register_intc_ops() and intc_hw_ops.

In reply to Oleksii Kurochko

Introduce the intc_hw_operations structure to encapsulate interrupt controller-specific data and operations. This structure includes:
- A pointer to interrupt controller information (intc_info)
- Callbacks to initialize the controller and set IRQ type/priority
- A reference to an interupt controller descriptor (host_irq_type)
- number of interrupt controller irqs.

Add function register_intc_ops() to mentioned above structure.

---
Changes in V4:
- Add Acked-by: Jan Beulich <[email protected]>.
---
Changes in V3:
- Drop inclusion of xen/irq.h in asm/intc.h as forward declaration is enogh for types used in asm/intc.h.
- Drop forward declaration for dt_device_node and hw_irq_controller.
- Declare intc_hw_ops as:

     const struct intc_hw_operations * __ro_after_init intc_hw_ops;

---
Changes in V2:
- Declare host_irq_type member of intc_hw_operations as pointer-to-const.
- Moved up forward declaration of irq_desc.
- Use attribute __init for register_intc_ops().
- Use attribute __ro_after_init for intc_hw_ops variable.
- Update prototype of register_intc_ops() because of what mention in the previous point.

---
 xen/arch/riscv/include/asm/intc.h | 19 +++++++++++++++++++
 xen/arch/riscv/intc.c             |  9 +++++++++
 2 files changed, 28 insertions(+)
xen/arch/riscv/include/asm/intc.h +19 -0
index 81f74736ba..434c9d0781 100644
@@ -14,11 +14,30 @@ enum intc_version {
     INTC_APLIC,
 };
 
+struct irq_desc;
+
 struct intc_info {
     enum intc_version hw_version;
     const struct dt_device_node *node;
 };
 
+struct intc_hw_operations {
+    /* Hold intc hw information */
+    const struct intc_info *info;
+    /* Initialize the intc and the boot CPU */
+    int (*init)(void);
+
+    /* hw_irq_controller to enable/disable/eoi host irq */
+    const struct hw_interrupt_type *host_irq_type;
+
+    /* Set IRQ type */
+    void (*set_irq_type)(struct irq_desc *desc, unsigned int type);
+    /* Set IRQ priority */
+    void (*set_irq_priority)(struct irq_desc *desc, unsigned int priority);
+};
+
 void intc_preinit(void);
 
+void register_intc_ops(const struct intc_hw_operations *ops);
+
 #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
xen/arch/riscv/intc.c +9 -0
index 4061a3c457..1ecd651bf3 100644
@@ -5,6 +5,15 @@
 #include <xen/init.h>
 #include <xen/lib.h>
 
+#include <asm/intc.h>
+
+static const struct intc_hw_operations *__ro_after_init intc_hw_ops;
+
+void __init register_intc_ops(const struct intc_hw_operations *ops)
+{
+    intc_hw_ops = ops;
+}
+
 void __init intc_preinit(void)
 {
     if ( acpi_disabled )
Signature · 1 line
2.49.0
#6 lore raw

PATCH v4 3/9 xen/riscv: imsic_init() implementation

In reply to Oleksii Kurochko

imsic_init() is introduced to parse device tree node, which has the following bindings [2], and based on the parsed information update IMSIC configuration which is stored in imsic_cfg.

The following helpers are introduces for imsic_init() usage:
- imsic_parse_node() parses IMSIC node from DTS
- imsic_get_parent_hartid() returns the hart ( CPU ) ID of the given device tree node.

This patch is based on the code from [1].

Since Microchip originally developed imsic.{c,h}, an internal discussion with them led to the decision to use the MIT license.

[1] https://gitlab.com/xen-project/people/olkur/xen/-/commit/0b1a94f2bc3bb1a81cd26bb75f0bf578f84cb4d4
[2] https://elixir.bootlin.com/linux/v6.12/source/Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml

---
Changes in V4:
- s/expected/intended in the comment above imsic_get_config().
- [imsic_parse_node()] s/irq_num can be 0/irq_num can't be 0 in panic()
message.
- [imsic_parse_node()] Move "if ( irq_range[1] == IRQ_M_EXT )" after 'for loop' which checks interrupts-extended property.
- [imsic_parse_node()] s/%d/%u for logging unsigned values.
- [imsic_parse_node()] drop redundant check "(imsic_cfg.nr_ids & IMSIC_MIN_ID) != IMSIC_MIN_ID)".
- [imsic_parse_node()] free irq_range much earlier and simlify error paths.
- [imsic_parse_node()] s/-EINVAL/-ENOENT in the case of incorrect value for riscv,group-index-shift and id number.
- [[imsic_parse_node()] s/xzalloc_array/xvzalloc_array.
- s/xen_cpuid/cpu.
- Identation fix.
- use IMSIC_MMIO_PAGE_SZ instead of PAGE_SZ to check if interrupt file base
addr is properly aligned.
- s/ASM__RISCV__IMSIC_H/ASM_RISCV_IMSIC_H.
- Drop *mmios from struct imsic_cfg as it is used only by imsic_init().
- Drop cpus[] form struct imsic_mmios as it isn't really used.
- Update declaration of hartid_to_cpuid() to return unsigned int instead of NR_CPUs as processor_id is in range [0, NR_CPUS) and NR_CPUs is less then unsigned int.
- Calculate hart_index_bits as fls(*nr_parent_irqs - 1) to cover the case if nr_parent_irqs is a power of two.
- Check an MMIO's size for IMSIC node.
---
Changes in V3:
- Drop year in imsic.h in copyrights.
- Correct identation in imsic_parse_node() and imsic_init()
where for imsic_cfg.base_addr a mask is applied.
- Use unsigned int istead of uint32_t for local variable nr_parent_irqs,
index, nr_handlers in imsic_init().
- Fix a leakage of ealiers successfull allocations in case if imsic_init() returns with an error.
- Excess blank in printk() message: "%s: unable to parse MMIO regset %d\n".
- Introduce hartid_to_cpuid() and use it in the check:
if ( hardid_to_cpuid(cpuid) >= num_possible_cpus() )
in imsic_init().
- Use "%u" for unsigned int in printk(...).
- Fix for loop condition: nr_mmios -> "j < nr_mmios".
- [imsic_init()] Drop usage of j in nested loop. It is enough to have only
index.
- Change 0x%lx to %#lx for formatting of an address in printk().
- [imsic_init()] Rename local variable cpuid to hartid.
- s/imsic_get_parent_cpuid/imsic_get_parent_hartid, s/cpuid/hartid for an imsic_get_parent_hartid()'s argument.
- Declare cpus member of struct imsic_mmios as cpumask_t.
- [imsic_init()] Allocate imsic_mmios.cpus by using of alloc_cpumask_var().
- [imsic_init()] Use cpumask_set_cpu() instead of bitmap_set().
- [imsic_parse_node()] add check for correctness of "interrupt-extended" property.
- [imsic_parse_node()] Use dt_node_name() or dt_full_node_name() instead of derefence of struct dt_node.
- [imsic_parse_node()] Add cleanup label and update 'rc = XXX; goto cleanup' instead of 'return rc' as now we have to cleanup dynamically allocated irq_range array.
- Add comments above imsic_init() and imsic_parse_node().
- s/xen/arch/riscv/imsic.h/xen/arch/riscv/include/asm/imsic.h in the comment of imsic.h.
---
Changes in V2:
- Drop years in copyrights.
- s/riscv_of_processor_hartid/dt_processor_cpuid.
- s/imsic_get_parent_hartid/imsic_get_parent_hartid.
Rename argument hartid to cpuid.
Make node argument const.
Return res instead of -EINVAL for the failure case of dt_processor_cpuid(). Drop local variable hart and use cpuid argument instead.
Drop useless return res;
- imsic_parse_node() changes:
- Make node argument const.
- Check the return value of dt_property_read_u32() directly instead of
saving it to rc variable.
- Update tmp usage, use short form "-=".
- Update a check (imsic_cfg.nr_ids >= IMSIC_MAX_ID) to (imsic_cfg.nr_ids > IMSIC_MAX_ID) as IMSIC_MAX_ID is changed to maximum valid value, not just the firsr out-of-range.
- Use rc to return value instead of explicitly use -EINVAL.
- Use do {} while() to find number of MMIO register sets.
- Set IMSIC_MAX_ID to 2047 (maximum possible IRQ number).
- imsic_init() changes:
- Use unsigned int in for's expression1.
- s/xfree/XFEE.
- Allocate msi and cpus array dynamically.
- Drop forward declaration before declaration of imsic_get_config() in asm/imsic.h as it is not used as parameter type.
- Align declaration of imisic_init with defintion.
- s/harts/cpus in imisic_mmios.
Also, change type from bool harts[NR_CPUS] to unsigned long *cpus.
- Allocate msi member of imsic_config dynamically to save some memory.
- Code style fixes.
- Update the commit message.

---
 xen/arch/riscv/Makefile            |   1 +
 xen/arch/riscv/imsic.c             | 358 +++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/imsic.h |  55 +++++
 xen/arch/riscv/include/asm/smp.h   |  13 ++
 4 files changed, 427 insertions(+)
 create mode 100644 xen/arch/riscv/imsic.c
 create mode 100644 xen/arch/riscv/include/asm/imsic.h
xen/arch/riscv/Makefile +1 -0
index a1c145c506..e2b8aa42c8 100644
@@ -2,6 +2,7 @@ obj-y += aplic.o
 obj-y += cpufeature.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-y += entry.o
+obj-y += imsic.o
 obj-y += intc.o
 obj-y += irq.o
 obj-y += mm.o
xen/arch/riscv/imsic.c new file +358 -0

358 lines not on this page · raw message

xen/arch/riscv/include/asm/imsic.h new file +55 -0

55 lines not on this page · raw message

xen/arch/riscv/include/asm/smp.h +13 -0
index eb58b6576b..57c8baa06c 100644
@@ -3,6 +3,7 @@
 #define ASM__RISCV__SMP_H
 
 #include <xen/cpumask.h>
+#include <xen/macros.h>
 #include <xen/percpu.h>
 
 #include <asm/current.h>
@@ -18,6 +19,18 @@ static inline unsigned long cpuid_to_hartid(unsigned long cpuid)
     return pcpu_info[cpuid].hart_id;
 }
 
+static inline unsigned int hartid_to_cpuid(unsigned long hartid)
+{
+    for ( unsigned int cpuid = 0; cpuid < ARRAY_SIZE(pcpu_info); cpuid++ )
+    {
+        if ( hartid == cpuid_to_hartid(cpuid) )
+            return cpuid;
+    }
+
+    /* hartid isn't valid for some reason */
+    return NR_CPUS;
+}
+
 static inline void set_cpuid_to_hartid(unsigned long cpuid,
                                        unsigned long hartid)
 {
Signature · 1 line
2.49.0
#9 lore raw

PATCH v4 4/9 xen/riscv: aplic_init() implementation

In reply to Oleksii Kurochko

aplic_init() function does the following few things:
- checks that IMSIC in device tree node ( by checking msi-parent property in APLIC node ) is present as current one implmenetaion of AIA is supported only MSI method.
- initialize IMSIC based on IMSIC device tree node
- Read value of APLIC's paddr start/end and size.
- Map aplic.regs
- Setup APLIC initial state interrupts (disable all interrupts, set interrupt type and default priority, confgifure APLIC domaincfg) by calling aplic_init_hw_interrutps().

aplic_init() is based on the code from [1] and [2].

Since Microchip originally developed aplic.c, an internal discussion with them led to the decision to use the MIT license.

[1] https://gitlab.com/xen-project/people/olkur/xen/-/commit/7cfb4bd4748ca268142497ac5c327d2766fb342d
[2] https://gitlab.com/xen-project/people/olkur/xen/-/commit/392a531bfad39bf4656ce8128e004b241b8b3f3e

---
Changes in V4:
- s/ASM__RISCV_PRIV_APLIC_H/ASM_RISCV_PRIV_APLIC_H
- Drop APLIC_MAX_NUM_WIRED_IRQ_SOURCES and use ARRAY_SIZE(aplic.regs->sourcecfg).
- Don't use 'else if' for the case when the earlier if() ends in an unconditional control flow change.
- Use const for aplic_ops and drop __ro_after_init.
- Add checks of APLIC's memory-mapped control region physical address and size.
- s/ASM__RISCV__APLIC_H/ASM_RISCV_APLIC_H.
- Use unsigned int in defintion of APLIC_DOMAINCFG_IE and APLIC_DOMAINCFG_DM.
- set aplic_info.num_irqs to ARRAY_SIZE(aplic.regs->sourcecfg) if DT node provides too much.
- Add some constraints for aplic.paddr_start and aplic.size.
---
Changes in V3:
- Correct the comment on top of aplic-priv.h:

     xen/arch/riscv/aplic.h -> .../aplic-priv.h

- Add __iomem for regs member of aplic_priv structure.
- [aplic_init_hw_interrupts] Use ~0U instead of -1U in aplic_init_hw_interrupts() to disable all interrupts.
- [aplic_init_hw_interrupts] Start 'i' (for-cycle variable) from 0, not from 1.
- [aplic_init()] Declare imsic_node as pointer-to-const.
- Use decimal for arrays in struct aplic_regs.
- [aplic_init()] Check that aplic_info.num_irqs are less then 1023.
- [aplic_init()] Drop out check of IMSIC's node interrupt-extended property from aplic_init().
---
Changes in V2:
- use __ro_after_init for aplic_ops.
- s/nr_irqs/num_irqs.
- s/dt_processor_hartid/dt_processor_cpuid.
- Drop confusing comment in aplic_init_hw_interrupts().
- Code style fixes.
- Drop years for Copyright.
- Revert changes which drop nr_irq define from asm/irq.h,
it shouldn't be, at least, part of this patch.
- Drop paddr_enf from struct aplic_regs. It is enough to have pair (paddr_start, size).
- Make struct aplic_priv of asm/aplic.h private by moving it to riscv/aplic-priv.h.
- Add the comment above the initialization of APLIC's target register.
- use writel() to access APLIC's registers.
- use 'unsinged int' for local variable i in aplic_init_hw_interrupts.
- Add the check that all modes in interrupts-extended property of imsic node are equal. And drop rc != EOVERFLOW when interrupts-extended property is read.
- Add cf_check to aplic_init().
- Fix a cycle of clrie register initialization in aplic_init_hw_interrupts(). Previous implementation leads to out-of-boundary.
- Declare member num_irqs in struct intc_info as it is used by APLIC code.

---
 xen/arch/riscv/aplic-priv.h        |  34 ++++++++++
 xen/arch/riscv/aplic.c             | 104 +++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/aplic.h |  64 ++++++++++++++++++
 xen/arch/riscv/include/asm/intc.h  |   3 +
 4 files changed, 205 insertions(+)
 create mode 100644 xen/arch/riscv/aplic-priv.h
 create mode 100644 xen/arch/riscv/include/asm/aplic.h
xen/arch/riscv/aplic-priv.h new file +34 -0
new file mode 100644
index 0000000000..7ed1ab8f4b
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: MIT */
+
+/*
+ * xen/arch/riscv/aplic-priv.h
+ *
+ * Private part of aplic.h header.
+ *
+ * RISC-V Advanced Platform-Level Interrupt Controller support
+ *
+ * Copyright (c) Microchip.
+ * Copyright (c) Vates.
+ */
+
+#ifndef ASM_RISCV_PRIV_APLIC_H
+#define ASM_RISCV_PRIV_APLIC_H
+
+#include <xen/types.h>
+
+#include <asm/aplic.h>
+#include <asm/imsic.h>
+
+struct aplic_priv {
+    /* base physical address and size */
+    paddr_t paddr_start;
+    size_t  size;
+
+    /* registers */
+    volatile struct aplic_regs __iomem *regs;
+
+    /* imsic configuration */
+    const struct imsic_config *imsic_cfg;
+};
+
+#endif /* ASM_RISCV_PRIV_APLIC_H */
xen/arch/riscv/aplic.c +104 -0

131 lines not on this page · raw message

xen/arch/riscv/include/asm/aplic.h new file +64 -0

64 lines not on this page · raw message

xen/arch/riscv/include/asm/intc.h +3 -0
index 434c9d0781..3c4b211f58 100644
@@ -19,6 +19,9 @@ struct irq_desc;
 struct intc_info {
     enum intc_version hw_version;
     const struct dt_device_node *node;
+
+    /* number of irqs */
+    unsigned int num_irqs;
 };
 
 struct intc_hw_operations {
Signature · 1 line
2.49.0
#0 lore raw

v4 cover riscv: introduce basic UART support and interrupts for hypervisor mode

The patch series introduces basic UART support (in interrupt mode) and support of interrupts for hypervisor mode.

To implement this the following has been added:
- APLIC and IMISC initialization.
- Introduce of intc_hw_operations abstraction.
- Introduce some APLIC and IMSIC operations.
- Introduce init_IRQ(), platform_get_irq() and setup_irq() functions.
- Update do_trap() handler to handle IRQ_S_EXT.
- Introduce some other functions such as: get_s_time(), smp_clear_cpu_maps(), ioremap().
- Enable UART.

CI tests: https://gitlab.com/xen-project/people/olkur/xen/-/pipelines/1829069921
---
Changes in V4:
- Merged to staging:
- xen/riscv: initialize bitmap to zero in riscv_fill_hwcap_from_isa_string()
- xen/asm-generic: introduce asm-generic/irq-dt.h
- xen/riscv: introduce smp_prepare_boot_cpu()
- xen/riscv: introduce support of Svpbmt extension
- add ioremap_*() variants using ioremap_attr()
- xen/riscv: introduce init_IRQ()
- xen/riscv: introduce platform_get_irq()
- All other changes are patch-specific. Please check each patch separately.
---
Changes in V2:
- Merged to staging:
xen/riscv: initialize bitmap to zero in riscv_fill_hwcap_from_isa_string() xen/asm-generic: introduce asm-generic/irq-dt.h
- All other changes are patch-specific. Please check each patch separately.
---

Oleksii Kurochko (9):
xen/riscv: dt_processor_hartid() implementation xen/riscv: introduce register_intc_ops() and intc_hw_ops. xen/riscv: imsic_init() implementation
xen/riscv: aplic_init() implementation
xen/riscv: introduce intc_init() and helpers xen/riscv: implementation of aplic and imsic operations xen/riscv: add external interrupt handling for hypervisor mode xen/riscv: implement setup_irq()
xen/riscv: add basic UART support

 xen/arch/riscv/Kconfig             |   1 +
 xen/arch/riscv/Makefile            |   1 +
 xen/arch/riscv/aplic-priv.h        |  38 +++
 xen/arch/riscv/aplic.c             | 296 ++++++++++++++++++
 xen/arch/riscv/imsic.c             | 478 +++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/aplic.h |  73 +++++
 xen/arch/riscv/include/asm/imsic.h |  74 +++++
 xen/arch/riscv/include/asm/intc.h  |  32 ++
 xen/arch/riscv/include/asm/irq.h   |   8 +-
 xen/arch/riscv/include/asm/smp.h   |  17 +
 xen/arch/riscv/intc.c              |  55 ++++
 xen/arch/riscv/irq.c               | 131 ++++++++
 xen/arch/riscv/setup.c             |  14 +
 xen/arch/riscv/smpboot.c           |  77 +++++
 xen/arch/riscv/traps.c             |  19 ++
 xen/drivers/char/Kconfig           |   3 +-
 16 files changed, 1314 insertions(+), 3 deletions(-)
 create mode 100644 xen/arch/riscv/aplic-priv.h
 create mode 100644 xen/arch/riscv/imsic.c
 create mode 100644 xen/arch/riscv/include/asm/aplic.h
 create mode 100644 xen/arch/riscv/include/asm/imsic.h
Signature · 1 line
2.49.0
#1 lore raw

PATCH v4 1/9 xen/riscv: dt_processor_hartid() implementation

Implements dt_processor_hartid() to get the hart ID of the given device tree node and do some checks if CPU is available and given device tree node has proper riscv,isa property.

As a helper function dt_get_hartid() is introduced to deal specifically with reg propery of a CPU device node.

---
Changes in V4:
- Sort properly inculsion of xen/types.h in riscv/smpboot.c.
- Add overflow check of got address cell value in dt_get_hartid().
- Use "%#lx" for loging recieved hartid from DTB.
- s/-EINVAL/-ENODEV for recieved isa string from DTB.
---
Changes in V3:
- s/dt_processor_cpuid/dt_processor_hartid.
- s/dt_get_cpuid/dt_get_hartid.
- use ~0UL in dt_get_cpuid() and in the comment above it.
- change type for local variable ac in dt_get_cpuid() to unsigned int.
- Update the return errors for dt_processor_cpuid().
- Update the commit message + subject: s/cpuid/hartid.
---
Changes in V2:
- s/of_get_cpu_hwid()/dt_get_cpu_id().
- Update prototype of dt_get_cpu_hwid(), use pointer-to-const for cpun arg.
- Add empty line before last return in dt_get_cpu_hwid().
- s/riscv_of_processor_hartid/dt_processor_cpuid().
- Use pointer-to_const for node argument of dt_processor_cpuid().
- Use for hart_id unsigned long type as according to the spec for RV128 mhartid register will be 128 bit long.
- Update commit message and subject.
- use 'CPU' instead of 'HART'.
- Drop thread argument of dt_get_cpu_id() (of_get_cpu_hwid) as it is expected to be always 0 according to RISC-V's DTS binding.

---
 xen/arch/riscv/include/asm/smp.h |  4 ++
 xen/arch/riscv/smpboot.c         | 77 ++++++++++++++++++++++++++++++++
 2 files changed, 81 insertions(+)
xen/arch/riscv/include/asm/smp.h +4 -0
index 5e170b57b3..eb58b6576b 100644
@@ -26,6 +26,10 @@ static inline void set_cpuid_to_hartid(unsigned long cpuid,
 
 void setup_tp(unsigned int cpuid);
 
+struct dt_device_node;
+int dt_processor_hartid(const struct dt_device_node *node,
+                        unsigned long *hartid);
+
 #endif
 
 /*
xen/arch/riscv/smpboot.c +77 -0

86 lines not on this page · raw message

Signature · 1 line
2.49.0
#12 lore raw

PATCH v4 6/9 xen/riscv: implementation of aplic and imsic operations

In reply to Oleksii Kurochko

Introduce interrupt controller descriptor for host APLIC to describe the low-lovel hardare. It includes implementation of the following functions:
- aplic_irq_startup()
- aplic_irq_enable()
- aplic_irq_disable()
- aplic_set_irq_affinity()

As APLIC is used in MSI mode it requires to enable/disable interrupts not only for APLIC but also for IMSIC. Thereby for the purpose of aplic_irq_{enable,disable}() it is introduced imsic_irq_{enable,disable)().

For the purpose of aplic_set_irq_affinity() aplic_get_cpu_from_mask() is introduced to get hart id.

Also, introduce additional interrupt controller h/w operations and host_irq_type for APLIC:
- aplic_host_irq_type

Patch is based on the code from [1].

[1] https://gitlab.com/xen-project/people/olkur/xen/-/commit/7390e2365828b83e27ead56b03114a56e3699dd5

---
Changes in V4:
- Code style fixes.
- Add cf_check for hook functions.
- Use min() macros instead of open-coding it in imsic_local_eix_update().
- Update the comment above ASSERT() in imsic_irq_disable():
s/aplic_irq_enable/aplic_irq_disable.
- Add an explanatory comment above initializing of hhxs in aplic_set_irq_affinity().
---
Changes in V3:
- Update the lock above lock member of struct aplic_priv and imsic_config struct.
- Use spin_{un}lock() in aplic_irq_{enable,disable}() as it is IRQ-safe.
Also drop local variable 'flags'.
- Add ASSERT(spin_is_locked(&desc->lock)) to aplic_set_irq_affinity() and aplic_set_irq_type().
- Use an initializer instead of spin_lock_init() for aplic.lock.
- Drop "(l)" in the comment in imsic_irq_enable() as it doesn't point to
anything.
- Use ASSERT(!local_irq_is_enabled()) + spin_lock() in imsic_irq_{enable,disable}().
- Use an initializer instead of spin_lock_init() for imsic_config.lock.
---
Changes in V2:
- Move imsic_ids_local_delivery() and connected to it parts to the current
patch to fix compilation issue. Also, add __init for
imsic_ids_local_delivery().
- Move introduction of aplic_set_irq_type() and aplic_set_irq_priority()
to patch [PATCH v1 12/14] xen/riscv: implement setup_irq() where they
really started to be used.
- Update the commit message.
- Drop is_used variable for imsic_cfg and use (aplic.regs->domaincfg & APLIC_DOMAINCFG_DM) instead.
- Use writel() to write to APLIC regs.
- Drop aplic_irq_shutdown() and use aplic_irq_disable explicitly.
- Drop local variable cpu in aplic_get_cpu_from_mask():
Use cpu_online_map instead of cpu_possible_map.
Remame possible_mask to mask.
- Code style fixes.
- Move spin_lock(&aplic.lock) down before write to the register in aplic_set_irq_affinity.
- Make aplic_host_irq_type const.
- imsic_local_eix_update() updates:
- move unsigned long isel, ireg; to inner loop.
- Drop unnecessary parentheses.
- Optimize inner loop of ireg's setting.
- Drop aplic_irq_ack() and aplic_host_irq_end() as they do nothing.
- Rename s/hwirq/irq.
- Add explanatory comment to imsic_irq_enable() about why there is not -1 for IRQ in comparison with APLIC's sourcecfg.
- Use IMSIC_MMIO_PAGE_SHIFT instead of constant 12 in aplic_set_irq_affinity().
- s/aplic_host_irq_type/aplic_xen_irq_type
- Drop set/clear of IRQ_DISABLED bit in aplic_{enable,disable}() as guest will always first request an interrupt and then only an interrupt will be enabled.
(for example, in Arm, the physical interrupts would be enabled when the
interrupt is initially routed. This could lead to problem because the guest
will usually boot with interrupt disabled.)

---
 xen/arch/riscv/aplic-priv.h        |   4 +
 xen/arch/riscv/aplic.c             | 123 ++++++++++++++++++++++++++++-
 xen/arch/riscv/imsic.c             | 122 +++++++++++++++++++++++++++-
 xen/arch/riscv/include/asm/aplic.h |   2 +
 xen/arch/riscv/include/asm/imsic.h |  18 +++++
 5 files changed, 267 insertions(+), 2 deletions(-)
xen/arch/riscv/aplic-priv.h +4 -0
index 7ed1ab8f4b..d377a7ac67 100644
@@ -14,6 +14,7 @@
 #ifndef ASM_RISCV_PRIV_APLIC_H
 #define ASM_RISCV_PRIV_APLIC_H
 
+#include <xen/spinlock.h>
 #include <xen/types.h>
 
 #include <asm/aplic.h>
@@ -27,6 +28,9 @@ struct aplic_priv {
     /* registers */
     volatile struct aplic_regs __iomem *regs;
 
+    /* lock to protect access to APLIC's registers */
+    spinlock_t lock;
+
     /* imsic configuration */
     const struct imsic_config *imsic_cfg;
 };
xen/arch/riscv/aplic.c +122 -1

148 lines not on this page · raw message

xen/arch/riscv/imsic.c +121 -1

134 lines not on this page · raw message

xen/arch/riscv/include/asm/aplic.h +2 -0
index fea95d4369..c3f00238f2 100644
@@ -18,6 +18,8 @@
 #define APLIC_DOMAINCFG_IE      BIT(8, U)
 #define APLIC_DOMAINCFG_DM      BIT(2, U)
 
+#define APLIC_TARGET_HART_IDX_SHIFT 18
+
 struct aplic_regs {
     uint32_t domaincfg;
     uint32_t sourcecfg[1023];
xen/arch/riscv/include/asm/imsic.h +18 -0
index 9cd12365b1..378e49d933 100644
@@ -11,6 +11,7 @@
 #ifndef ASM_RISCV_IMSIC_H
 #define ASM_RISCV_IMSIC_H
 
+#include <xen/spinlock.h>
 #include <xen/types.h>
 
 #define IMSIC_MMIO_PAGE_SHIFT   12
@@ -19,6 +20,15 @@
 #define IMSIC_MIN_ID            63
 #define IMSIC_MAX_ID            2047
 
+#define IMSIC_EIDELIVERY        0x70
+
+#define IMSIC_EITHRESHOLD       0x72
+
+#define IMSIC_EIP0              0x80
+#define IMSIC_EIPx_BITS         32
+
+#define IMSIC_EIE0              0xC0
+
 struct imsic_msi {
     paddr_t base_addr;
     unsigned long offset;
@@ -45,6 +55,9 @@ struct imsic_config {
 
     /* MSI */
     const struct imsic_msi *msi;
+
+    /* Lock to protect access to IMSIC's stuff */
+    spinlock_t lock;
 };
 
 struct dt_device_node;
@@ -52,4 +65,9 @@ int imsic_init(const struct dt_device_node *node);
 
 const struct imsic_config *imsic_get_config(void);
 
+void imsic_irq_enable(unsigned int hwirq);
+void imsic_irq_disable(unsigned int hwirq);
+
+void imsic_ids_local_delivery(bool enable);
+
 #endif /* ASM_RISCV_IMSIC_H */
Signature · 1 line
2.49.0
#14 lore raw

PATCH v4 7/9 xen/riscv: add external interrupt handling for hypervisor mode

In reply to Oleksii Kurochko

Implement functions necessarry to have working external interrupts in hypervisor mode. The following changes are done:
- Add a common function intc_handle_external_irq() to call APLIC specific function to handle an interrupt.
- Update do_trap() function to handle IRQ_S_EXT case; add the check to catch case when cause of trap is an interrupt.
- Add handle_interrrupt() member to intc_hw_operations structure.
- Enable local interrupt delivery for IMSIC by calling of imsic_ids_local_delivery() in imsic_init(); additionally introduce helper imsic_csr_write() to update IMSIC_EITHRESHOLD and IMSIC_EITHRESHOLD.
- Enable hypervisor external interrupts.
- Implement aplic_handler_interrupt() and use it to init ->handle_interrupt member of intc_hw_operations for APLIC.
- Add implementation of do_IRQ() to dispatch the interrupt.

The current patch is based on the code from [1].

[1] https://gitlab.com/xen-project/people/olkur/xen/-/commit/7390e2365828b83e27ead56b03114a56e3699dd5

---
Changes in V4:
- Nothing changed. Only rebase.
---
Changes in V3:
- Add ASSERT(spin_is_locked(&desc->lock)) to aplic_set_irq_type().
- Fix code style for switch () in aplic_set_irq_type().
- Drop fallthrough between "case IRQ_TYPE_NONE: case IRQ_TYPE_INVALID:" as there is no other statements in "case IRQ_TYPE_NONE".
- Add Acked-by: Jan Beulich <[email protected]>.
---
Changes in V2:
- use BIT() macros instead of 1UL << bit_num in aplic.c.
- Drop passing of a cause to aplic_handle_interrupt() function. And update prototype of handle_interrupt() callback.
- Drop ASSERT() in intc_handle_external_irqs() as it is useless.
- Code style fixes.
- Drop cause argument for intc_handle_external_irqs().
- Update the commit message: drop words that imsic_ids_local_delivery() is implemented in this patch, it is only called.
- Add cf_check for aplic_handle_interrupt(), aplic_set_irq_type().
- Move forward declarations in asm/intc.h up.
- Use plain C operator instead if {clear,set}_bit() for desc->status as it is always used under spinlock().
- use writel() for writing to APLIC's sourcecfg in aplic_set_irq_type().

---
 xen/arch/riscv/aplic.c             | 71 ++++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/aplic.h |  7 +++
 xen/arch/riscv/include/asm/imsic.h |  1 +
 xen/arch/riscv/include/asm/intc.h  |  6 +++
 xen/arch/riscv/include/asm/irq.h   |  6 ++-
 xen/arch/riscv/intc.c              |  5 +++
 xen/arch/riscv/irq.c               | 47 ++++++++++++++++++++
 xen/arch/riscv/traps.c             | 19 ++++++++
 8 files changed, 161 insertions(+), 1 deletion(-)
xen/arch/riscv/aplic.c +71 -0
index 7d2a1ea7aa..763c78617b 100644
@@ -9,6 +9,7 @@
  * Copyright (c) 2024-2025 Vates
  */
 
+#include <xen/const.h>
 #include <xen/device_tree.h>
 #include <xen/errno.h>
 #include <xen/init.h>
@@ -228,6 +229,71 @@ static void cf_check aplic_set_irq_affinity(struct irq_desc *desc, const cpumask
     spin_unlock(&aplic.lock);
 }
 
+static void cf_check aplic_handle_interrupt(struct cpu_user_regs *regs)
+{
+    /* disable to avoid more external interrupts */
+    csr_clear(CSR_SIE, BIT(IRQ_S_EXT, UL));
+
+    /* clear the pending bit */
+    csr_clear(CSR_SIP, BIT(IRQ_S_EXT, UL));
+
+    /* dispatch the interrupt */
+    do_IRQ(regs, csr_swap(CSR_STOPEI, 0) >> TOPI_IID_SHIFT);
+
+    /* enable external interrupts */
+    csr_set(CSR_SIE, BIT(IRQ_S_EXT, UL));
+}
+
+static void cf_check aplic_set_irq_type(struct irq_desc *desc, unsigned int type)
+{
+    /*
+    * Interrupt 0 isn't possible based on the spec:
+    *   Each of an APLIC’s interrupt sources has a fixed unique identity number in the range 1 to N,
+    *   where N is the total number of sources at the APLIC. The number zero is not a valid interrupt
+    *   identity number at an APLIC. The maximum number of interrupt sources an APLIC may support
+    *   is 1023.
+    *
+    * Thereby interrupt 1 will correspond to bit 0 in sourcecfg[] register,
+    * interrupt 2 ->sourcecfg[1] and so on.
+    *
+    * And that is the reason why we need -1.
+    */
+    unsigned int irq_bit = desc->irq - 1;
+
+    ASSERT(spin_is_locked(&desc->lock));
+
+    spin_lock(&aplic.lock);
+
+    switch ( type )
+    {
+    case IRQ_TYPE_EDGE_RISING:
+        writel(APLIC_SOURCECFG_SM_EDGE_RISE, &aplic.regs->sourcecfg[irq_bit]);
+        break;
+
+    case IRQ_TYPE_EDGE_FALLING:
+        writel(APLIC_SOURCECFG_SM_EDGE_FALL, &aplic.regs->sourcecfg[irq_bit]);
+        break;
+
+    case IRQ_TYPE_LEVEL_HIGH:
+        writel(APLIC_SOURCECFG_SM_LEVEL_HIGH, &aplic.regs->sourcecfg[irq_bit]);
+        break;
+
+    case IRQ_TYPE_LEVEL_LOW:
+        writel(APLIC_SOURCECFG_SM_LEVEL_LOW, &aplic.regs->sourcecfg[irq_bit]);
+        break;
+
+    case IRQ_TYPE_NONE:
+    case IRQ_TYPE_INVALID:
+        writel(APLIC_SOURCECFG_SM_INACTIVE, &aplic.regs->sourcecfg[irq_bit]);
+        break;
+
+    default:
+        panic("%s: APLIC doesnt support IRQ type: 0x%x?\n", __func__, type);
+    }
+
+    spin_unlock(&aplic.lock);
+}
+
 static const hw_irq_controller aplic_xen_irq_type = {
     .typename     = "aplic",
     .startup      = aplic_irq_startup,
@@ -241,6 +307,8 @@ static const struct intc_hw_operations aplic_ops = {
     .info                = &aplic_info,
     .init                = aplic_init,
     .host_irq_type       = &aplic_xen_irq_type,
+    .handle_interrupt    = aplic_handle_interrupt,
+    .set_irq_type        = aplic_set_irq_type,
 };
 
 static int cf_check aplic_irq_xlate(const uint32_t *intspec,
@@ -280,6 +348,9 @@ static int __init aplic_preinit(struct dt_device_node *node, const void *dat)
 
     register_intc_ops(&aplic_ops);
 
+    /* Enable supervisor external interrupt */
+    csr_set(CSR_SIE, BIT(IRQ_S_EXT, UL));
+
     return 0;
 }
 
xen/arch/riscv/include/asm/aplic.h +7 -0
index c3f00238f2..b335b29d34 100644
@@ -18,6 +18,13 @@
 #define APLIC_DOMAINCFG_IE      BIT(8, U)
 #define APLIC_DOMAINCFG_DM      BIT(2, U)
 
+#define APLIC_SOURCECFG_SM_INACTIVE     0x0
+#define APLIC_SOURCECFG_SM_DETACH       0x1
+#define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
+#define APLIC_SOURCECFG_SM_EDGE_FALL    0x5
+#define APLIC_SOURCECFG_SM_LEVEL_HIGH   0x6
+#define APLIC_SOURCECFG_SM_LEVEL_LOW    0x7
+
 #define APLIC_TARGET_HART_IDX_SHIFT 18
 
 struct aplic_regs {
xen/arch/riscv/include/asm/imsic.h +1 -0
index 378e49d933..c6c59215df 100644
@@ -12,6 +12,7 @@
 #define ASM_RISCV_IMSIC_H
 
 #include <xen/spinlock.h>
+#include <xen/stdbool.h>
 #include <xen/types.h>
 
 #define IMSIC_MMIO_PAGE_SHIFT   12
xen/arch/riscv/include/asm/intc.h +6 -0
index a11b7aa55e..ecdc8a5e65 100644
@@ -14,6 +14,7 @@ enum intc_version {
     INTC_APLIC,
 };
 
+struct cpu_user_regs;
 struct irq_desc;
 
 struct intc_info {
@@ -37,6 +38,9 @@ struct intc_hw_operations {
     void (*set_irq_type)(struct irq_desc *desc, unsigned int type);
     /* Set IRQ priority */
     void (*set_irq_priority)(struct irq_desc *desc, unsigned int priority);
+
+    /* handle external interrupt */
+    void (*handle_interrupt)(struct cpu_user_regs *regs);
 };
 
 void intc_preinit(void);
@@ -47,4 +51,6 @@ void intc_init(void);
 
 void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority);
 
+void intc_handle_external_irqs(struct cpu_user_regs *regs);
+
 #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
xen/arch/riscv/include/asm/irq.h +5 -1
index 84c3c2904d..94151eb083 100644
@@ -33,16 +33,20 @@ struct arch_irq_desc {
     unsigned int type;
 };
 
+struct cpu_user_regs;
+struct dt_device_node;
+
 static inline void arch_move_irqs(struct vcpu *v)
 {
     BUG_ON("unimplemented");
 }
 
-struct dt_device_node;
 int platform_get_irq(const struct dt_device_node *device, int index);
 
 void init_IRQ(void);
 
+void do_IRQ(struct cpu_user_regs *regs, unsigned int irq);
+
 #endif /* ASM__RISCV__IRQ_H */
 
 /*
xen/arch/riscv/intc.c +5 -0
index f2823267a9..ea317aea5a 100644
@@ -50,6 +50,11 @@ static void intc_set_irq_priority(struct irq_desc *desc, unsigned int priority)
         intc_hw_ops->set_irq_priority(desc, priority);
 }
 
+void intc_handle_external_irqs(struct cpu_user_regs *regs)
+{
+    intc_hw_ops->handle_interrupt(regs);
+}
+
 void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
 {
     ASSERT(desc->status & IRQ_DISABLED);
xen/arch/riscv/irq.c +47 -0

56 lines not on this page · raw message

xen/arch/riscv/traps.c +19 -0
index ea3638a54f..f061004d83 100644
@@ -11,6 +11,7 @@
 #include <xen/nospec.h>
 #include <xen/sched.h>
 
+#include <asm/intc.h>
 #include <asm/processor.h>
 #include <asm/riscv_encoding.h>
 #include <asm/traps.h>
@@ -128,6 +129,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
         }
         fallthrough;
     default:
+        if ( cause & CAUSE_IRQ_FLAG )
+        {
+            /* Handle interrupt */
+            unsigned long icause = cause & ~CAUSE_IRQ_FLAG;
+
+            switch ( icause )
+            {
+            case IRQ_S_EXT:
+                intc_handle_external_irqs(cpu_regs);
+                break;
+
+            default:
+                break;
+            }
+
+            break;
+        }
+
         do_unexpected_trap(cpu_regs);
         break;
     }
Signature · 1 line
2.49.0
#17 lore raw

PATCH v4 9/9 xen/riscv: add basic UART support

In reply to Oleksii Kurochko

Update Kconfig to select GENERIC_UART_INIT for basic UART init ( find a dt node and call device specific device_init() ).

Drop default n if RISCV statement for config HAS_NS16550 as now ns16550 is ready to be compiled and used by RISC-V. Also, make the config user selectable for everyone except X86.

Initialize a minimal amount of stuff to have UART and Xen console:
- Initialize uart by calling uart_init().
- Initialize Xen console by calling console_init_{pre,post}irq().
- Initialize timer and its internal lists which are used by init_timer() which is called by ns16550_init_postirq(); otherwise "Unhandled exception: Store/AMO Page Fault" occurs.
- Enable local interrupt to recieve an input from UART

---
Changes in V4:
- Nothing changed. Only rebase.
---
Changes in v3:
- Drop inclusion of <xen/percpu.h> as nothing in setup.c requires it.
- Add Acked-by: Jan Beulich <[email protected]>.
---
Changes in v2:
- Drop #include <xen/keyhandler.h> in setup.c, isn't needed anymore.
- Drop call of percpu_init_areas() as it was needed when I used polling mode for UART, for this case percpu is used to receive serial port info:

     struct serial_port *port = this_cpu(poll_port);
   So percpu isn't really needed at the current development state.
 - Make HAS_NS16550 user selectable for everyone, except X86.
 - Update the commit message.
---
 xen/arch/riscv/Kconfig   |  1 +
 xen/arch/riscv/setup.c   | 12 ++++++++++++
 xen/drivers/char/Kconfig |  3 +--
 3 files changed, 14 insertions(+), 2 deletions(-)
xen/arch/riscv/Kconfig +1 -0
index 62c5b7ba34..96bef90751 100644
@@ -2,6 +2,7 @@ config RISCV
 	def_bool y
 	select FUNCTION_ALIGNMENT_16B
 	select GENERIC_BUG_FRAME
+	select GENERIC_UART_INIT
 	select HAS_DEVICE_TREE
 	select HAS_PMAP
 	select HAS_UBSAN
xen/arch/riscv/setup.c +12 -0
index 0e7398159c..a17096bf02 100644
@@ -4,12 +4,15 @@
 #include <xen/bug.h>
 #include <xen/bootfdt.h>
 #include <xen/compile.h>
+#include <xen/console.h>
 #include <xen/device_tree.h>
 #include <xen/init.h>
 #include <xen/irq.h>
 #include <xen/mm.h>
+#include <xen/serial.h>
 #include <xen/shutdown.h>
 #include <xen/smp.h>
+#include <xen/timer.h>
 #include <xen/vmap.h>
 #include <xen/xvmalloc.h>
 
@@ -134,8 +137,17 @@ void __init noreturn start_xen(unsigned long bootcpu_id,
 
     intc_preinit();
 
+    uart_init();
+    console_init_preirq();
+
     intc_init();
 
+    timer_init();
+
+    local_irq_enable();
+
+    console_init_postirq();
+
     printk("All set up\n");
 
     machine_halt();
xen/drivers/char/Kconfig +1 -2
index e6e12bb413..8e49a52c73 100644
@@ -2,8 +2,7 @@ config GENERIC_UART_INIT
 	bool
 
 config HAS_NS16550
-	bool "NS16550 UART driver" if ARM
-	default n if RISCV
+	bool "NS16550 UART driver" if !X86
 	default y
 	help
 	  This selects the 16550-series UART support. For most systems, say Y.
Signature · 1 line
2.49.0
#15 lore raw

PATCH v4 8/9 xen/riscv: implement setup_irq()

In reply to Oleksii Kurochko

Introduce support for IRQ setup on RISC-V by implementing setup_irq() and __setup_irq(), adapted and extended from an initial implementation by [1].

__setup_irq() does the following:
- Sets up an IRQ action.
- Validates that shared IRQs have non-NULL dev_id and are only used when existing handlers allow sharing.
- Uses smp_wmb() to enforce memory ordering after assigning desc->action to ensure visibility before enabling the IRQ.
- Supports multi-action setups via CONFIG_IRQ_HAS_MULTIPLE_ACTION.

setup_irq() does the following:
- Converts IRQ number to descriptor and acquires its lock.
- Rejects registration if the IRQ is already assigned to a guest domain, printing an error.
- Delegates the core setup to __setup_irq().
- On first-time setup, disables the IRQ, routes it to Xen using intc_route_irq_to_xen(), sets default CPU affinity (current CPU), calls the handler’s startup routine, and finally enables the IRQ.

irq_set_affinity() invokes set_affinity() callback from the IRQ handler if present.

Defined IRQ_NO_PRIORITY as default priority used when routing IRQs to Xen.

[1] https://gitlab.com/xen-project/people/olkur/xen/-/commit/7390e2365828b83e27ead56b03114a56e3699dd5

---
Changes in V3-4:
- Nothing changed. Only rebase.
---
Changes in V2:
- Added implenmtation of aplic_set_irq_type() as it is going to be used in this commit. And also, update the implementation of it. Make default case of switch to do panic().
- Move all forward declaration up in asm/irq.h.
- s/__setup_irq/_setup_irq.
- Code style fixes.
- Update commit message.
- use smp_wmb() instead of smp_mb() in _setup_irq().
- Drop irq_set_affinity().
- Use plain C operator instead if {clear,set}_bit() for desc->status as it is always used under spinlock().
- Drop set_bit(_IRQ_DISABLED, &desc->status) in setup_irq() as in the case when IRQ is setuped for a first time, desc->status should be already set to IRQ_DISABLED in init_one_irq_desc().
----

 xen/arch/riscv/include/asm/irq.h |  2 +
 xen/arch/riscv/irq.c             | 84 ++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+)
xen/arch/riscv/include/asm/irq.h +2 -0
index 94151eb083..f633636dc3 100644
@@ -17,6 +17,8 @@
  */
 #define NR_IRQS 1024
 
+#define IRQ_NO_PRIORITY 0
+
 /* TODO */
 #define nr_irqs 0U
 #define nr_static_irqs 0
xen/arch/riscv/irq.c +84 -0

96 lines not on this page · raw message

Signature · 1 line
2.49.0
#11 lore raw

PATCH v4 5/9 xen/riscv: introduce intc_init() and helpers

In reply to Oleksii Kurochko

Introduce intc_init() to initialize the interrupt controller using the registered hardware ops.
Also add intc_route_irq_to_xen() to route IRQs to Xen, with support for setting IRQ type and priority via new internal helpers intc_set_irq_type() and intc_set_irq_priority().

Call intc_init() to do basic initialization steps for APLIC and IMSIC.

---
Changes in V4:
- Add Acked-by: Jan Beulich <[email protected]>.
---
Changes in V3:
- Drop ASSERIT(intc_hw_ops) in intc_init().
- Drop ASSERT(intc_hw_ops) in intc_set_irq_type() and intc_set_irq_priority() as intc_init() will be called first and so if it won't crash, then intc_hw_ops is registered.
---
Changes in V2:
- This patch was part of "xen/riscv: Introduce intc_hw_operations abstraction" and splitted to have ability to merge patch "xen/riscv: initialize interrupt controller" to the current patch (where intc_init() call is actually introduced).
- Add checks of that callbacks aren't set to NULL in intc_set_irq_priority() and intc_set_irq_type().
- add num_irqs member to struct intc_info as it is used now in intc_route_irq_to_xen().
- Add ASSERT(spin_is_locked(&desc->lock)) to intc_set_irq_priority() in
the case this function will be called outside intc_route_irq_to_xen().

---
 xen/arch/riscv/include/asm/intc.h |  4 +++
 xen/arch/riscv/intc.c             | 41 +++++++++++++++++++++++++++++++
 xen/arch/riscv/setup.c            |  2 ++
 3 files changed, 47 insertions(+)
xen/arch/riscv/include/asm/intc.h +4 -0
index 3c4b211f58..a11b7aa55e 100644
@@ -43,4 +43,8 @@ void intc_preinit(void);
 
 void register_intc_ops(const struct intc_hw_operations *ops);
 
+void intc_init(void);
+
+void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority);
+
 #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
xen/arch/riscv/intc.c +41 -0

53 lines not on this page · raw message

xen/arch/riscv/setup.c +2 -0
index 8bcd19218d..0e7398159c 100644
@@ -134,6 +134,8 @@ void __init noreturn start_xen(unsigned long bootcpu_id,
 
     intc_preinit();
 
+    intc_init();
+
     printk("All set up\n");
 
     machine_halt();
Signature · 1 line
2.49.0
#2 lore raw

Re: [PATCH v4 1/9] xen/riscv: dt_processor_hartid() implementation

In reply to Oleksii Kurochko
Quoted 13 lines · Oleksii Kurochko
xen/arch/riscv/smpboot.c +9 -0
@@ -14,3 +17,77 @@ void __init smp_prepare_boot_cpu(void)
     cpumask_set_cpu(0, &cpu_possible_map);
     cpumask_set_cpu(0, &cpu_online_map);
 }
+
+/**
+ * dt_get_hartid - Get the hartid from a CPU device node
+ *
+ * @cpun: CPU number(logical index) for which device node is required
+ *
+ * Return: The hartid for the CPU node or ~0UL if not found.
+ */
+static unsigned long dt_get_hartid(const struct dt_device_node *cpun)

xen/arch/riscv/smpboot.c void __init smp_prepare_boot_cpu(void) · line 38

Quoted 10 lines
xen/arch/riscv/smpboot.c +10 -0
+{
+    const __be32 *cell;
+    unsigned int ac;
+    uint32_t len;
+    unsigned int max_cells = UINT32_MAX / sizeof(*cell);
+
+    ac = dt_n_addr_cells(cpun);
+    cell = dt_get_property(cpun, "reg", &len);
+
+    if (ac > max_cells) {

Besides the (double) style issue, why's this needed? Can't you simply ...

xen/arch/riscv/smpboot.c void __init smp_prepare_boot_cpu(void) · line 44

Quoted 6 lines
xen/arch/riscv/smpboot.c +6 -0
+        printk("%s: cell count overflow (ac=%u, max=%u)\n", __func__, ac,
+               max_cells);
+        return ~0UL;
+    }
+
+    if ( !cell || !ac || ((sizeof(*cell) * ac) > len) )

... write the last part here in a way that there can't be overflow? ac > len / sizeof(*cell) that is? (Remaining question then is what to do when len isn't evenly divisible by sizeof(*cell).)

xen/arch/riscv/smpboot.c void __init smp_prepare_boot_cpu(void) · line 47

Quoted 3 lines
xen/arch/riscv/smpboot.c +3 -0
+        return ~0UL;
+
+    return dt_read_number(cell, ac);

What meaning does this have for ac > 2? (As per your checking above it can be up to UINT32_MAX / 4.)

Jan

#7 lore raw

Re: [PATCH v4 3/9] xen/riscv: imsic_init() implementation

In reply to Oleksii Kurochko
Quoted 18 lines · Oleksii Kurochko
xen/arch/riscv/imsic.c new file +15 -0
@@ -0,0 +1,358 @@
+/* SPDX-License-Identifier: MIT */
+
+/*
+ * xen/arch/riscv/imsic.c
+ *
+ * RISC-V Incoming MSI Controller support
+ *
+ * (c) Microchip Technology Inc.
+ * (c) Vates
+ */
+
+#include <xen/bitops.h>
+#include <xen/const.h>
+#include <xen/cpumask.h>
+#include <xen/device_tree.h>

xen/arch/riscv/imsic.c line 25

Quoted 10 lines
xen/arch/riscv/imsic.c new file +10 -0
+#include <xen/errno.h>
+#include <xen/init.h>
+#include <xen/macros.h>
+#include <xen/smp.h>
+#include <xen/spinlock.h>
+#include <xen/xvmalloc.h>
+
+#include <asm/imsic.h>
+
+#define IMSIC_HART_SIZE(guest_bits_) (BIT(guest_bits_, U) * IMSIC_MMIO_PAGE_SZ)

Minor: Any particular reason for the trailing underscore here?

Quoted 66 lines
xen/arch/riscv/imsic.c new file +66 -0
+/*
+ * Initialize the imsic_cfg structure based on the IMSIC DT node.
+ *
+ * Returns 0 if initialization is successful, a negative value on failure,
+ * or IRQ_M_EXT if the IMSIC node corresponds to a machine-mode IMSIC,
+ * which should be ignored by the hypervisor.
+ */
+int __init imsic_init(const struct dt_device_node *node)
+{
+    int rc;
+    unsigned long reloff, hartid;
+    unsigned int nr_parent_irqs, index, nr_handlers = 0;
+    paddr_t base_addr;
+    unsigned int nr_mmios;
+    struct imsic_mmios *mmios;
+    struct imsic_msi *msi = NULL;
+
+    /* Parse IMSIC node */
+    rc = imsic_parse_node(node, &nr_parent_irqs, &nr_mmios);
+    /*
+     * If machine mode imsic node => ignore it.
+     * If rc < 0 => parsing of IMSIC DT node failed.
+     */
+    if ( (rc == IRQ_M_EXT) || (rc < 0) )
+        return rc;
+
+    /* Allocate MMIO resource array */
+    mmios = xvzalloc_array(struct imsic_mmios, nr_mmios);
+    if ( !mmios )
+    {
+        rc = -ENOMEM;
+        goto imsic_init_err;
+    }
+
+    msi = xvzalloc_array(struct imsic_msi, nr_parent_irqs);
+    if ( !msi )
+    {
+        rc = -ENOMEM;
+        goto imsic_init_err;
+    }
+
+    /* Check MMIO register sets */
+    for ( unsigned int i = 0; i < nr_mmios; i++ )
+    {
+        unsigned int guest_bits = imsic_cfg.guest_index_bits;
+        unsigned long expected_mmio_size =
+            IMSIC_HART_SIZE(guest_bits) * nr_parent_irqs;
+
+        rc = dt_device_get_address(node, i, &mmios[i].base_addr,
+                                   &mmios[i].size);
+        if ( rc )
+        {
+            printk(XENLOG_ERR "%s: unable to parse MMIO regset %u\n",
+                   node->name, i);
+            goto imsic_init_err;
+        }
+
+        base_addr = mmios[i].base_addr;
+        base_addr &= ~(BIT(guest_bits +
+                           imsic_cfg.hart_index_bits +
+                           IMSIC_MMIO_PAGE_SHIFT, UL) - 1);
+        base_addr &= ~((BIT(imsic_cfg.group_index_bits, UL) - 1) <<
+                       imsic_cfg.group_index_shift);
+        if ( base_addr != imsic_cfg.base_addr )
+        {
+            rc = -EINVAL;

xen/arch/riscv/imsic.c line 273

Quoted 10 lines
xen/arch/riscv/imsic.c new file +10 -0
+            printk(XENLOG_ERR "%s: address mismatch for regset %u\n",
+                   node->name, i);
+            goto imsic_init_err;
+        }
+
+        if ( mmios[i].size != expected_mmio_size )
+        {
+            rc = -EINVAL;
+            printk(XENLOG_ERR "%s: IMSIC MMIO size is incorrect %ld, "
+                   "expeced MMIO size: %ld\n", node->name, mmios[i].size,

To aid grep-ability, please avoid wrapping format strings across lines. (Same at least once more elsewhere.)

xen/arch/riscv/imsic.c line 282

Quoted 9 lines
xen/arch/riscv/imsic.c new file +9 -0
+                   expected_mmio_size);
+            goto imsic_init_err;
+        }
+    }
+
+    /* Configure handlers for target CPUs */
+    for ( unsigned int i = 0; i < nr_parent_irqs; i++ )
+    {
+        unsigned long cpu;

Along the lines of questions on earlier versions: Any reason this isn't unsigned int?

Quoted 9 lines
xen/arch/riscv/imsic.c new file +9 -0
+        rc = imsic_get_parent_hartid(node, i, &hartid);
+        if ( rc )
+        {
+            printk(XENLOG_WARNING "%s: cpu ID for parent irq%u not found\n",
+                   node->name, i);
+            continue;
+        }
+
+        cpu = hartid_to_cpuid(hartid);

xen/arch/riscv/imsic.c line 302

Quoted 10 lines
xen/arch/riscv/imsic.c new file +10 -0
+
+        if ( cpu >= num_possible_cpus() )
+        {
+            printk(XENLOG_WARNING "%s: unsupported hart ID=%#lx for parent "
+                   "irq%u\n", node->name, hartid, i);
+            continue;
+        }
+
+        /* Find MMIO location of MSI page */
+        reloff = i * BIT(imsic_cfg.guest_index_bits, UL) * IMSIC_MMIO_PAGE_SZ;

Any reason to open-code IMSIC_HART_SIZE() here and ...

xen/arch/riscv/imsic.c line 314

Quoted 12 lines
xen/arch/riscv/imsic.c new file +12 -0
+        for ( index = 0; index < nr_mmios; index++ )
+        {
+            if ( reloff < mmios[index].size )
+                break;
+
+            /*
+             * MMIO region size may not be aligned to
+             * BIT(global->guest_index_bits) * IMSIC_MMIO_PAGE_SZ
+             * if holes are present.
+             */
+            reloff -= ROUNDUP(mmios[index].size,
+                      BIT(imsic_cfg.guest_index_bits, UL) * IMSIC_MMIO_PAGE_SZ);

... here?

xen/arch/riscv/imsic.c line 324

Quoted 10 lines
xen/arch/riscv/imsic.c new file +10 -0
+        }
+
+        if ( index == nr_mmios )
+        {
+            printk(XENLOG_WARNING "%s: MMIO not found for parent irq%u\n",
+                   node->name, i);
+            continue;
+        }
+
+        if ( !IS_ALIGNED(msi[cpu].base_addr + reloff,

DYM mmios[index].base_addr here, considering that ...

xen/arch/riscv/imsic.c line 335

Quoted 11 lines
xen/arch/riscv/imsic.c new file +11 -0
+                         IMSIC_MMIO_PAGE_SZ) )
+        {
+            printk(XENLOG_WARNING "%s: MMIO address %#lx is not aligned on "
+                   "a page\n", node->name, msi[cpu].base_addr + reloff);
+            msi[cpu].offset = 0;
+            msi[cpu].base_addr = 0;
+            continue;
+        }
+
+        msi[cpu].base_addr = mmios[index].base_addr;
+        msi[cpu].offset = reloff;

msi[cpu] is set only here?

Also is the setting to zero of both fields on the "continue" path really needed, seeing that the array starts out zero-filled? Can the same CPU be found twice, making it necessary(?) to invalidate the array slot later?

xen/arch/riscv/imsic.c line 349

Quoted 13 lines
xen/arch/riscv/imsic.c new file +13 -0
+        nr_handlers++;
+    }
+
+    if ( !nr_handlers )
+    {
+        printk(XENLOG_ERR "%s: No CPU handlers found\n", node->name);
+        rc = -ENODEV;
+        goto imsic_init_err;
+    }
+
+    imsic_cfg.msi = msi;
+
+    XFREE(mmios);

No need to use the macro, and you really mean xvfree() now.

xen/arch/riscv/imsic.c line 355

Quoted 5 lines
xen/arch/riscv/imsic.c new file +5 -0
+    return 0;
+
+ imsic_init_err:
+    XFREE(mmios);
+    XFREE(msi);

Same here.

Jan

#10 lore raw

Re: [PATCH v4 4/9] xen/riscv: aplic_init() implementation

In reply to Oleksii Kurochko
Quoted 14 lines · Oleksii Kurochko
xen/arch/riscv/aplic.c +14 -0
+static int __init cf_check aplic_init(void)
+{
+    dt_phandle imsic_phandle;
+    const __be32 *prop;
+    uint64_t size, paddr;
+    const struct dt_device_node *imsic_node;
+    const struct dt_device_node *node = aplic_info.node;
+    int rc;
+
+    /* Check for associated imsic node */
+    if ( !dt_property_read_u32(node, "msi-parent", &imsic_phandle) )
+        panic("%s: IDC mode not supported\n", node->full_name);
+
+    imsic_node = dt_find_node_by_phandle(imsic_phandle);

xen/arch/riscv/aplic.c line 82

Quoted 10 lines
xen/arch/riscv/aplic.c +10 -0
+    if ( !imsic_node )
+        panic("%s: unable to find IMSIC node\n", node->full_name);
+
+    rc = imsic_init(imsic_node);
+    if ( rc == IRQ_M_EXT )
+        /* Machine mode imsic node, ignore this aplic node */
+        return 0;
+
+    if ( rc )
+        panic("%s: Failded to initialize IMSIC\n", node->full_name);

Nit: Stray 'd'.

Quoted 9 lines
xen/arch/riscv/aplic.c +9 -0
+    /* Find out number of interrupt sources */
+    if ( !dt_property_read_u32(node, "riscv,num-sources",
+                               &aplic_info.num_irqs) )
+        panic("%s: failed to get number of interrupt sources\n",
+              node->full_name);
+
+    if ( aplic_info.num_irqs > ARRAY_SIZE(aplic.regs->sourcecfg) )
+        aplic_info.num_irqs = ARRAY_SIZE(aplic.regs->sourcecfg);
+

xen/arch/riscv/aplic.c line 102

Quoted 10 lines
xen/arch/riscv/aplic.c +10 -0
+    prop = dt_get_property(node, "reg", NULL);
+    dt_get_range(&prop, node, &paddr, &size);
+    if ( !paddr )
+        panic("%s: first MMIO resource not found\n", node->full_name);
+
+    if ( !IS_ALIGNED(paddr, KB(4)) )
+        panic("%s: paddr of memory-mapped control region should be 4Kb "
+              "aligned:%#lx\n", __func__, paddr);
+
+    if ( !IS_ALIGNED(size, KB(4)) && (size < KB(16)) )

DYM || instead of && here?

xen/arch/riscv/aplic.c line 105

Quoted 3 lines
xen/arch/riscv/aplic.c +3 -0
+        panic("%s: memory-mapped control region should be a multiple of 4 KiB "
+              "in size and the smallest valid control is 16Kb: %#lx\n",
+              __func__, size);

Here and above: See my remark on the earlier patch regarding the wrapping of format strings.

Jan

Acked-by on v4 6/9
#13 lore raw

Re: [PATCH v4 6/9] xen/riscv: implementation of aplic and imsic operations

In reply to Oleksii Kurochko
Quoted 18 lines · Oleksii Kurochko

Introduce interrupt controller descriptor for host APLIC to describe the low-lovel hardare. It includes implementation of the following functions:
- aplic_irq_startup()
- aplic_irq_enable()
- aplic_irq_disable()
- aplic_set_irq_affinity()

As APLIC is used in MSI mode it requires to enable/disable interrupts not only for APLIC but also for IMSIC. Thereby for the purpose of aplic_irq_{enable,disable}() it is introduced imsic_irq_{enable,disable)().

For the purpose of aplic_set_irq_affinity() aplic_get_cpu_from_mask() is introduced to get hart id.

Also, introduce additional interrupt controller h/w operations and host_irq_type for APLIC:
- aplic_host_irq_type

Quoted 6 lines

Patch is based on the code from [1].

[1] https://gitlab.com/xen-project/people/olkur/xen/-/commit/7390e2365828b83e27ead56b03114a56e3699dd5

Acked-by on v4 8/9
#16 lore raw

Re: [PATCH v4 8/9] xen/riscv: implement setup_irq()

In reply to Oleksii Kurochko
Quoted 23 lines · Oleksii Kurochko

Introduce support for IRQ setup on RISC-V by implementing setup_irq() and __setup_irq(), adapted and extended from an initial implementation by [1].

__setup_irq() does the following:
- Sets up an IRQ action.
- Validates that shared IRQs have non-NULL dev_id and are only used when existing handlers allow sharing.
- Uses smp_wmb() to enforce memory ordering after assigning desc->action to ensure visibility before enabling the IRQ.
- Supports multi-action setups via CONFIG_IRQ_HAS_MULTIPLE_ACTION.

setup_irq() does the following:
- Converts IRQ number to descriptor and acquires its lock.
- Rejects registration if the IRQ is already assigned to a guest domain, printing an error.
- Delegates the core setup to __setup_irq().
- On first-time setup, disables the IRQ, routes it to Xen using intc_route_irq_to_xen(), sets default CPU affinity (current CPU), calls the handler’s startup routine, and finally enables the IRQ.

irq_set_affinity() invokes set_affinity() callback from the IRQ handler if present.

Quoted 6 lines

Defined IRQ_NO_PRIORITY as default priority used when routing IRQs to Xen.

[1] https://gitlab.com/xen-project/people/olkur/xen/-/commit/7390e2365828b83e27ead56b03114a56e3699dd5

#3 lore raw

Re: [PATCH v4 1/9] xen/riscv: dt_processor_hartid() implementation

In reply to Jan Beulich
Quoted 32 lines · Jan Beulich
Quoted 13 lines · Oleksii Kurochko
xen/arch/riscv/smpboot.c +9 -0
@@ -14,3 +17,77 @@ void __init smp_prepare_boot_cpu(void)
      cpumask_set_cpu(0, &cpu_possible_map);
      cpumask_set_cpu(0, &cpu_online_map);
  }
+
+/**
+ * dt_get_hartid - Get the hartid from a CPU device node
+ *
+ * @cpun: CPU number(logical index) for which device node is required
+ *
+ * Return: The hartid for the CPU node or ~0UL if not found.
+ */
+static unsigned long dt_get_hartid(const struct dt_device_node *cpun)

xen/arch/riscv/smpboot.c void __init smp_prepare_boot_cpu(void) · line 38

Quoted 10 lines
xen/arch/riscv/smpboot.c +10 -0
+{
+    const __be32 *cell;
+    unsigned int ac;
+    uint32_t len;
+    unsigned int max_cells = UINT32_MAX / sizeof(*cell);
+
+    ac = dt_n_addr_cells(cpun);
+    cell = dt_get_property(cpun, "reg", &len);
+
+    if (ac > max_cells) {

Besides the (double) style issue, why's this needed? Can't you simply ...

xen/arch/riscv/smpboot.c void __init smp_prepare_boot_cpu(void) · line 44

Quoted 6 lines
xen/arch/riscv/smpboot.c +6 -0
+        printk("%s: cell count overflow (ac=%u, max=%u)\n", __func__, ac,
+               max_cells);
+        return ~0UL;
+    }
+
+    if ( !cell || !ac || ((sizeof(*cell) * ac) > len) )
Quoted 3 lines

... write the last part here in a way that there can't be overflow? ac > len / sizeof(*cell) that is? (Remaining question then is what to do when len isn't evenly divisible by sizeof(*cell).)

reg property should be always evenly divisible by sizeof(*cell) according to device
tree binding:
The reg property describes the address of the device’s resources within
the address space defined by its parent bus. Most commonly this means the offsets and lengths of memory-mapped IO register blocks, but may have a different meaning on some bus types. Addresses in the address space defined by the root node are CPU real addresses.

The value is a <prop-encoded-array>, composed of an arbitrary number of
pairs of address and length, <address length>. The number of <u32> cells required to specify the address and length are bus-specific and are specified by the #address-cells and #size-cells properties in the parent of the device node. If the parent node specifies a value of 0 for #size-cells, the length field in the value of reg shall be omitted. So it is guaranteed by DTC compiler and it would be enough to check overflow in suggested by you way: ac > len / sizeof(*cell) But considering what you noticed below ...

Quoted 6 lines

xen/arch/riscv/smpboot.c void __init smp_prepare_boot_cpu(void) · line 47

Quoted 4 lines
xen/arch/riscv/smpboot.c +3 -0
+        return ~0UL;
+
+    return dt_read_number(cell, ac);

What meaning does this have for ac > 2? (As per your checking above it can be up to UINT32_MAX / 4.)

... It will be an issue for dt_read_number() which could deal only with uint64_t what means we can't have ac > 2. (UINT32_MAX / 4 it is a theoretical maximum IIUC)

Thereby we could do in the following way:
@@ -30,19 +30,18 @@ static unsigned long dt_get_hartid(const struct dt_device_node *cpun)

     const __be32 *cell;
     unsigned int ac;
     uint32_t len;

- unsigned int max_cells = UINT32_MAX / sizeof(*cell);

     ac = dt_n_addr_cells(cpun);
     cell = dt_get_property(cpun, "reg", &len);

- if (ac > max_cells) {
- printk("%s: cell count overflow (ac=%u, max=%u)\n", __func__, ac,
- max_cells);
+ if ( !cell || !ac || (ac > len / sizeof(*cell)) )

         return ~0UL;

- }

- if ( !cell || !ac || ((sizeof(*cell) * ac) > len) )
- return ~0UL;
+ /*

+     * If ac > 2, the result may be truncated or meaningless unless
+     * dt_read_number() supports wider integers.
+     */

+ BUG_ON(ac > 2);

     return dt_read_number(cell, ac);

}

I am not sure that BUG_ON() should be in dt_get_hartid(). Probably it would be better move it to dt_read_number() as if one day support for RV128 will be needed I assume that it will be needed to change a prototype of dt_read_number() to work with address-cells = 3. What do you think? Could I go with the suggested above changes or it would be better to move BUG_ON() to dt_read_number()?

~ Oleksii

#4 lore raw

Re: [PATCH v4 1/9] xen/riscv: dt_processor_hartid() implementation

Quoted 94 lines · Oleksii Kurochko
Quoted 32 lines · Jan Beulich
Quoted 13 lines · Oleksii Kurochko
xen/arch/riscv/smpboot.c +9 -0
@@ -14,3 +17,77 @@ void __init smp_prepare_boot_cpu(void)
      cpumask_set_cpu(0, &cpu_possible_map);
      cpumask_set_cpu(0, &cpu_online_map);
  }
+
+/**
+ * dt_get_hartid - Get the hartid from a CPU device node
+ *
+ * @cpun: CPU number(logical index) for which device node is required
+ *
+ * Return: The hartid for the CPU node or ~0UL if not found.
+ */
+static unsigned long dt_get_hartid(const struct dt_device_node *cpun)

xen/arch/riscv/smpboot.c void __init smp_prepare_boot_cpu(void) · line 38

Quoted 10 lines
xen/arch/riscv/smpboot.c +10 -0
+{
+    const __be32 *cell;
+    unsigned int ac;
+    uint32_t len;
+    unsigned int max_cells = UINT32_MAX / sizeof(*cell);
+
+    ac = dt_n_addr_cells(cpun);
+    cell = dt_get_property(cpun, "reg", &len);
+
+    if (ac > max_cells) {

Besides the (double) style issue, why's this needed? Can't you simply ...

xen/arch/riscv/smpboot.c void __init smp_prepare_boot_cpu(void) · line 44

Quoted 6 lines
xen/arch/riscv/smpboot.c +6 -0
+        printk("%s: cell count overflow (ac=%u, max=%u)\n", __func__, ac,
+               max_cells);
+        return ~0UL;
+    }
+
+    if ( !cell || !ac || ((sizeof(*cell) * ac) > len) )
Quoted 3 lines

... write the last part here in a way that there can't be overflow? ac > len / sizeof(*cell) that is? (Remaining question then is what to do when len isn't evenly divisible by sizeof(*cell).)

reg property should be always evenly divisible by sizeof(*cell) according to device tree binding:
The reg property describes the address of the device’s resources within
the address space defined by its parent bus. Most commonly this means the offsets and lengths of memory-mapped IO register blocks, but may have a different meaning on some bus types. Addresses in the address space defined by the root node are CPU real addresses.

The value is a <prop-encoded-array>, composed of an arbitrary number of
pairs of address and length, <address length>. The number of <u32> cells required to specify the address and length are bus-specific and are specified by the #address-cells and #size-cells properties in the parent of the device node. If the parent node specifies a value of 0 for #size-cells, the length field in the value of reg shall be omitted. So it is guaranteed by DTC compiler and it would be enough to check overflow in suggested by you way: ac > len / sizeof(*cell)
But considering what you noticed below ...

Quoted 6 lines

xen/arch/riscv/smpboot.c void __init smp_prepare_boot_cpu(void) · line 47

Quoted 4 lines
xen/arch/riscv/smpboot.c +3 -0
+        return ~0UL;
+
+    return dt_read_number(cell, ac);

What meaning does this have for ac > 2? (As per your checking above it can be up to UINT32_MAX / 4.)

... It will be an issue for dt_read_number() which could deal only with uint64_t what means we can't have ac > 2. (UINT32_MAX / 4 it is a theoretical maximum IIUC)

Thereby we could do in the following way:

(quoted diff) +6 -7
@@ -30,19 +30,18 @@ static unsigned long dt_get_hartid(const struct dt_device_node *cpun)
      const __be32 *cell;
      unsigned int ac;
      uint32_t len;
-    unsigned int max_cells = UINT32_MAX / sizeof(*cell);
 
      ac = dt_n_addr_cells(cpun);
      cell = dt_get_property(cpun, "reg", &len);
 
-    if (ac > max_cells) {
-        printk("%s: cell count overflow (ac=%u, max=%u)\n", __func__, ac,
-               max_cells);
+    if ( !cell || !ac || (ac > len / sizeof(*cell)) )
          return ~0UL;
-    }
 
-    if ( !cell || !ac || ((sizeof(*cell) * ac) > len) )
-        return ~0UL;
+    /*
+     * If ac > 2, the result may be truncated or meaningless unless
+     * dt_read_number() supports wider integers.
+     */
+    BUG_ON(ac > 2);
 
      return dt_read_number(cell, ac);
  }
Quoted 5 lines

I am not sure that BUG_ON() should be in dt_get_hartid(). Probably it would be better move it to dt_read_number() as if one day support for RV128 will be needed I assume that it will be needed to change a prototype of dt_read_number() to work with address-cells = 3. What do you think? Could I go with the suggested above changes or it would be better to move BUG_ON() to dt_read_number()?

Don't know; the DT maintainers would have to judge. I don't, however, think it should be BUG_ON() - as said several times before, that's a check suitable to cover for possible mistakes in Xen code. Here however you're trying to cover for a flaw in DT.

Jan

#8 lore raw

Re: [PATCH v4 3/9] xen/riscv: imsic_init() implementation

In reply to Jan Beulich
Quoted 22 lines · Jan Beulich
Quoted 21 lines · Oleksii Kurochko
xen/arch/riscv/imsic.c new file +18 -0
@@ -0,0 +1,358 @@
+/* SPDX-License-Identifier: MIT */
+
+/*
+ * xen/arch/riscv/imsic.c
+ *
+ * RISC-V Incoming MSI Controller support
+ *
+ * (c) Microchip Technology Inc.
+ * (c) Vates
+ */
+
+#include <xen/bitops.h>
+#include <xen/const.h>
+#include <xen/cpumask.h>
+#include <xen/device_tree.h>
+#include <xen/errno.h>
+#include <xen/init.h>
+#include <xen/macros.h>
Quoted 8 lines

xen/arch/riscv/imsic.c line 25

Quoted 7 lines
xen/arch/riscv/imsic.c new file +7 -0
+#include <xen/smp.h>
+#include <xen/spinlock.h>
+#include <xen/xvmalloc.h>
+
+#include <asm/imsic.h>
+
+#define IMSIC_HART_SIZE(guest_bits_) (BIT(guest_bits_, U) * IMSIC_MMIO_PAGE_SZ)

Minor: Any particular reason for the trailing underscore here?

Not really, I will drop it.

Quoted 89 lines
Quoted 67 lines
xen/arch/riscv/imsic.c new file +66 -0
+/*
+ * Initialize the imsic_cfg structure based on the IMSIC DT node.
+ *
+ * Returns 0 if initialization is successful, a negative value on failure,
+ * or IRQ_M_EXT if the IMSIC node corresponds to a machine-mode IMSIC,
+ * which should be ignored by the hypervisor.
+ */
+int __init imsic_init(const struct dt_device_node *node)
+{
+    int rc;
+    unsigned long reloff, hartid;
+    unsigned int nr_parent_irqs, index, nr_handlers = 0;
+    paddr_t base_addr;
+    unsigned int nr_mmios;
+    struct imsic_mmios *mmios;
+    struct imsic_msi *msi = NULL;
+
+    /* Parse IMSIC node */
+    rc = imsic_parse_node(node, &nr_parent_irqs, &nr_mmios);
+    /*
+     * If machine mode imsic node => ignore it.
+     * If rc < 0 => parsing of IMSIC DT node failed.
+     */
+    if ( (rc == IRQ_M_EXT) || (rc < 0) )
+        return rc;
+
+    /* Allocate MMIO resource array */
+    mmios = xvzalloc_array(struct imsic_mmios, nr_mmios);
+    if ( !mmios )
+    {
+        rc = -ENOMEM;
+        goto imsic_init_err;
+    }
+
+    msi = xvzalloc_array(struct imsic_msi, nr_parent_irqs);
+    if ( !msi )
+    {
+        rc = -ENOMEM;
+        goto imsic_init_err;
+    }
+
+    /* Check MMIO register sets */
+    for ( unsigned int i = 0; i < nr_mmios; i++ )
+    {
+        unsigned int guest_bits = imsic_cfg.guest_index_bits;
+        unsigned long expected_mmio_size =
+            IMSIC_HART_SIZE(guest_bits) * nr_parent_irqs;
+
+        rc = dt_device_get_address(node, i, &mmios[i].base_addr,
+                                   &mmios[i].size);
+        if ( rc )
+        {
+            printk(XENLOG_ERR "%s: unable to parse MMIO regset %u\n",
+                   node->name, i);
+            goto imsic_init_err;
+        }
+
+        base_addr = mmios[i].base_addr;
+        base_addr &= ~(BIT(guest_bits +
+                           imsic_cfg.hart_index_bits +
+                           IMSIC_MMIO_PAGE_SHIFT, UL) - 1);
+        base_addr &= ~((BIT(imsic_cfg.group_index_bits, UL) - 1) <<
+                       imsic_cfg.group_index_shift);
+        if ( base_addr != imsic_cfg.base_addr )
+        {
+            rc = -EINVAL;

xen/arch/riscv/imsic.c line 273

Quoted 10 lines
xen/arch/riscv/imsic.c new file +10 -0
+            printk(XENLOG_ERR "%s: address mismatch for regset %u\n",
+                   node->name, i);
+            goto imsic_init_err;
+        }
+
+        if ( mmios[i].size != expected_mmio_size )
+        {
+            rc = -EINVAL;
+            printk(XENLOG_ERR "%s: IMSIC MMIO size is incorrect %ld, "
+                   "expeced MMIO size: %ld\n", node->name, mmios[i].size,

To aid grep-ability, please avoid wrapping format strings across lines. (Same at least once more elsewhere.)

xen/arch/riscv/imsic.c line 282

Quoted 9 lines
xen/arch/riscv/imsic.c new file +9 -0
+                   expected_mmio_size);
+            goto imsic_init_err;
+        }
+    }
+
+    /* Configure handlers for target CPUs */
+    for ( unsigned int i = 0; i < nr_parent_irqs; i++ )
+    {
+        unsigned long cpu;
Quoted 2 lines

Along the lines of questions on earlier versions: Any reason this isn't unsigned int?

Failed to do proper cleanup. It should be unsigned int.

Quoted 34 lines
Quoted 10 lines
xen/arch/riscv/imsic.c new file +9 -0
+        rc = imsic_get_parent_hartid(node, i, &hartid);
+        if ( rc )
+        {
+            printk(XENLOG_WARNING "%s: cpu ID for parent irq%u not found\n",
+                   node->name, i);
+            continue;
+        }
+
+        cpu = hartid_to_cpuid(hartid);

xen/arch/riscv/imsic.c line 302

Quoted 10 lines
xen/arch/riscv/imsic.c new file +10 -0
+
+        if ( cpu >= num_possible_cpus() )
+        {
+            printk(XENLOG_WARNING "%s: unsupported hart ID=%#lx for parent "
+                   "irq%u\n", node->name, hartid, i);
+            continue;
+        }
+
+        /* Find MMIO location of MSI page */
+        reloff = i * BIT(imsic_cfg.guest_index_bits, UL) * IMSIC_MMIO_PAGE_SZ;

Any reason to open-code IMSIC_HART_SIZE() here and ...

xen/arch/riscv/imsic.c line 314

Quoted 12 lines
xen/arch/riscv/imsic.c new file +12 -0
+        for ( index = 0; index < nr_mmios; index++ )
+        {
+            if ( reloff < mmios[index].size )
+                break;
+
+            /*
+             * MMIO region size may not be aligned to
+             * BIT(global->guest_index_bits) * IMSIC_MMIO_PAGE_SZ
+             * if holes are present.
+             */
+            reloff -= ROUNDUP(mmios[index].size,
+                      BIT(imsic_cfg.guest_index_bits, UL) * IMSIC_MMIO_PAGE_SZ);
Quoted 1 line

... here?

Thanks, missed that. I'll update such places.

Quoted 24 lines

xen/arch/riscv/imsic.c line 324

Quoted 11 lines
xen/arch/riscv/imsic.c new file +10 -0
+        }
+
+        if ( index == nr_mmios )
+        {
+            printk(XENLOG_WARNING "%s: MMIO not found for parent irq%u\n",
+                   node->name, i);
+            continue;
+        }
+
+        if ( !IS_ALIGNED(msi[cpu].base_addr + reloff,

DYM mmios[index].base_addr here, considering that ...

xen/arch/riscv/imsic.c line 335

Quoted 11 lines
xen/arch/riscv/imsic.c new file +11 -0
+                         IMSIC_MMIO_PAGE_SZ) )
+        {
+            printk(XENLOG_WARNING "%s: MMIO address %#lx is not aligned on "
+                   "a page\n", node->name, msi[cpu].base_addr + reloff);
+            msi[cpu].offset = 0;
+            msi[cpu].base_addr = 0;
+            continue;
+        }
+
+        msi[cpu].base_addr = mmios[index].base_addr;
+        msi[cpu].offset = reloff;
Quoted 1 line

msi[cpu] is set only here?

Yes, mmios[] should be used instead of msi[]...

Quoted 4 lines

Also is the setting to zero of both fields on the "continue" path really needed, seeing that the array starts out zero-filled? Can the same CPU be found twice, making it necessary(?) to invalidate the array slot later?

Normally no, it can't be that the same CPU is found twice.

On other side, I tried to compile by hand DTS which has two the same CPUs mentioned in IMSIC node and it allows me to do that:
(XEN) Latest ChangeSet:
(XEN) build-id: 2cd98ed5a91bf443a02a9c4e83f05df4c1f7ec61 (XEN) !!! hartid:0
(XEN) imsics: unsupported hart ID=0 for parent irq0
(XEN) !!! hartid:2
(XEN) imsics: unsupported hart ID=0x2 for parent irq1
(XEN) !!! hartid:2
(XEN) imsics: unsupported hart ID=0x2 for parent irq2
(XEN) !!! hartid:3
(XEN) Allocated console ring of 16 KiB.

(unsupported hart ID=... is printed here because I told QEMU that you have 1 cpu, but in DTS I mentioned more then 1)

It seems like it is needed also to check if IMSIC node provides unique cpu in its interrupts-extended property. And to verify that I think it would be enough to check if msi[cpu].base_addr isn't 0:
@@ -408,15 +407,19 @@ int __init imsic_init(const struct dt_device_node *node)

         cpu = hartid_to_cpuid(hartid);

+ if ( msi[cpu].base_addr )
+ panic("%s: cpu%d is found twice in interrupts-extended prop\n",
+ node->name, cpu);
+
Or probably it would be better just ignore by doing 'continue' instead of panic() as it is done in other places in imsic_init().

Thanks.

~ Oleksii