Skip to contentSkip to conversation

Added Interrupt controller emulation for loongarch kvm

RFC Superseded Xianglai Li 2 participants 6 messages Jul 17, 2024

Subsystems (2)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover Added Interrupt controller emulation for loongarch kvm

Before this, the interrupt controller simulation has been completed in the user mode program. In order to reduce the loss caused by frequent switching of the virtual machine monitor from kernel mode to user mode when the guest accesses the interrupt controller, we add the interrupt controller simulation in kvm.

In qemu side implementation is simple, just make a new IPI EXTIOI PCH KVM related several classes, And the interface to access kvm related data is implemented.

Most of the simulation work of the interrupt controller is done in kvm. Because KVM the changes have not been the Linux community acceptance, the patches of this series will have RFC label until KVM patch into the community.

For the implementation of kvm simulation, refer to the following documents.

IPI simulation implementation reference:
https://github.com/loongson/LoongArch-Documentation/tree/main/docs/Loongson-3A5000-usermanual-EN/inter-processor-interrupts-and-communication

EXTIOI simulation implementation reference:
https://github.com/loongson/LoongArch-Documentation/tree/main/docs/Loongson-3A5000-usermanual-EN/io-interrupts/extended-io-interrupts

PCH-PIC simulation implementation reference:
https://github.com/loongson/LoongArch-Documentation/blob/main/docs/Loongson-7A1000-usermanual-EN/interrupt-controller.adoc

For PCH-MSI, we used irqfd mechanism to send the interrupt signal generated by user state to kernel state and then to EXTIOI without maintaining PCH-MSI state in kernel state.

You can easily get the code from the link below:
the kernel:
https://github.com/lixianglai/linux
the branch is: interrupt

the qemu:
https://github.com/lixianglai/qemu
the branch is: interrupt

Please note that the code above is regularly updated based on community reviews.

Cc Paolo Bonzini <[email protected]>
Cc Song Gao <[email protected]>
Cc Huacai Chen <[email protected]>
Cc Jiaxun Yang <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Cornelia Huck <[email protected]>
Cc Bibo Mao <[email protected]>

Xianglai Li (4):
hw/loongarch: Add KVM IPI device support
hw/loongarch: Add KVM extioi device support
hw/loongarch: Add KVM pch pic device support
hw/loongarch: Add KVM pch msi device support

 hw/intc/Kconfig                     |  12 ++
 hw/intc/loongarch_extioi_kvm.c      | 141 +++++++++++++++++++
 hw/intc/loongarch_ipi_kvm.c         | 207 ++++++++++++++++++++++++++++
 hw/intc/loongarch_pch_msi.c         |  42 ++++--
 hw/intc/loongarch_pch_pic.c         |  20 ++-
 hw/intc/loongarch_pch_pic_kvm.c     | 189 +++++++++++++++++++++++++
 hw/intc/meson.build                 |   3 +
 hw/loongarch/virt.c                 | 141 ++++++++++++-------
 include/hw/intc/loongarch_extioi.h  |  34 ++++-
 include/hw/intc/loongarch_pch_msi.h |   2 +-
 include/hw/intc/loongarch_pch_pic.h |  51 ++++++-
 include/hw/intc/loongson_ipi.h      |  22 +++
 include/hw/loongarch/virt.h         |  15 ++
 linux-headers/asm-loongarch/kvm.h   |   7 +
 linux-headers/linux/kvm.h           |   6 +
 15 files changed, 823 insertions(+), 69 deletions(-)
 create mode 100644 hw/intc/loongarch_extioi_kvm.c
 create mode 100644 hw/intc/loongarch_ipi_kvm.c
 create mode 100644 hw/intc/loongarch_pch_pic_kvm.c
Signature · 1 line
2.39.1
#1 lore raw

RFC 1/4 hw/loongarch: Add KVM IPI device support

Added ipi interrupt controller for kvm emulation.
The main process is to send the command word for creating an ipi device to the kernel.
When the VM is saved, the ioctl obtains the ipi interrupt controller data in the kernel and saves it. When the VM is recovered, the saved data is sent to the kernel.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>

 hw/intc/Kconfig                   |   4 +
 hw/intc/loongarch_ipi_kvm.c       | 207 ++++++++++++++++++++++++++++++
 hw/intc/meson.build               |   1 +
 hw/loongarch/virt.c               |  40 ++++--
 include/hw/intc/loongson_ipi.h    |  22 ++++
 linux-headers/asm-loongarch/kvm.h |   1 +
 linux-headers/linux/kvm.h         |   2 +
 target/loongarch/kvm/kvm.c        |   1 +
 8 files changed, 265 insertions(+), 13 deletions(-)
 create mode 100644 hw/intc/loongarch_ipi_kvm.c
hw/intc/Kconfig +4 -0
index 58b6d3a710..19d6809129 100644
@@ -90,6 +90,10 @@ config M68K_IRQC
 config LOONGSON_IPI
     bool
 
+config LOONGARCH_IPI_KVM
+    bool
+    default y
+
 config LOONGARCH_PCH_PIC
     bool
     select UNIMP
hw/intc/loongarch_ipi_kvm.c new file +207 -0

207 lines not on this page · raw message

hw/intc/meson.build +1 -0
index afd1aa51ee..354d43b4f0 100644
@@ -70,6 +70,7 @@ specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_XIVE'],
 		if_true: files('spapr_xive_kvm.c'))
 specific_ss.add(when: 'CONFIG_M68K_IRQC', if_true: files('m68k_irqc.c'))
 specific_ss.add(when: 'CONFIG_LOONGSON_IPI', if_true: files('loongson_ipi.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_IPI_KVM', if_true: files('loongarch_ipi_kvm.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c'))
hw/loongarch/virt.c +27 -13
index e592b1b6b7..2ae5298b62 100644
@@ -48,6 +48,7 @@
 #include "hw/block/flash.h"
 #include "hw/virtio/virtio-iommu.h"
 #include "qemu/error-report.h"
+#include "sysemu/kvm.h"
 
 static bool virt_is_veiointc_enabled(LoongArchVirtMachineState *lvms)
 {
@@ -788,15 +789,32 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
      */
 
     /* Create IPI device */
-    ipi = qdev_new(TYPE_LOONGSON_IPI);
-    qdev_prop_set_uint32(ipi, "num-cpu", ms->smp.cpus);
-    sysbus_realize_and_unref(SYS_BUS_DEVICE(ipi), &error_fatal);
-
-    /* IPI iocsr memory region */
-    memory_region_add_subregion(&lvms->system_iocsr, SMP_IPI_MAILBOX,
-                   sysbus_mmio_get_region(SYS_BUS_DEVICE(ipi), 0));
-    memory_region_add_subregion(&lvms->system_iocsr, MAIL_SEND_ADDR,
-                   sysbus_mmio_get_region(SYS_BUS_DEVICE(ipi), 1));
+    if (kvm_kernel_irqchip_allowed()) {
+        ipi = qdev_new(TYPE_KVM_LOONGARCH_IPI);
+        qdev_prop_set_int32(ipi, "num-cpu", ms->smp.cpus);
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(ipi), &error_fatal);
+    } else {
+        ipi = qdev_new(TYPE_LOONGSON_IPI);
+        qdev_prop_set_uint32(ipi, "num-cpu", ms->smp.cpus);
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(ipi), &error_fatal);
+
+        /* IPI iocsr memory region */
+        memory_region_add_subregion(&lvms->system_iocsr, SMP_IPI_MAILBOX,
+                       sysbus_mmio_get_region(SYS_BUS_DEVICE(ipi), 0));
+        memory_region_add_subregion(&lvms->system_iocsr, MAIL_SEND_ADDR,
+                       sysbus_mmio_get_region(SYS_BUS_DEVICE(ipi), 1));
+
+        for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+            cpu_state = qemu_get_cpu(cpu);
+            cpudev = DEVICE(cpu_state);
+            lacpu = LOONGARCH_CPU(cpu_state);
+            env = &(lacpu->env);
+
+            /* connect ipi irq to cpu irq */
+            qdev_connect_gpio_out(ipi, cpu, qdev_get_gpio_in(cpudev, IRQ_IPI));
+            env->ipistate = ipi;
+        }
+    }
 
     /* Add cpu interrupt-controller */
     fdt_add_cpuic_node(lvms, &cpuintc_phandle);
@@ -807,10 +825,6 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
         lacpu = LOONGARCH_CPU(cpu_state);
         env = &(lacpu->env);
         env->address_space_iocsr = &lvms->as_iocsr;
-
-        /* connect ipi irq to cpu irq */
-        qdev_connect_gpio_out(ipi, cpu, qdev_get_gpio_in(cpudev, IRQ_IPI));
-        env->ipistate = ipi;
     }
 
     /* Create EXTIOI device */
include/hw/intc/loongson_ipi.h +22 -0
index 3f795edbf3..55d7435cfd 100644
@@ -32,6 +32,7 @@
 
 #define TYPE_LOONGSON_IPI "loongson_ipi"
 OBJECT_DECLARE_SIMPLE_TYPE(LoongsonIPI, LOONGSON_IPI)
+#define TYPE_KVM_LOONGARCH_IPI "loongarch-ipi-kvm"
 
 typedef struct IPICore {
     LoongsonIPI *ipi;
@@ -53,4 +54,25 @@ struct LoongsonIPI {
     IPICore *cpu;
 };
 
+struct KVMLoongArchIPI {
+    SysBusDevice parent_obj;
+    uint32_t num_cpu;
+    IPICore *cpu;
+};
+typedef struct KVMLoongArchIPI KVMLoongArchIPI;
+DECLARE_INSTANCE_CHECKER(KVMLoongArchIPI, KVM_LOONGARCH_IPI,
+                         TYPE_KVM_LOONGARCH_IPI)
+
+struct KVMLoongArchIPIClass {
+    SysBusDeviceClass parent_class;
+    DeviceRealize parent_realize;
+
+    bool is_created;
+    int dev_fd;
+
+};
+typedef struct KVMLoongArchIPIClass KVMLoongArchIPIClass;
+DECLARE_CLASS_CHECKERS(KVMLoongArchIPIClass, KVM_LOONGARCH_IPI,
+                       TYPE_KVM_LOONGARCH_IPI)
+
 #endif
linux-headers/asm-loongarch/kvm.h +1 -0
index f9abef3823..91c6f591fb 100644
@@ -108,4 +108,5 @@ struct kvm_iocsr_entry {
 #define KVM_IRQCHIP_NUM_PINS	64
 #define KVM_MAX_CORES		256
 
+#define KVM_DEV_LOONGARCH_IPI_GRP_REGS		1
 #endif /* __UAPI_ASM_LOONGARCH_KVM_H */
linux-headers/linux/kvm.h +2 -0
index c93876ca0b..3714ff1a94 100644
@@ -1138,6 +1138,8 @@ enum kvm_device_type {
 #define KVM_DEV_TYPE_ARM_PV_TIME	KVM_DEV_TYPE_ARM_PV_TIME
 	KVM_DEV_TYPE_RISCV_AIA,
 #define KVM_DEV_TYPE_RISCV_AIA		KVM_DEV_TYPE_RISCV_AIA
+	KVM_DEV_TYPE_LA_IPI,
+#define KVM_DEV_TYPE_LA_IPI		KVM_DEV_TYPE_LA_IPI
 	KVM_DEV_TYPE_MAX,
 };
 
target/loongarch/kvm/kvm.c +1 -0
index e1be6a6959..c07dcfd85f 100644
@@ -719,6 +719,7 @@ int kvm_arch_get_default_type(MachineState *ms)
 
 int kvm_arch_init(MachineState *ms, KVMState *s)
 {
+    s->kernel_irqchip_allowed = false;
     cap_has_mp_state = kvm_check_extension(s, KVM_CAP_MP_STATE);
     return 0;
 }
Signature · 1 line
2.39.1
#2 lore raw

RFC 2/4 hw/loongarch: Add KVM extioi device support

In reply to Xianglai Li

Added extioi interrupt controller for kvm emulation. The main process is to send the command word for creating an extioi device to the kernel.
When the VM is saved, the ioctl obtains the related data of the extioi interrupt controller in the kernel and saves it. When the VM is recovered, the saved data is sent to the kernel.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>

 hw/intc/Kconfig                    |   4 +
 hw/intc/loongarch_extioi_kvm.c     | 141 +++++++++++++++++++++++++++++
 hw/intc/meson.build                |   1 +
 hw/loongarch/virt.c                |  48 +++++-----
 include/hw/intc/loongarch_extioi.h |  34 ++++++-
 include/hw/loongarch/virt.h        |  15 +++
 linux-headers/asm-loongarch/kvm.h  |   1 +
 linux-headers/linux/kvm.h          |   2 +
 8 files changed, 223 insertions(+), 23 deletions(-)
 create mode 100644 hw/intc/loongarch_extioi_kvm.c
hw/intc/Kconfig +4 -0
index 19d6809129..c672774c6e 100644
@@ -105,3 +105,7 @@ config LOONGARCH_PCH_MSI
 
 config LOONGARCH_EXTIOI
     bool
+
+config LOONGARCH_EXTIOI_KVM
+    bool
+    default y
hw/intc/loongarch_extioi_kvm.c new file +141 -0

141 lines not on this page · raw message

hw/intc/meson.build +1 -0
index 354d43b4f0..3f29b3f5fd 100644
@@ -74,3 +74,4 @@ specific_ss.add(when: 'CONFIG_LOONGARCH_IPI_KVM', if_true: files('loongarch_ipi_
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI_KVM', if_true: files('loongarch_extioi_kvm.c'))
hw/loongarch/virt.c +27 -21

55 lines not on this page · raw message

include/hw/intc/loongarch_extioi.h +32 -2
index eccc2e0d18..9f396f645c 100644
@@ -15,7 +15,7 @@
 #define EXTIOI_IRQS                (256)
 #define EXTIOI_IRQS_BITMAP_SIZE    (256 / 8)
 /* irq from EXTIOI is routed to no more than 4 cpus */
-#define EXTIOI_CPUS                (4)
+#define EXTIOI_CPUS                (256)
 /* map to ipnum per 32 irqs */
 #define EXTIOI_IRQS_IPMAP_SIZE     (256 / 32)
 #define EXTIOI_IRQS_COREMAP_SIZE   256
@@ -59,13 +59,16 @@
 #define EXTIOI_VIRT_COREMAP_START    (0x40)
 #define EXTIOI_VIRT_COREMAP_END      (0x240)
 
+#define EXTIOI_SW_COREMAP_FLAG       (1 << 0)
+
 typedef struct ExtIOICore {
     uint32_t coreisr[EXTIOI_IRQS_GROUP_COUNT];
     DECLARE_BITMAP(sw_isr[LS3A_INTC_IP], EXTIOI_IRQS);
     qemu_irq parent_irq[LS3A_INTC_IP];
 } ExtIOICore;
 
-#define TYPE_LOONGARCH_EXTIOI "loongarch.extioi"
+#define TYPE_LOONGARCH_EXTIOI        "loongarch-extioi"
+#define TYPE_KVM_LOONGARCH_EXTIOI    "loongarch-kvm-extioi"
 OBJECT_DECLARE_SIMPLE_TYPE(LoongArchExtIOI, LOONGARCH_EXTIOI)
 struct LoongArchExtIOI {
     SysBusDevice parent_obj;
@@ -87,4 +90,31 @@ struct LoongArchExtIOI {
     MemoryRegion extioi_system_mem;
     MemoryRegion virt_extend;
 };
+
+struct KVMLoongArchExtIOI {
+    SysBusDevice parent_obj;
+    /* hardware state */
+    uint32_t nodetype[EXTIOI_IRQS_NODETYPE_COUNT / 2];
+    uint32_t bounce[EXTIOI_IRQS_GROUP_COUNT];
+    uint32_t isr[EXTIOI_IRQS / 32];
+    uint32_t coreisr[EXTIOI_CPUS][EXTIOI_IRQS_GROUP_COUNT];
+    uint32_t enable[EXTIOI_IRQS / 32];
+    uint32_t ipmap[EXTIOI_IRQS_IPMAP_SIZE / 4];
+    uint32_t coremap[EXTIOI_IRQS / 4];
+    uint8_t  sw_coremap[EXTIOI_IRQS];
+};
+typedef struct KVMLoongArchExtIOI KVMLoongArchExtIOI;
+DECLARE_INSTANCE_CHECKER(KVMLoongArchExtIOI, KVM_LOONGARCH_EXTIOI,
+                         TYPE_KVM_LOONGARCH_EXTIOI)
+
+struct KVMLoongArchExtIOIClass {
+    SysBusDeviceClass parent_class;
+    DeviceRealize parent_realize;
+
+    bool is_created;
+    int dev_fd;
+};
+typedef struct KVMLoongArchExtIOIClass KVMLoongArchExtIOIClass;
+DECLARE_CLASS_CHECKERS(KVMLoongArchExtIOIClass, KVM_LOONGARCH_EXTIOI,
+                       TYPE_KVM_LOONGARCH_EXTIOI)
 #endif /* LOONGARCH_EXTIOI_H */
include/hw/loongarch/virt.h +15 -0
index 8fdfacf268..3f658356fa 100644
@@ -37,6 +37,21 @@
 
 #define FDT_BASE                0x100000
 
+/* KVM_IRQ_LINE irq field index values */
+#define KVM_LOONGARCH_IRQ_TYPE_SHIFT            24
+#define KVM_LOONGARCH_IRQ_TYPE_MASK             0xff
+#define KVM_LOONGARCH_IRQ_VCPU_SHIFT            16
+#define KVM_LOONGARCH_IRQ_VCPU_MASK             0xff
+#define KVM_LOONGARCH_IRQ_NUM_SHIFT             0
+#define KVM_LOONGARCH_IRQ_NUM_MASK              0xffff
+
+/* irq_type field */
+#define KVM_LOONGARCH_IRQ_TYPE_CPU_IP           0
+#define KVM_LOONGARCH_IRQ_TYPE_CPU_IO           1
+#define KVM_LOONGARCH_IRQ_TYPE_HT               2
+#define KVM_LOONGARCH_IRQ_TYPE_MSI              3
+#define KVM_LOONGARCH_IRQ_TYPE_IOAPIC           4
+
 struct LoongArchVirtMachineState {
     /*< private >*/
     MachineState parent_obj;
linux-headers/asm-loongarch/kvm.h +1 -0
index 91c6f591fb..c71221d1b0 100644
@@ -109,4 +109,5 @@ struct kvm_iocsr_entry {
 #define KVM_MAX_CORES		256
 
 #define KVM_DEV_LOONGARCH_IPI_GRP_REGS		1
+#define KVM_DEV_LOONGARCH_EXTIOI_GRP_REGS	1
 #endif /* __UAPI_ASM_LOONGARCH_KVM_H */
linux-headers/linux/kvm.h +2 -0
index 3714ff1a94..1dca283ecd 100644
@@ -1140,6 +1140,8 @@ enum kvm_device_type {
 #define KVM_DEV_TYPE_RISCV_AIA		KVM_DEV_TYPE_RISCV_AIA
 	KVM_DEV_TYPE_LA_IPI,
 #define KVM_DEV_TYPE_LA_IPI		KVM_DEV_TYPE_LA_IPI
+	KVM_DEV_TYPE_LA_EXTIOI,
+#define KVM_DEV_TYPE_LA_EXTIOI		KVM_DEV_TYPE_LA_EXTIOI
 	KVM_DEV_TYPE_MAX,
 };
 
Signature · 1 line
2.39.1
#3 lore raw

RFC 3/4 hw/loongarch: Add KVM pch pic device support

In reply to Xianglai Li

Added pch_pic interrupt controller for kvm emulation. The main process is to send the command word for creating an pch_pic device to the kernel,
Delivers the pch pic interrupt controller configuration register base address to the kernel.
When the VM is saved, the ioctl obtains the pch_pic interrupt controller data in the kernel and saves it. When the VM is recovered, the saved data is sent to the kernel.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>

 hw/intc/Kconfig                     |   4 +
 hw/intc/loongarch_pch_pic.c         |  20 ++-
 hw/intc/loongarch_pch_pic_kvm.c     | 189 ++++++++++++++++++++++++++++
 hw/intc/meson.build                 |   1 +
 hw/loongarch/virt.c                 |  65 +++++-----
 include/hw/intc/loongarch_pch_pic.h |  51 +++++++-
 linux-headers/asm-loongarch/kvm.h   |   5 +
 linux-headers/linux/kvm.h           |   2 +
 8 files changed, 304 insertions(+), 33 deletions(-)
 create mode 100644 hw/intc/loongarch_pch_pic_kvm.c
hw/intc/Kconfig +4 -0
index c672774c6e..c9f96bffd6 100644
@@ -98,6 +98,10 @@ config LOONGARCH_PCH_PIC
     bool
     select UNIMP
 
+config LOONGARCH_PCH_PIC_KVM
+    bool
+    default y
+
 config LOONGARCH_PCH_MSI
     select MSI_NONBROKEN
     bool
hw/intc/loongarch_pch_pic.c +18 -2
index 2d5e65abff..b64e3db77f 100644
@@ -16,18 +16,27 @@
 #include "migration/vmstate.h"
 #include "trace.h"
 #include "qapi/error.h"
+#include "sysemu/kvm.h"
 
 static void pch_pic_update_irq(LoongArchPCHPIC *s, uint64_t mask, int level)
 {
     uint64_t val;
     int irq;
+    int kvm_irq;
 
     if (level) {
         val = mask & s->intirr & ~s->int_mask;
         if (val) {
             irq = ctz64(val);
             s->intisr |= MAKE_64BIT_MASK(irq, 1);
-            qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 1);
+            if (kvm_kernel_irqchip_allowed()) {
+                kvm_irq = (
+                KVM_LOONGARCH_IRQ_TYPE_IOAPIC << KVM_LOONGARCH_IRQ_TYPE_SHIFT)
+                | (0 <<  KVM_LOONGARCH_IRQ_VCPU_SHIFT) | s->htmsi_vector[irq];
+                kvm_set_irq(kvm_state, kvm_irq, !!level);
+            } else {
+                qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 1);
+            }
         }
     } else {
         /*
@@ -38,7 +47,14 @@ static void pch_pic_update_irq(LoongArchPCHPIC *s, uint64_t mask, int level)
         if (val) {
             irq = ctz64(val);
             s->intisr &= ~MAKE_64BIT_MASK(irq, 1);
-            qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 0);
+            if (kvm_kernel_irqchip_allowed()) {
+                kvm_irq = (
+                KVM_LOONGARCH_IRQ_TYPE_IOAPIC << KVM_LOONGARCH_IRQ_TYPE_SHIFT)
+                | (0 <<  KVM_LOONGARCH_IRQ_VCPU_SHIFT) | s->htmsi_vector[irq];
+                kvm_set_irq(kvm_state, kvm_irq, !!level);
+            } else {
+                qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 0);
+            }
         }
     }
 }
hw/intc/loongarch_pch_pic_kvm.c new file +189 -0

189 lines not on this page · raw message

hw/intc/meson.build +1 -0
index 3f29b3f5fd..4fddc80383 100644
@@ -75,3 +75,4 @@ specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI_KVM', if_true: files('loongarch_extioi_kvm.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC_KVM', if_true: files('loongarch_pch_pic_kvm.c'))
hw/loongarch/virt.c +35 -30

75 lines not on this page · raw message

include/hw/intc/loongarch_pch_pic.h +50 -1

66 lines not on this page · raw message

linux-headers/asm-loongarch/kvm.h +5 -0
index c71221d1b0..cd0aebd805 100644
@@ -110,4 +110,9 @@ struct kvm_iocsr_entry {
 
 #define KVM_DEV_LOONGARCH_IPI_GRP_REGS		1
 #define KVM_DEV_LOONGARCH_EXTIOI_GRP_REGS	1
+
+#define KVM_DEV_LOONGARCH_PCH_PIC_GRP_CTRL	0
+#define KVM_DEV_LOONGARCH_PCH_PIC_CTRL_INIT	0
+
+#define KVM_DEV_LOONGARCH_PCH_PIC_GRP_REGS	1
 #endif /* __UAPI_ASM_LOONGARCH_KVM_H */
linux-headers/linux/kvm.h +2 -0
index 1dca283ecd..48eaeb91f2 100644
@@ -1138,6 +1138,8 @@ enum kvm_device_type {
 #define KVM_DEV_TYPE_ARM_PV_TIME	KVM_DEV_TYPE_ARM_PV_TIME
 	KVM_DEV_TYPE_RISCV_AIA,
 #define KVM_DEV_TYPE_RISCV_AIA		KVM_DEV_TYPE_RISCV_AIA
+	KVM_DEV_TYPE_LA_PCH_PIC,
+#define KVM_DEV_TYPE_LA_PCH_PIC		KVM_DEV_TYPE_LA_PCH_PIC
 	KVM_DEV_TYPE_LA_IPI,
 #define KVM_DEV_TYPE_LA_IPI		KVM_DEV_TYPE_LA_IPI
 	KVM_DEV_TYPE_LA_EXTIOI,
Signature · 1 line
2.39.1
#4 lore raw

RFC 4/4 hw/loongarch: Add KVM pch msi device support

In reply to Xianglai Li

Added pch_msi interrupt controller handling during kernel emulation of irq chip.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>

 hw/intc/loongarch_pch_msi.c         | 42 +++++++++++++++++++++--------
 hw/loongarch/virt.c                 | 34 +++++++++++++++--------
 include/hw/intc/loongarch_pch_msi.h |  2 +-
 target/loongarch/kvm/kvm.c          |  1 -
 4 files changed, 55 insertions(+), 24 deletions(-)
hw/intc/loongarch_pch_msi.c +31 -11
index ecf3ed0267..c0a8b3a547 100644
@@ -2,7 +2,7 @@
 /*
  * QEMU Loongson 7A1000 msi interrupt controller.
  *
- * Copyright (C) 2021 Loongson Technology Corporation Limited
+ * Copyright (C) 2024 Loongson Technology Corporation Limited
  */
 
 #include "qemu/osdep.h"
@@ -14,6 +14,8 @@
 #include "hw/misc/unimp.h"
 #include "migration/vmstate.h"
 #include "trace.h"
+#include "sysemu/kvm.h"
+#include "hw/loongarch/virt.h"
 
 static uint64_t loongarch_msi_mem_read(void *opaque, hwaddr addr, unsigned size)
 {
@@ -26,14 +28,24 @@ static void loongarch_msi_mem_write(void *opaque, hwaddr addr,
     LoongArchPCHMSI *s = (LoongArchPCHMSI *)opaque;
     int irq_num;
 
-    /*
-     * vector number is irq number from upper extioi intc
-     * need subtract irq base to get msi vector offset
-     */
-    irq_num = (val & 0xff) - s->irq_base;
-    trace_loongarch_msi_set_irq(irq_num);
-    assert(irq_num < s->irq_num);
-    qemu_set_irq(s->pch_msi_irq[irq_num], 1);
+    MSIMessage msg = {
+        .address = addr,
+        .data = val,
+    };
+
+    if (kvm_kernel_irqchip_allowed()) {
+        kvm_irqchip_send_msi(kvm_state, msg);
+    } else {
+        /*
+         * vector number is irq number from upper extioi intc
+         * need subtract irq base to get msi vector offset
+         */
+        irq_num = (val & 0xff) - s->irq_base;
+        trace_loongarch_msi_set_irq(irq_num);
+        assert(irq_num < s->irq_num);
+
+        qemu_set_irq(s->pch_msi_irq[irq_num], 1);
+    }
 }
 
 static const MemoryRegionOps loongarch_pch_msi_ops = {
@@ -45,8 +57,16 @@ static const MemoryRegionOps loongarch_pch_msi_ops = {
 static void pch_msi_irq_handler(void *opaque, int irq, int level)
 {
     LoongArchPCHMSI *s = LOONGARCH_PCH_MSI(opaque);
-
-    qemu_set_irq(s->pch_msi_irq[irq], level);
+    MSIMessage msg = {
+        .address = 0,
+        .data = irq,
+    };
+
+    if (kvm_kernel_irqchip_allowed()) {
+        kvm_irqchip_send_msi(kvm_state, msg);
+    } else {
+        qemu_set_irq(s->pch_msi_irq[irq], level);
+    }
 }
 
 static void loongarch_pch_msi_realize(DeviceState *dev, Error **errp)
hw/loongarch/virt.c +23 -11
index 9e9e8f5d14..750a60ba91 100644
@@ -884,24 +884,36 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
         for (i = 0; i < num; i++) {
             qdev_connect_gpio_out(DEVICE(d), i, qdev_get_gpio_in(extioi, i));
         }
+    }
 
-        /* Add PCH PIC node */
-        fdt_add_pch_pic_node(lvms, &eiointc_phandle, &pch_pic_phandle);
+    /* Add PCH PIC node */
+    fdt_add_pch_pic_node(lvms, &eiointc_phandle, &pch_pic_phandle);
 
-        pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
-        start   =  num;
-        num = EXTIOI_IRQS - start;
-        qdev_prop_set_uint32(pch_msi, "msi_irq_base", start);
-        qdev_prop_set_uint32(pch_msi, "msi_irq_num", num);
-        d = SYS_BUS_DEVICE(pch_msi);
-        sysbus_realize_and_unref(d, &error_fatal);
-        sysbus_mmio_map(d, 0, VIRT_PCH_MSI_ADDR_LOW);
+    pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
+    num = VIRT_PCH_PIC_IRQ_NUM;
+    start   =  num;
+    num = EXTIOI_IRQS - start;
+    qdev_prop_set_uint32(pch_msi, "msi_irq_base", start);
+    qdev_prop_set_uint32(pch_msi, "msi_irq_num", num);
+    d = SYS_BUS_DEVICE(pch_msi);
+    sysbus_realize_and_unref(d, &error_fatal);
+
+    if (kvm_kernel_irqchip_allowed()) {
+        if (kvm_has_gsi_routing()) {
+            for (i = 0; i < 64; ++i) {
+                kvm_irqchip_add_irq_route(kvm_state, i, 0, i);
+            }
+            kvm_gsi_routing_allowed = true;
+        }
+        kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
+    } else {
+        /* Connect pch_msi irqs to extioi */
         for (i = 0; i < num; i++) {
-            /* Connect pch_msi irqs to extioi */
             qdev_connect_gpio_out(DEVICE(d), i,
                                   qdev_get_gpio_in(extioi, i + start));
         }
     }
+    sysbus_mmio_map(d, 0, VIRT_PCH_MSI_ADDR_LOW);
 
     /* Add PCH MSI node */
     fdt_add_pch_msi_node(lvms, &eiointc_phandle, &pch_msi_phandle);
include/hw/intc/loongarch_pch_msi.h +1 -1
index b8586fb3b6..fd4ea97a83 100644
@@ -7,7 +7,7 @@
 
 #include "hw/sysbus.h"
 
-#define TYPE_LOONGARCH_PCH_MSI "loongarch_pch_msi"
+#define TYPE_LOONGARCH_PCH_MSI     "loongarch_pch_msi"
 OBJECT_DECLARE_SIMPLE_TYPE(LoongArchPCHMSI, LOONGARCH_PCH_MSI)
 
 /* MSI irq start from 32 to 255 */
target/loongarch/kvm/kvm.c +0 -1
index c07dcfd85f..e1be6a6959 100644
@@ -719,7 +719,6 @@ int kvm_arch_get_default_type(MachineState *ms)
 
 int kvm_arch_init(MachineState *ms, KVMState *s)
 {
-    s->kernel_irqchip_allowed = false;
     cap_has_mp_state = kvm_check_extension(s, KVM_CAP_MP_STATE);
     return 0;
 }
Signature · 1 line
2.39.1
#5 lore raw
In reply to Xianglai Li
Quoted 56 lines · Xianglai Li

Before this, the interrupt controller simulation has been completed in the user mode program. In order to reduce the loss caused by frequent switching of the virtual machine monitor from kernel mode to user mode when the guest accesses the interrupt controller, we add the interrupt controller simulation in kvm.

In qemu side implementation is simple, just make a new IPI EXTIOI PCH KVM related several classes, And the interface to access kvm related data is implemented.

Most of the simulation work of the interrupt controller is done in kvm. Because KVM the changes have not been the Linux community acceptance, the patches of this series will have RFC label until KVM patch into the community.

For the implementation of kvm simulation, refer to the following documents.

IPI simulation implementation reference:
https://github.com/loongson/LoongArch-Documentation/tree/main/docs/Loongson-3A5000-usermanual-EN/inter-processor-interrupts-and-communication

EXTIOI simulation implementation reference:
https://github.com/loongson/LoongArch-Documentation/tree/main/docs/Loongson-3A5000-usermanual-EN/io-interrupts/extended-io-interrupts

PCH-PIC simulation implementation reference:
https://github.com/loongson/LoongArch-Documentation/blob/main/docs/Loongson-7A1000-usermanual-EN/interrupt-controller.adoc

For PCH-MSI, we used irqfd mechanism to send the interrupt signal generated by user state to kernel state and then to EXTIOI without maintaining PCH-MSI state in kernel state.

You can easily get the code from the link below:
the kernel:
https://github.com/lixianglai/linux
the branch is: interrupt

the qemu:
https://github.com/lixianglai/qemu
the branch is: interrupt

Please note that the code above is regularly updated based on community reviews.

Cc Paolo Bonzini <[email protected]>
Cc Song Gao <[email protected]>
Cc Huacai Chen <[email protected]>
Cc Jiaxun Yang <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Cornelia Huck <[email protected]>
Cc Bibo Mao <[email protected]>

Xianglai Li (4):
hw/loongarch: Add KVM IPI device support
hw/loongarch: Add KVM extioi device support
hw/loongarch: Add KVM pch pic device support
hw/loongarch: Add KVM pch msi device support

Quoted 15 lines
 hw/intc/Kconfig                     |  12 ++
 hw/intc/loongarch_extioi_kvm.c      | 141 +++++++++++++++++++
 hw/intc/loongarch_ipi_kvm.c         | 207 ++++++++++++++++++++++++++++
 hw/intc/loongarch_pch_msi.c         |  42 ++++--
 hw/intc/loongarch_pch_pic.c         |  20 ++-
 hw/intc/loongarch_pch_pic_kvm.c     | 189 +++++++++++++++++++++++++
 hw/intc/meson.build                 |   3 +
 hw/loongarch/virt.c                 | 141 ++++++++++++-------
 include/hw/intc/loongarch_extioi.h  |  34 ++++-
 include/hw/intc/loongarch_pch_msi.h |   2 +-
 include/hw/intc/loongarch_pch_pic.h |  51 ++++++-
 include/hw/intc/loongson_ipi.h      |  22 +++
 include/hw/loongarch/virt.h         |  15 ++
 linux-headers/asm-loongarch/kvm.h   |   7 +
 linux-headers/linux/kvm.h           |   6 +

Please split out any headers changes into a separate patch -- just put them into a placeholder patch at the beginning of the series as long as the changes are not yet upstream (and replace that with a full headers sync later.)

Quoted 4 lines
 15 files changed, 823 insertions(+), 69 deletions(-)
 create mode 100644 hw/intc/loongarch_extioi_kvm.c
 create mode 100644 hw/intc/loongarch_ipi_kvm.c
 create mode 100644 hw/intc/loongarch_pch_pic_kvm.c