Skip to contentSkip to conversation

Added Interrupt controller emulation for loongarch kvm

RFC PATCH Superseded Xianglai Li 4 participants 12 messages Sep 10, 2024 – Sep 14, 2024

Subsystems (2)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover Added Interrupt controller emulation for loongarch kvm

Before this, the interrupt controller simulation has been completed in the user mode program. In order to reduce the loss caused by frequent switching of the virtual machine monitor from kernel mode to user mode when the guest accesses the interrupt controller, we add the interrupt controller simulation in kvm.

In qemu side implementation is simple, just make a new IPI EXTIOI PCH KVM related several classes, And the interface to access kvm related data is implemented.

Most of the simulation work of the interrupt controller is done in kvm. Because KVM the changes have not been the Linux community acceptance, the patches of this series will have RFC label until KVM patch into the community.

For the implementation of kvm simulation, refer to the following documents.

IPI simulation implementation reference:
https://github.com/loongson/LoongArch-Documentation/tree/main/docs/Loongson-3A5000-usermanual-EN/inter-processor-interrupts-and-communication

EXTIOI simulation implementation reference:
https://github.com/loongson/LoongArch-Documentation/tree/main/docs/Loongson-3A5000-usermanual-EN/io-interrupts/extended-io-interrupts

PCH-PIC simulation implementation reference:
https://github.com/loongson/LoongArch-Documentation/blob/main/docs/Loongson-7A1000-usermanual-EN/interrupt-controller.adoc

For PCH-MSI, we used irqfd mechanism to send the interrupt signal generated by user state to kernel state and then to EXTIOI without maintaining PCH-MSI state in kernel state.

You can easily get the code from the link below:
the kernel:
https://github.com/lixianglai/linux
the branch is: interrupt

the qemu:
https://github.com/lixianglai/qemu
the branch is: interrupt

Please note that the code above is regularly updated based on community reviews.

changelog:
V1->V2
1.Make changes involving header files a separate patch 2.rebase based on the latest qemu code
3.Optimize the kvm ipi class into the ipi common class and the ipi kvm class.
4.Optimized the interface for kernel reading and writing data on ipi extioi pch_pic device.

Cc Paolo Bonzini <[email protected]>
Cc Song Gao <[email protected]>
Cc Jiaxun Yang <[email protected]>
Cc Huacai Chen <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Cornelia Huck <[email protected]>
Cc Bibo Mao <[email protected]>
Cc Xianglai Li <[email protected]>

Xianglai Li (5):
include: Add macro definitions needed for interrupt controller kvm
emulation
hw/loongarch: Add KVM IPI device support
hw/loongarch: Add KVM extioi device support hw/loongarch: Add KVM pch pic device support hw/loongarch: Add KVM pch msi device support

 hw/intc/Kconfig                       |   9 +
 hw/intc/loongarch_extioi_kvm.c        | 250 ++++++++++++++++++++++++++
 hw/intc/loongarch_ipi_kvm.c           | 128 +++++++++++++
 hw/intc/loongarch_pch_msi.c           |  42 +++--
 hw/intc/loongarch_pch_pic.c           |  40 +++--
 hw/intc/loongarch_pch_pic_kvm.c       | 180 +++++++++++++++++++
 hw/intc/loongson_ipi_common.c         |  28 +++
 hw/intc/meson.build                   |   3 +
 hw/loongarch/Kconfig                  |   3 +
 hw/loongarch/virt.c                   | 138 ++++++++------
 include/hw/intc/loongarch_extioi.h    |  38 +++-
 include/hw/intc/loongarch_ipi.h       |  15 ++
 include/hw/intc/loongarch_pch_pic.h   |  58 +++++-
 include/hw/intc/loongson_ipi.h        |   1 -
 include/hw/intc/loongson_ipi_common.h |   2 +
 include/hw/loongarch/virt.h           |  15 ++
 linux-headers/asm-loongarch/kvm.h     |  18 ++
 linux-headers/linux/kvm.h             |   6 +
 18 files changed, 891 insertions(+), 83 deletions(-)
 create mode 100644 hw/intc/loongarch_extioi_kvm.c
 create mode 100644 hw/intc/loongarch_ipi_kvm.c
 create mode 100644 hw/intc/loongarch_pch_pic_kvm.c
Signature · 1 line
2.39.1
#1 lore raw

RFC PATCH v2 1/5 include: Add macro definitions needed for interrupt controller kvm emulation

Add macro definitions needed for interrupt controller kvm emulation.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

 include/hw/intc/loongarch_extioi.h    | 38 ++++++++++++++++--
 include/hw/intc/loongarch_ipi.h       | 15 +++++++
 include/hw/intc/loongarch_pch_pic.h   | 58 +++++++++++++++++++++++++--
 include/hw/intc/loongson_ipi.h        |  1 -
 include/hw/intc/loongson_ipi_common.h |  2 +
 include/hw/loongarch/virt.h           | 15 +++++++
 linux-headers/asm-loongarch/kvm.h     | 18 +++++++++
 linux-headers/linux/kvm.h             |  6 +++
 8 files changed, 146 insertions(+), 7 deletions(-)
include/hw/intc/loongarch_extioi.h +35 -3
index 626a37dfa1..97f6aa4d60 100644
@@ -15,7 +15,7 @@
 #define EXTIOI_IRQS                (256)
 #define EXTIOI_IRQS_BITMAP_SIZE    (256 / 8)
 /* irq from EXTIOI is routed to no more than 4 cpus */
-#define EXTIOI_CPUS                (4)
+#define EXTIOI_CPUS                (256)
 /* map to ipnum per 32 irqs */
 #define EXTIOI_IRQS_IPMAP_SIZE     (256 / 32)
 #define EXTIOI_IRQS_COREMAP_SIZE   256
@@ -36,7 +36,7 @@
 #define EXTIOI_ISR_START             (0x700 - APIC_OFFSET)
 #define EXTIOI_ISR_END               (0x720 - APIC_OFFSET)
 #define EXTIOI_COREISR_START         (0x800 - APIC_OFFSET)
-#define EXTIOI_COREISR_END           (0xB20 - APIC_OFFSET)
+#define EXTIOI_COREISR_END           (0x820 - APIC_OFFSET)
 #define EXTIOI_COREMAP_START         (0xC00 - APIC_OFFSET)
 #define EXTIOI_COREMAP_END           (0xD00 - APIC_OFFSET)
 #define EXTIOI_SIZE                  0x800
@@ -64,7 +64,8 @@ typedef struct ExtIOICore {
     qemu_irq parent_irq[LS3A_INTC_IP];
 } ExtIOICore;
 
-#define TYPE_LOONGARCH_EXTIOI "loongarch.extioi"
+#define TYPE_LOONGARCH_EXTIOI        "loongarch-extioi"
+#define TYPE_KVM_LOONGARCH_EXTIOI    "loongarch-kvm-extioi"
 OBJECT_DECLARE_SIMPLE_TYPE(LoongArchExtIOI, LOONGARCH_EXTIOI)
 struct LoongArchExtIOI {
     SysBusDevice parent_obj;
@@ -86,4 +87,35 @@ struct LoongArchExtIOI {
     MemoryRegion extioi_system_mem;
     MemoryRegion virt_extend;
 };
+
+struct KVMLoongArchExtIOI {
+    SysBusDevice parent_obj;
+    uint32_t num_cpu;
+    uint32_t features;
+    uint32_t status;
+
+    /* hardware state */
+    uint32_t nodetype[EXTIOI_IRQS_NODETYPE_COUNT / 2];
+    uint32_t bounce[EXTIOI_IRQS_GROUP_COUNT];
+    uint32_t isr[EXTIOI_IRQS / 32];
+    uint32_t coreisr[EXTIOI_CPUS][EXTIOI_IRQS_GROUP_COUNT];
+    uint32_t enable[EXTIOI_IRQS / 32];
+    uint32_t ipmap[EXTIOI_IRQS_IPMAP_SIZE / 4];
+    uint32_t coremap[EXTIOI_IRQS / 4];
+    uint8_t  sw_coremap[EXTIOI_IRQS];
+};
+typedef struct KVMLoongArchExtIOI KVMLoongArchExtIOI;
+DECLARE_INSTANCE_CHECKER(KVMLoongArchExtIOI, KVM_LOONGARCH_EXTIOI,
+                         TYPE_KVM_LOONGARCH_EXTIOI)
+
+struct KVMLoongArchExtIOIClass {
+    SysBusDeviceClass parent_class;
+    DeviceRealize parent_realize;
+
+    bool is_created;
+    int dev_fd;
+};
+typedef struct KVMLoongArchExtIOIClass KVMLoongArchExtIOIClass;
+DECLARE_CLASS_CHECKERS(KVMLoongArchExtIOIClass, KVM_LOONGARCH_EXTIOI,
+                       TYPE_KVM_LOONGARCH_EXTIOI)
 #endif /* LOONGARCH_EXTIOI_H */
include/hw/intc/loongarch_ipi.h +15 -0
index 276b3040a3..64ebbdcba6 100644
@@ -22,4 +22,19 @@ struct LoongarchIPIClass {
     LoongsonIPICommonClass parent_class;
 };
 
+#define TYPE_KVM_LOONGARCH_IPI  "kvm_loongarch_ipi"
+
+OBJECT_DECLARE_TYPE(KVMLoongarchIPIState,
+                    KVMLoongArchIPIClass, KVM_LOONGARCH_IPI)
+
+struct KVMLoongarchIPIState {
+    LoongsonIPICommonState parent_obj;
+    int dev_fd;
+};
+
+struct KVMLoongArchIPIClass {
+   LoongsonIPICommonClass parent_class;
+   DeviceRealize parent_realize;
+    DeviceUnrealize parent_unrealize;
+};
 #endif
include/hw/intc/loongarch_pch_pic.h +55 -3

82 lines not on this page · raw message

include/hw/intc/loongson_ipi.h +0 -1
index 4e517cc8dc..57e0d94e2b 100644
@@ -27,5 +27,4 @@ struct LoongsonIPIState {
 
     MemoryRegion *ipi_mmio_mem;
 };
-
 #endif
include/hw/intc/loongson_ipi_common.h +2 -0
index df9d9c5168..adba8ffd49 100644
@@ -45,6 +45,8 @@ struct LoongsonIPICommonClass {
     DeviceUnrealize parent_unrealize;
     AddressSpace *(*get_iocsr_as)(CPUState *cpu);
     CPUState *(*cpu_by_arch_id)(int64_t id);
+    void (*pre_save)(LoongsonIPICommonState *s);
+    void (*post_load)(LoongsonIPICommonState *s, int version_id);
 };
 
 MemTxResult loongson_ipi_core_readl(void *opaque, hwaddr addr, uint64_t *data,
include/hw/loongarch/virt.h +15 -0
index c373e48f27..c1769fc20c 100644
@@ -36,6 +36,21 @@
 
 #define FDT_BASE                0x100000
 
+/* KVM_IRQ_LINE irq field index values */
+#define KVM_LOONGARCH_IRQ_TYPE_SHIFT            24
+#define KVM_LOONGARCH_IRQ_TYPE_MASK             0xff
+#define KVM_LOONGARCH_IRQ_VCPU_SHIFT            16
+#define KVM_LOONGARCH_IRQ_VCPU_MASK             0xff
+#define KVM_LOONGARCH_IRQ_NUM_SHIFT             0
+#define KVM_LOONGARCH_IRQ_NUM_MASK              0xffff
+
+/* irq_type field */
+#define KVM_LOONGARCH_IRQ_TYPE_CPU_IP           0
+#define KVM_LOONGARCH_IRQ_TYPE_CPU_IO           1
+#define KVM_LOONGARCH_IRQ_TYPE_HT               2
+#define KVM_LOONGARCH_IRQ_TYPE_MSI              3
+#define KVM_LOONGARCH_IRQ_TYPE_IOAPIC           4
+
 struct LoongArchVirtMachineState {
     /*< private >*/
     MachineState parent_obj;
linux-headers/asm-loongarch/kvm.h +18 -0
index f9abef3823..e102d500a3 100644
@@ -108,4 +108,22 @@ struct kvm_iocsr_entry {
 #define KVM_IRQCHIP_NUM_PINS	64
 #define KVM_MAX_CORES		256
 
+#define KVM_DEV_LOONGARCH_IPI_GRP_REGS		        0x40000001
+
+#define KVM_DEV_LOONGARCH_EXTIOI_GRP_REGS	        0x40000002
+
+#define KVM_DEV_LOONGARCH_EXTIOI_GRP_SW_STATUS	        0x40000003
+#define KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_NUM_CPU   	0x0
+#define KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_FEATURE   	0x1
+#define KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_STATE   	0x2
+
+#define KVM_DEV_LOONGARCH_EXTIOI_GRP_CTRL	        0x40000004
+#define KVM_DEV_LOONGARCH_EXTIOI_CTRL_INIT_NUM_CPU   	0x0
+#define KVM_DEV_LOONGARCH_EXTIOI_CTRL_INIT_FEATURE 	0x1
+#define KVM_DEV_LOONGARCH_EXTIOI_CTRL_LOAD_FINISHED   	0x3
+
+#define KVM_DEV_LOONGARCH_PCH_PIC_GRP_REGS	        0x40000005
+#define KVM_DEV_LOONGARCH_PCH_PIC_GRP_CTRL	        0x40000006
+#define KVM_DEV_LOONGARCH_PCH_PIC_CTRL_INIT	        0
+
 #endif /* __UAPI_ASM_LOONGARCH_KVM_H */
linux-headers/linux/kvm.h +6 -0
index c93876ca0b..cd2aed735c 100644
@@ -1138,6 +1138,12 @@ enum kvm_device_type {
 #define KVM_DEV_TYPE_ARM_PV_TIME	KVM_DEV_TYPE_ARM_PV_TIME
 	KVM_DEV_TYPE_RISCV_AIA,
 #define KVM_DEV_TYPE_RISCV_AIA		KVM_DEV_TYPE_RISCV_AIA
+	KVM_DEV_TYPE_LOONGARCH_PCH_PIC,
+#define KVM_DEV_TYPE_LOONGARCH_PCH_PIC  KVM_DEV_TYPE_LOONGARCH_PCH_PIC
+	KVM_DEV_TYPE_LOONGARCH_IPI,
+#define KVM_DEV_TYPE_LOONGARCH_IPI      KVM_DEV_TYPE_LOONGARCH_IPI
+	KVM_DEV_TYPE_LOONGARCH_EXTIOI,
+#define KVM_DEV_TYPE_LOONGARCH_EXTIOI   KVM_DEV_TYPE_LOONGARCH_EXTIOI
 	KVM_DEV_TYPE_MAX,
 };
 
Signature · 1 line
2.39.1
#2 lore raw

Re: [RFC PATCH V2 1/5] include: Add macro definitions needed for interrupt controller kvm emulation

Quoted 14 lines · Xianglai Li

Add macro definitions needed for interrupt controller kvm emulation.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

Quoted 9 lines
 include/hw/intc/loongarch_extioi.h    | 38 ++++++++++++++++--
 include/hw/intc/loongarch_ipi.h       | 15 +++++++
 include/hw/intc/loongarch_pch_pic.h   | 58 +++++++++++++++++++++++++--
 include/hw/intc/loongson_ipi.h        |  1 -
 include/hw/intc/loongson_ipi_common.h |  2 +
 include/hw/loongarch/virt.h           | 15 +++++++
 linux-headers/asm-loongarch/kvm.h     | 18 +++++++++
 linux-headers/linux/kvm.h             |  6 +++
 8 files changed, 146 insertions(+), 7 deletions(-)

The parts you need to split out into a separate patch are the changes under linux-headers/ (because they get updated via a script); the changes under include/hw/ are internal to QEMU and should go where it makes sense (probably with the actual changes in .c files, but I didn't check what the patch actually does.)

author
#3 lore raw

Re: [RFC PATCH V2 1/5] include: Add macro definitions needed for interrupt controller kvm emulation

Hi Cornelia Huck:

Quoted 25 lines · Xianglai Li
Quoted 15 lines

Add macro definitions needed for interrupt controller kvm emulation.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

Quoted 9 lines
  include/hw/intc/loongarch_extioi.h    | 38 ++++++++++++++++--
  include/hw/intc/loongarch_ipi.h       | 15 +++++++
  include/hw/intc/loongarch_pch_pic.h   | 58 +++++++++++++++++++++++++--
  include/hw/intc/loongson_ipi.h        |  1 -
  include/hw/intc/loongson_ipi_common.h |  2 +
  include/hw/loongarch/virt.h           | 15 +++++++
  linux-headers/asm-loongarch/kvm.h     | 18 +++++++++
  linux-headers/linux/kvm.h             |  6 +++
  8 files changed, 146 insertions(+), 7 deletions(-)
Quoted 5 lines

The parts you need to split out into a separate patch are the changes under linux-headers/ (because they get updated via a script); the changes under include/hw/ are internal to QEMU and should go where it makes sense (probably with the actual changes in .c files, but I didn't check what the patch actually does.)

Ok, I'll correct it in the next version.
Thanks!
Xianglai.

#4 lore raw

Re: [RFC PATCH V2 1/5] include: Add macro definitions needed for interrupt controller kvm emulation

In reply to Xianglai Li
Quoted 1 line · 在2024年9月10日九月 下午1:18,Xianglai Li写道:

Add macro definitions needed for interrupt controller kvm emulation.

Hi Xianglai and other LoongArch folks,

As I can see your recent works had enabled IOCSR to be treated like a address space, should we migrate IOCSR stuff to iofd/eventfd infra?

Thanks
- Jiaxun

Quoted 304 lines

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

 include/hw/intc/loongarch_extioi.h    | 38 ++++++++++++++++--
 include/hw/intc/loongarch_ipi.h       | 15 +++++++
 include/hw/intc/loongarch_pch_pic.h   | 58 +++++++++++++++++++++++++--
 include/hw/intc/loongson_ipi.h        |  1 -
 include/hw/intc/loongson_ipi_common.h |  2 +
 include/hw/loongarch/virt.h           | 15 +++++++
 linux-headers/asm-loongarch/kvm.h     | 18 +++++++++
 linux-headers/linux/kvm.h             |  6 +++
 8 files changed, 146 insertions(+), 7 deletions(-)
include/hw/intc/loongarch_extioi.h +35 -3
index 626a37dfa1..97f6aa4d60 100644
@@ -15,7 +15,7 @@
 #define EXTIOI_IRQS                (256)
 #define EXTIOI_IRQS_BITMAP_SIZE    (256 / 8)
 /* irq from EXTIOI is routed to no more than 4 cpus */
-#define EXTIOI_CPUS                (4)
+#define EXTIOI_CPUS                (256)
 /* map to ipnum per 32 irqs */
 #define EXTIOI_IRQS_IPMAP_SIZE     (256 / 32)
 #define EXTIOI_IRQS_COREMAP_SIZE   256
@@ -36,7 +36,7 @@
 #define EXTIOI_ISR_START             (0x700 - APIC_OFFSET)
 #define EXTIOI_ISR_END               (0x720 - APIC_OFFSET)
 #define EXTIOI_COREISR_START         (0x800 - APIC_OFFSET)
-#define EXTIOI_COREISR_END           (0xB20 - APIC_OFFSET)
+#define EXTIOI_COREISR_END           (0x820 - APIC_OFFSET)
 #define EXTIOI_COREMAP_START         (0xC00 - APIC_OFFSET)
 #define EXTIOI_COREMAP_END           (0xD00 - APIC_OFFSET)
 #define EXTIOI_SIZE                  0x800
@@ -64,7 +64,8 @@ typedef struct ExtIOICore {
     qemu_irq parent_irq[LS3A_INTC_IP];
 } ExtIOICore;
 
-#define TYPE_LOONGARCH_EXTIOI "loongarch.extioi"
+#define TYPE_LOONGARCH_EXTIOI        "loongarch-extioi"
+#define TYPE_KVM_LOONGARCH_EXTIOI    "loongarch-kvm-extioi"
 OBJECT_DECLARE_SIMPLE_TYPE(LoongArchExtIOI, LOONGARCH_EXTIOI)
 struct LoongArchExtIOI {
     SysBusDevice parent_obj;
@@ -86,4 +87,35 @@ struct LoongArchExtIOI {
     MemoryRegion extioi_system_mem;
     MemoryRegion virt_extend;
 };
+
+struct KVMLoongArchExtIOI {
+    SysBusDevice parent_obj;
+    uint32_t num_cpu;
+    uint32_t features;
+    uint32_t status;
+
+    /* hardware state */
+    uint32_t nodetype[EXTIOI_IRQS_NODETYPE_COUNT / 2];
+    uint32_t bounce[EXTIOI_IRQS_GROUP_COUNT];
+    uint32_t isr[EXTIOI_IRQS / 32];
+    uint32_t coreisr[EXTIOI_CPUS][EXTIOI_IRQS_GROUP_COUNT];
+    uint32_t enable[EXTIOI_IRQS / 32];
+    uint32_t ipmap[EXTIOI_IRQS_IPMAP_SIZE / 4];
+    uint32_t coremap[EXTIOI_IRQS / 4];
+    uint8_t  sw_coremap[EXTIOI_IRQS];
+};
+typedef struct KVMLoongArchExtIOI KVMLoongArchExtIOI;
+DECLARE_INSTANCE_CHECKER(KVMLoongArchExtIOI, KVM_LOONGARCH_EXTIOI,
+                         TYPE_KVM_LOONGARCH_EXTIOI)
+
+struct KVMLoongArchExtIOIClass {
+    SysBusDeviceClass parent_class;
+    DeviceRealize parent_realize;
+
+    bool is_created;
+    int dev_fd;
+};
+typedef struct KVMLoongArchExtIOIClass KVMLoongArchExtIOIClass;
+DECLARE_CLASS_CHECKERS(KVMLoongArchExtIOIClass, KVM_LOONGARCH_EXTIOI,
+                       TYPE_KVM_LOONGARCH_EXTIOI)
 #endif /* LOONGARCH_EXTIOI_H */
include/hw/intc/loongarch_ipi.h +15 -0
index 276b3040a3..64ebbdcba6 100644
@@ -22,4 +22,19 @@ struct LoongarchIPIClass {
     LoongsonIPICommonClass parent_class;
 };
 
+#define TYPE_KVM_LOONGARCH_IPI  "kvm_loongarch_ipi"
+
+OBJECT_DECLARE_TYPE(KVMLoongarchIPIState,
+                    KVMLoongArchIPIClass, KVM_LOONGARCH_IPI)
+
+struct KVMLoongarchIPIState {
+    LoongsonIPICommonState parent_obj;
+    int dev_fd;
+};
+
+struct KVMLoongArchIPIClass {
+   LoongsonIPICommonClass parent_class;
+   DeviceRealize parent_realize;
+    DeviceUnrealize parent_unrealize;
+};
 #endif
include/hw/intc/loongarch_pch_pic.h +18 -3
index d5437e88f2..bbde9e6de9 100644
@@ -5,9 +5,13 @@
  * Copyright (c) 2021 Loongson Technology Corporation Limited
  */
 
+#ifndef LOONGARCH_PCH_PIC_H
+#define LOONGARCH_PCH_PIC_H
+
 #include "hw/sysbus.h"
 
-#define TYPE_LOONGARCH_PCH_PIC "loongarch_pch_pic"
+#define TYPE_LOONGARCH_PCH_PIC          "loongarch_pch_pic"
+#define TYPE_KVM_LOONGARCH_PCH_PIC      "loongarch_kvm_pch_pic"
 #define PCH_PIC_NAME(name) TYPE_LOONGARCH_PCH_PIC#name
 OBJECT_DECLARE_SIMPLE_TYPE(LoongArchPCHPIC, LOONGARCH_PCH_PIC)
 
@@ -28,15 +32,26 @@ OBJECT_DECLARE_SIMPLE_TYPE(LoongArchPCHPIC, 
 LOONGARCH_PCH_PIC)
 #define PCH_PIC_AUTO_CTRL0_HI           0xc4
 #define PCH_PIC_AUTO_CTRL1_LO           0xe0
 #define PCH_PIC_AUTO_CTRL1_HI           0xe4
-#define PCH_PIC_ROUTE_ENTRY_OFFSET      0x100
+#define PCH_PIC_ROUTE_ENTRY_START       0x100
 #define PCH_PIC_ROUTE_ENTRY_END         0x13f
-#define PCH_PIC_HTMSI_VEC_OFFSET        0x200
+#define PCH_PIC_HTMSI_VEC_START         0x200
 #define PCH_PIC_HTMSI_VEC_END           0x23f
 #define PCH_PIC_INT_STATUS_LO           0x3a0
 #define PCH_PIC_INT_STATUS_HI           0x3a4
 #define PCH_PIC_INT_POL_LO              0x3e0
 #define PCH_PIC_INT_POL_HI              0x3e4
 
+#define PCH_PIC_INT_ID_START            PCH_PIC_INT_ID_LO
+#define PCH_PIC_MASK_START              PCH_PIC_INT_MASK_LO
+#define PCH_PIC_HTMSI_EN_START          PCH_PIC_HTMSI_EN_LO
+#define PCH_PIC_EDGE_START              PCH_PIC_INT_EDGE_LO
+#define PCH_PIC_CLEAR_START             PCH_PIC_INT_CLEAR_LO
+#define PCH_PIC_AUTO_CTRL0_START        PCH_PIC_AUTO_CTRL0_LO
+#define PCH_PIC_AUTO_CTRL1_START        PCH_PIC_AUTO_CTRL1_LO
+#define PCH_PIC_INT_IRR_START           0x380
+#define PCH_PIC_INT_ISR_START           PCH_PIC_INT_STATUS_LO
+#define PCH_PIC_POLARITY_START          PCH_PIC_INT_POL_LO
+
 #define STATUS_LO_START                 0
 #define STATUS_HI_START                 0x4

#define POL_LO_START 0x40

include/hw/intc/loongarch_pch_pic.h +37 -0
@@ -67,3 +82,40 @@ struct LoongArchPCHPIC {
     MemoryRegion iomem8;
     unsigned int irq_num;
 };
+
+struct KVMLoongArchPCHPIC {
+    SysBusDevice parent_obj;
+    uint64_t int_mask; /*0x020 interrupt mask register*/
+    uint64_t htmsi_en; /*0x040 1=msi*/
+    uint64_t intedge; /*0x060 edge=1 level  =0*/
+    uint64_t intclr; /*0x080 for clean edge int,set 1 clean,set 0 is noused*/
+    uint64_t auto_crtl0; /*0x0c0*/
+    uint64_t auto_crtl1; /*0x0e0*/
+    uint64_t last_intirr;    /* edge detection */
+    uint64_t intirr; /* 0x380 interrupt request register */
+    uint64_t intisr; /* 0x3a0 interrupt service register */
+    /*
+     * 0x3e0 interrupt level polarity selection
+     * register 0 for high level trigger
+     */
+    uint64_t int_polarity;
+
+    uint8_t route_entry[64]; /*0x100 - 0x138*/
+    uint8_t htmsi_vector[64]; /*0x200 - 0x238*/
+};
+typedef struct KVMLoongArchPCHPIC KVMLoongArchPCHPIC;
+DECLARE_INSTANCE_CHECKER(KVMLoongArchPCHPIC, KVM_LOONGARCH_PCH_PIC,
+                         TYPE_KVM_LOONGARCH_PCH_PIC)
+
+struct KVMLoongArchPCHPICClass {
+    SysBusDeviceClass parent_class;
+    DeviceRealize parent_realize;
+
+    bool is_created;
+    int dev_fd;
+};
+typedef struct KVMLoongArchPCHPICClass KVMLoongArchPCHPICClass;
+DECLARE_CLASS_CHECKERS(KVMLoongArchPCHPICClass, KVM_LOONGARCH_PCH_PIC,
+                       TYPE_KVM_LOONGARCH_PCH_PIC)
+
+#endif /* LOONGARCH_PCH_PIC_H */
include/hw/intc/loongson_ipi.h +0 -1
index 4e517cc8dc..57e0d94e2b 100644
@@ -27,5 +27,4 @@ struct LoongsonIPIState {
 
     MemoryRegion *ipi_mmio_mem;
 };
-
 #endif
include/hw/intc/loongson_ipi_common.h +2 -0
index df9d9c5168..adba8ffd49 100644
@@ -45,6 +45,8 @@ struct LoongsonIPICommonClass {
     DeviceUnrealize parent_unrealize;
     AddressSpace *(*get_iocsr_as)(CPUState *cpu);
     CPUState *(*cpu_by_arch_id)(int64_t id);
+    void (*pre_save)(LoongsonIPICommonState *s);
+    void (*post_load)(LoongsonIPICommonState *s, int version_id);
 };
 
 MemTxResult loongson_ipi_core_readl(void *opaque, hwaddr addr, uint64_t *data,
include/hw/loongarch/virt.h +15 -0
index c373e48f27..c1769fc20c 100644
@@ -36,6 +36,21 @@
 
 #define FDT_BASE                0x100000
 
+/* KVM_IRQ_LINE irq field index values */
+#define KVM_LOONGARCH_IRQ_TYPE_SHIFT            24
+#define KVM_LOONGARCH_IRQ_TYPE_MASK             0xff
+#define KVM_LOONGARCH_IRQ_VCPU_SHIFT            16
+#define KVM_LOONGARCH_IRQ_VCPU_MASK             0xff
+#define KVM_LOONGARCH_IRQ_NUM_SHIFT             0
+#define KVM_LOONGARCH_IRQ_NUM_MASK              0xffff
+
+/* irq_type field */
+#define KVM_LOONGARCH_IRQ_TYPE_CPU_IP           0
+#define KVM_LOONGARCH_IRQ_TYPE_CPU_IO           1
+#define KVM_LOONGARCH_IRQ_TYPE_HT               2
+#define KVM_LOONGARCH_IRQ_TYPE_MSI              3
+#define KVM_LOONGARCH_IRQ_TYPE_IOAPIC           4
+
 struct LoongArchVirtMachineState {
     /*< private >*/
     MachineState parent_obj;
linux-headers/asm-loongarch/kvm.h +18 -0
index f9abef3823..e102d500a3 100644
@@ -108,4 +108,22 @@ struct kvm_iocsr_entry {
 #define KVM_IRQCHIP_NUM_PINS	64
 #define KVM_MAX_CORES		256
 
+#define KVM_DEV_LOONGARCH_IPI_GRP_REGS		        0x40000001
+
+#define KVM_DEV_LOONGARCH_EXTIOI_GRP_REGS	        0x40000002
+
+#define KVM_DEV_LOONGARCH_EXTIOI_GRP_SW_STATUS	        0x40000003
+#define KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_NUM_CPU   	0x0
+#define KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_FEATURE   	0x1
+#define KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_STATE   	0x2
+
+#define KVM_DEV_LOONGARCH_EXTIOI_GRP_CTRL	        0x40000004
+#define KVM_DEV_LOONGARCH_EXTIOI_CTRL_INIT_NUM_CPU   	0x0
+#define KVM_DEV_LOONGARCH_EXTIOI_CTRL_INIT_FEATURE 	0x1
+#define KVM_DEV_LOONGARCH_EXTIOI_CTRL_LOAD_FINISHED   	0x3
+
+#define KVM_DEV_LOONGARCH_PCH_PIC_GRP_REGS	        0x40000005
+#define KVM_DEV_LOONGARCH_PCH_PIC_GRP_CTRL	        0x40000006
+#define KVM_DEV_LOONGARCH_PCH_PIC_CTRL_INIT	        0
+
 #endif /* __UAPI_ASM_LOONGARCH_KVM_H */
linux-headers/linux/kvm.h +6 -0
index c93876ca0b..cd2aed735c 100644
@@ -1138,6 +1138,12 @@ enum kvm_device_type {
 #define KVM_DEV_TYPE_ARM_PV_TIME	KVM_DEV_TYPE_ARM_PV_TIME
 	KVM_DEV_TYPE_RISCV_AIA,
 #define KVM_DEV_TYPE_RISCV_AIA		KVM_DEV_TYPE_RISCV_AIA
+	KVM_DEV_TYPE_LOONGARCH_PCH_PIC,
+#define KVM_DEV_TYPE_LOONGARCH_PCH_PIC  KVM_DEV_TYPE_LOONGARCH_PCH_PIC
+	KVM_DEV_TYPE_LOONGARCH_IPI,
+#define KVM_DEV_TYPE_LOONGARCH_IPI      KVM_DEV_TYPE_LOONGARCH_IPI
+	KVM_DEV_TYPE_LOONGARCH_EXTIOI,
+#define KVM_DEV_TYPE_LOONGARCH_EXTIOI   KVM_DEV_TYPE_LOONGARCH_EXTIOI
 	KVM_DEV_TYPE_MAX,
 };
 
Signature · 1 line
2.39.1
Signature · 1 line
- Jiaxun
#5 lore raw

RFC PATCH v2 2/5 hw/loongarch: Add KVM IPI device support

In reply to Xianglai Li

Added ipi interrupt controller for kvm emulation.
The main process is to send the command word for creating an ipi device to the kernel.
When the VM is saved, the ioctl obtains the ipi interrupt controller data in the kernel and saves it. When the VM is recovered, the saved data is sent to the kernel.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

 hw/intc/Kconfig               |   3 +
 hw/intc/loongarch_ipi_kvm.c   | 128 ++++++++++++++++++++++++++++++++++
 hw/intc/loongson_ipi_common.c |  28 ++++++++
 hw/intc/meson.build           |   1 +
 hw/loongarch/Kconfig          |   1 +
 hw/loongarch/virt.c           |  40 +++++++----
 target/loongarch/kvm/kvm.c    |   1 +
 7 files changed, 189 insertions(+), 13 deletions(-)
 create mode 100644 hw/intc/loongarch_ipi_kvm.c
hw/intc/Kconfig +3 -0
index dd405bdb5d..5201505f23 100644
@@ -98,6 +98,9 @@ config LOONGARCH_IPI
     bool
     select LOONGSON_IPI_COMMON
 
+config LOONGARCH_IPI_KVM
+    bool
+
 config LOONGARCH_PCH_PIC
     bool
     select UNIMP
hw/intc/loongarch_ipi_kvm.c new file +128 -0

128 lines not on this page · raw message

hw/intc/loongson_ipi_common.c +28 -0
index a6ce0181f6..8b0683bd40 100644
@@ -289,10 +289,38 @@ static void loongson_ipi_common_unrealize(DeviceState *dev)
     g_free(s->cpu);
 }
 
+static int loongson_ipi_pre_save(void *opaque)
+{
+    IPICore *ipicore = (IPICore *)opaque;
+    LoongsonIPICommonState *s = ipicore->ipi;
+    LoongsonIPICommonClass *licc = LOONGSON_IPI_COMMON_GET_CLASS(s);
+
+    if (licc->pre_save) {
+        licc->pre_save(s);
+    }
+
+    return 0;
+}
+
+static int loongson_ipi_post_load(void *opaque, int version_id)
+{
+    IPICore *ipicore = (IPICore *)opaque;
+    LoongsonIPICommonState *s = ipicore->ipi;
+    LoongsonIPICommonClass *licc = LOONGSON_IPI_COMMON_GET_CLASS(s);
+
+    if (licc->post_load) {
+        licc->post_load(s, version_id);
+    }
+
+    return 0;
+}
+
 static const VMStateDescription vmstate_ipi_core = {
     .name = "ipi-single",
     .version_id = 2,
     .minimum_version_id = 2,
+    .pre_save  = loongson_ipi_pre_save,
+    .post_load = loongson_ipi_post_load,
     .fields = (const VMStateField[]) {
         VMSTATE_UINT32(status, IPICore),
         VMSTATE_UINT32(en, IPICore),
hw/intc/meson.build +1 -0
index f4d81eb8e4..f55eb1b80b 100644
@@ -72,6 +72,7 @@ specific_ss.add(when: 'CONFIG_M68K_IRQC', if_true: files('m68k_irqc.c'))
 specific_ss.add(when: 'CONFIG_LOONGSON_IPI_COMMON', if_true: files('loongson_ipi_common.c'))
 specific_ss.add(when: 'CONFIG_LOONGSON_IPI', if_true: files('loongson_ipi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_IPI', if_true: files('loongarch_ipi.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_IPI_KVM', if_true: files('loongarch_ipi_kvm.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c'))
hw/loongarch/Kconfig +1 -0
index 0de713a439..f8fcac3e7b 100644
@@ -16,6 +16,7 @@ config LOONGARCH_VIRT
     select LOONGARCH_PCH_PIC
     select LOONGARCH_PCH_MSI
     select LOONGARCH_EXTIOI
+    select LOONGARCH_IPI_KVM if KVM
     select LS7A_RTC
     select SMBIOS
     select ACPI_PCI
hw/loongarch/virt.c +27 -13
index 29040422aa..3b28e8e671 100644
@@ -48,6 +48,7 @@
 #include "hw/block/flash.h"
 #include "hw/virtio/virtio-iommu.h"
 #include "qemu/error-report.h"
+#include "sysemu/kvm.h"
 
 static bool virt_is_veiointc_enabled(LoongArchVirtMachineState *lvms)
 {
@@ -788,15 +789,32 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
      */
 
     /* Create IPI device */
-    ipi = qdev_new(TYPE_LOONGARCH_IPI);
-    qdev_prop_set_uint32(ipi, "num-cpu", ms->smp.cpus);
-    sysbus_realize_and_unref(SYS_BUS_DEVICE(ipi), &error_fatal);
-
-    /* IPI iocsr memory region */
-    memory_region_add_subregion(&lvms->system_iocsr, SMP_IPI_MAILBOX,
-                   sysbus_mmio_get_region(SYS_BUS_DEVICE(ipi), 0));
-    memory_region_add_subregion(&lvms->system_iocsr, MAIL_SEND_ADDR,
-                   sysbus_mmio_get_region(SYS_BUS_DEVICE(ipi), 1));
+    if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+        ipi = qdev_new(TYPE_KVM_LOONGARCH_IPI);
+        qdev_prop_set_int32(ipi, "num-cpu", ms->smp.cpus);
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(ipi), &error_fatal);
+    } else {
+        ipi = qdev_new(TYPE_LOONGARCH_IPI);
+        qdev_prop_set_uint32(ipi, "num-cpu", ms->smp.cpus);
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(ipi), &error_fatal);
+
+        /* IPI iocsr memory region */
+        memory_region_add_subregion(&lvms->system_iocsr, SMP_IPI_MAILBOX,
+                       sysbus_mmio_get_region(SYS_BUS_DEVICE(ipi), 0));
+        memory_region_add_subregion(&lvms->system_iocsr, MAIL_SEND_ADDR,
+                       sysbus_mmio_get_region(SYS_BUS_DEVICE(ipi), 1));
+
+        for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+            cpu_state = qemu_get_cpu(cpu);
+            cpudev = DEVICE(cpu_state);
+            lacpu = LOONGARCH_CPU(cpu_state);
+            env = &(lacpu->env);
+
+            /* connect ipi irq to cpu irq */
+            qdev_connect_gpio_out(ipi, cpu, qdev_get_gpio_in(cpudev, IRQ_IPI));
+            env->ipistate = ipi;
+        }
+    }
 
     /* Add cpu interrupt-controller */
     fdt_add_cpuic_node(lvms, &cpuintc_phandle);
@@ -807,10 +825,6 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
         lacpu = LOONGARCH_CPU(cpu_state);
         env = &(lacpu->env);
         env->address_space_iocsr = &lvms->as_iocsr;
-
-        /* connect ipi irq to cpu irq */
-        qdev_connect_gpio_out(ipi, cpu, qdev_get_gpio_in(cpudev, IRQ_IPI));
-        env->ipistate = ipi;
     }
 
     /* Create EXTIOI device */
target/loongarch/kvm/kvm.c +1 -0
index e1be6a6959..c07dcfd85f 100644
@@ -719,6 +719,7 @@ int kvm_arch_get_default_type(MachineState *ms)
 
 int kvm_arch_init(MachineState *ms, KVMState *s)
 {
+    s->kernel_irqchip_allowed = false;
     cap_has_mp_state = kvm_check_extension(s, KVM_CAP_MP_STATE);
     return 0;
 }
Signature · 1 line
2.39.1
#6 lore raw

RFC PATCH v2 3/5 hw/loongarch: Add KVM extioi device support

In reply to Xianglai Li

Added extioi interrupt controller for kvm emulation. The main process is to send the command word for creating an extioi device to the kernel.
When the VM is saved, the ioctl obtains the related data of the extioi interrupt controller in the kernel and saves it. When the VM is recovered, the saved data is sent to the kernel.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

 hw/intc/Kconfig                |   3 +
 hw/intc/loongarch_extioi_kvm.c | 250 +++++++++++++++++++++++++++++++++
 hw/intc/meson.build            |   1 +
 hw/loongarch/Kconfig           |   1 +
 hw/loongarch/virt.c            |  51 ++++---
 5 files changed, 285 insertions(+), 21 deletions(-)
 create mode 100644 hw/intc/loongarch_extioi_kvm.c
hw/intc/Kconfig +3 -0
index 5201505f23..df9352d41d 100644
@@ -112,3 +112,6 @@ config LOONGARCH_PCH_MSI
 
 config LOONGARCH_EXTIOI
     bool
+
+config LOONGARCH_EXTIOI_KVM
+    bool
hw/intc/loongarch_extioi_kvm.c new file +250 -0

250 lines not on this page · raw message

hw/intc/meson.build +1 -0
index f55eb1b80b..85174d1af1 100644
@@ -76,3 +76,4 @@ specific_ss.add(when: 'CONFIG_LOONGARCH_IPI_KVM', if_true: files('loongarch_ipi_
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI_KVM', if_true: files('loongarch_extioi_kvm.c'))
hw/loongarch/Kconfig +1 -0
index f8fcac3e7b..99a523171f 100644
@@ -17,6 +17,7 @@ config LOONGARCH_VIRT
     select LOONGARCH_PCH_MSI
     select LOONGARCH_EXTIOI
     select LOONGARCH_IPI_KVM if KVM
+    select LOONGARCH_EXTIOI_KVM if KVM
     select LS7A_RTC
     select SMBIOS
     select ACPI_PCI
hw/loongarch/virt.c +30 -21

58 lines not on this page · raw message

Signature · 1 line
2.39.1
#7 lore raw

Re: [RFC PATCH V2 3/5] hw/loongarch: Add KVM extioi device support

Quoted 226 lines · Xianglai Li

Added extioi interrupt controller for kvm emulation. The main process is to send the command word for creating an extioi device to the kernel.
When the VM is saved, the ioctl obtains the related data of the extioi interrupt controller in the kernel and saves it. When the VM is recovered, the saved data is sent to the kernel.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

  hw/intc/Kconfig                |   3 +
  hw/intc/loongarch_extioi_kvm.c | 250 +++++++++++++++++++++++++++++++++
  hw/intc/meson.build            |   1 +
  hw/loongarch/Kconfig           |   1 +
  hw/loongarch/virt.c            |  51 ++++---
  5 files changed, 285 insertions(+), 21 deletions(-)

create mode 100644 hw/intc/loongarch_extioi_kvm.c

hw/intc/Kconfig +3 -0
index 5201505f23..df9352d41d 100644
@@ -112,3 +112,6 @@ config LOONGARCH_PCH_MSI
 
  config LOONGARCH_EXTIOI
      bool
+
+config LOONGARCH_EXTIOI_KVM
+    bool
hw/intc/loongarch_extioi_kvm.c new file +180 -0
new file mode 100644
index 0000000000..139a00ac2a
@@ -0,0 +1,250 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch kvm extioi interrupt support
+ *
+ * Copyright (C) 2024 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "hw/qdev-properties.h"
+#include "qemu/typedefs.h"
+#include "hw/intc/loongarch_extioi.h"
+#include "hw/sysbus.h"
+#include "linux/kvm.h"
+#include "migration/vmstate.h"
+#include "qapi/error.h"
+#include "sysemu/kvm.h"
+
+static void kvm_extioi_access_regs(int fd, uint64_t addr,
+                                       void *val, bool is_write)
+{
+        kvm_device_access(fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_REGS,
+                          addr, val, is_write, &error_abort);
+}
+
+static void kvm_extioi_access_sw_status(int fd, uint64_t addr,
+                                       void *val, bool is_write)
+{
+        kvm_device_access(fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_SW_STATUS,
+                          addr, val, is_write, &error_abort);
+}
+
+static void kvm_extioi_save_load_sw_status(void *opaque, bool is_write)
+{
+    KVMLoongArchExtIOI *s = (KVMLoongArchExtIOI *)opaque;
+    KVMLoongArchExtIOIClass *class = KVM_LOONGARCH_EXTIOI_GET_CLASS(s);
+    int fd = class->dev_fd;
+    int addr;
+
+    addr = KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_NUM_CPU;
+    kvm_extioi_access_sw_status(fd, addr, (void *)&s->num_cpu, is_write);
+
+    addr = KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_FEATURE;
+    kvm_extioi_access_sw_status(fd, addr, (void *)&s->features, is_write);
+
+    addr = KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_STATE;
+    kvm_extioi_access_sw_status(fd, addr, (void *)&s->status, is_write);
+}
+
+static void kvm_extioi_save_load_regs(void *opaque, bool is_write)
+{
+    KVMLoongArchExtIOI *s = (KVMLoongArchExtIOI *)opaque;
+    KVMLoongArchExtIOIClass *class = KVM_LOONGARCH_EXTIOI_GET_CLASS(s);
+    int fd = class->dev_fd;
+    int addr, offset, cpuid;
+
+    for (addr = EXTIOI_NODETYPE_START; addr < EXTIOI_NODETYPE_END; addr += 4) {
+        offset = (addr - EXTIOI_NODETYPE_START) / 4;
+        kvm_extioi_access_regs(fd, addr,
+                               (void *)&s->nodetype[offset], is_write);
+    }
+
+    for (addr = EXTIOI_IPMAP_START; addr < EXTIOI_IPMAP_END; addr += 4) {
+        offset = (addr - EXTIOI_IPMAP_START) / 4;
+        kvm_extioi_access_regs(fd, addr, (void *)&s->ipmap[offset], is_write);
+    }
+
+    for (addr = EXTIOI_ENABLE_START; addr < EXTIOI_ENABLE_END; addr += 4) {
+        offset = (addr - EXTIOI_ENABLE_START) / 4;
+        kvm_extioi_access_regs(fd, addr,
+                               (void *)&s->enable[offset], is_write);
+    }
+
+    for (addr = EXTIOI_BOUNCE_START; addr < EXTIOI_BOUNCE_END; addr += 4) {
+        offset = (addr - EXTIOI_BOUNCE_START) / 4;
+        kvm_extioi_access_regs(fd, addr,
+                               (void *)&s->bounce[offset], is_write);
+    }
+
+    for (addr = EXTIOI_ISR_START; addr < EXTIOI_ISR_END; addr += 4) {
+        offset = (addr - EXTIOI_ISR_START) / 4;
+        kvm_extioi_access_regs(fd, addr,
+                               (void *)&s->isr[offset], is_write);
+    }
+
+    for (addr = EXTIOI_COREMAP_START; addr < EXTIOI_COREMAP_END; addr += 4) {
+        offset = (addr - EXTIOI_COREMAP_START) / 4;
+        kvm_extioi_access_regs(fd, addr,
+                               (void *)&s->coremap[offset], is_write);
+    }
+
+    for (cpuid = 0; cpuid < s->num_cpu; cpuid++) {
+        for (addr = EXTIOI_COREISR_START;
+             addr < EXTIOI_COREISR_END; addr += 4) {
+            offset = (addr - EXTIOI_COREISR_START) / 4;
+            addr = (cpuid << 16) | addr;
+            kvm_extioi_access_regs(fd, addr,
+                              (void *)&s->coreisr[cpuid][offset], is_write);
+        }
+    }
+}
+
+static int kvm_loongarch_extioi_pre_save(void *opaque)
+{
+    kvm_extioi_save_load_regs(opaque, false);
+    kvm_extioi_save_load_sw_status(opaque, false);
+    return 0;
+}
+
+static int kvm_loongarch_extioi_post_load(void *opaque, int version_id)
+{
+    KVMLoongArchExtIOI *s = (KVMLoongArchExtIOI *)opaque;
+    KVMLoongArchExtIOIClass *class = KVM_LOONGARCH_EXTIOI_GET_CLASS(s);
+    int fd = class->dev_fd;
+
+    kvm_extioi_save_load_regs(opaque, true);
+    kvm_extioi_save_load_sw_status(opaque, true);
+
+    kvm_device_access(fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_CTRL,
+                      KVM_DEV_LOONGARCH_EXTIOI_CTRL_LOAD_FINISHED,
+                      NULL, true, &error_abort);
+    return 0;
+}
+
+static void kvm_loongarch_extioi_realize(DeviceState *dev, Error **errp)
+{
+    KVMLoongArchExtIOIClass *extioi_class = KVM_LOONGARCH_EXTIOI_GET_CLASS(dev);
+    KVMLoongArchExtIOI *s = KVM_LOONGARCH_EXTIOI(dev);
+    struct kvm_create_device cd = {0};
+    Error *err = NULL;
+    int ret, i;
+
+    extioi_class->parent_realize(dev, &err);
+    if (err) {
+        error_propagate(errp, err);
+        return;
+    }
+
+    if (s->num_cpu == 0) {
+        error_setg(errp, "num-cpu must be at least 1");
+        return;
+    }
+
+
+    if (extioi_class->is_created) {
+        error_setg(errp, "extioi had be created");
+        return;
+    }
+
+    if (s->features & BIT(EXTIOI_HAS_VIRT_EXTENSION)) {
+        s->features |= EXTIOI_VIRT_HAS_FEATURES;
+    }
+
+    cd.type = KVM_DEV_TYPE_LOONGARCH_EXTIOI;
+    ret = kvm_vm_ioctl(kvm_state, KVM_CREATE_DEVICE, &cd);
+    if (ret < 0) {
+        error_setg_errno(errp, errno,
+                         "Creating the KVM extioi device failed");
+        return;
+    }
+    extioi_class->is_created = true;
+    extioi_class->dev_fd = cd.fd;
+
+    ret = kvm_device_access(cd.fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_CTRL,
+                            KVM_DEV_LOONGARCH_EXTIOI_CTRL_INIT_NUM_CPU,
+                            &s->num_cpu, true, NULL);
+    if (ret < 0) {
+        error_setg_errno(errp, errno,
+                         "KVM EXTIOI: failed to set the num-cpu of EXTIOI");
+        exit(1);
+    }
+
+    ret = kvm_device_access(cd.fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_CTRL,
+                            KVM_DEV_LOONGARCH_EXTIOI_CTRL_INIT_FEATURE,
+                            &s->features, true, NULL);
+    if (ret < 0) {
+        error_setg_errno(errp, errno,
+                         "KVM EXTIOI: failed to set the feature of EXTIOI");
+        exit(1);
+    }
+

hw/intc/loongarch_extioi_kvm.c line 190

Quoted 10 lines
hw/intc/loongarch_extioi_kvm.c new file +10 -0
+    fprintf(stdout, "Create LoongArch extioi irqchip in KVM done!\n");
+
+    kvm_async_interrupts_allowed = true;
+    kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
+    if (kvm_has_gsi_routing()) {
+        for (i = 0; i < 64; ++i) {
+            kvm_irqchip_add_irq_route(kvm_state, i, 0, i);
+        }
+        kvm_gsi_routing_allowed = true;
+    }

Does memory region need be created to extioi with irqchip_in_kernel mode?

Quoted 16 lines
hw/intc/loongarch_extioi_kvm.c new file +16 -0
+}
+
+static const VMStateDescription vmstate_kvm_extioi_core = {
+    .name = "kvm-extioi-single",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .pre_save = kvm_loongarch_extioi_pre_save,
+    .post_load = kvm_loongarch_extioi_post_load,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT32_ARRAY(nodetype, KVMLoongArchExtIOI,
+                             EXTIOI_IRQS_NODETYPE_COUNT / 2),
+        VMSTATE_UINT32_ARRAY(bounce, KVMLoongArchExtIOI,
+                             EXTIOI_IRQS_GROUP_COUNT),
+        VMSTATE_UINT32_ARRAY(isr, KVMLoongArchExtIOI, EXTIOI_IRQS / 32),
+        VMSTATE_UINT32_2DARRAY(coreisr, KVMLoongArchExtIOI, EXTIOI_CPUS,
+                               EXTIOI_IRQS_GROUP_COUNT),

hw/intc/loongarch_extioi_kvm.c line 216

Quoted 10 lines
hw/intc/loongarch_extioi_kvm.c new file +10 -0
+        VMSTATE_UINT32_ARRAY(enable, KVMLoongArchExtIOI, EXTIOI_IRQS / 32),
+        VMSTATE_UINT32_ARRAY(ipmap, KVMLoongArchExtIOI,
+                             EXTIOI_IRQS_IPMAP_SIZE / 4),
+        VMSTATE_UINT32_ARRAY(coremap, KVMLoongArchExtIOI, EXTIOI_IRQS / 4),
+        VMSTATE_UINT8_ARRAY(sw_coremap, KVMLoongArchExtIOI, EXTIOI_IRQS),
+        VMSTATE_UINT32(features, KVMLoongArchExtIOI),
+        VMSTATE_UINT32(status, KVMLoongArchExtIOI),
+        VMSTATE_END_OF_LIST()
+    }
+};

It is duplicated with structure vmstate_loongarch_extioi in file hw/intc/loongarch_extioi.c

hw/intc/loongarch_extioi_kvm.c line 223

Quoted 7 lines
hw/intc/loongarch_extioi_kvm.c new file +7 -0
+
+static Property extioi_properties[] = {
+    DEFINE_PROP_UINT32("num-cpu", KVMLoongArchExtIOI, num_cpu, 1),
+    DEFINE_PROP_BIT("has-virtualization-extension", KVMLoongArchExtIOI,
+                    features, EXTIOI_HAS_VIRT_EXTENSION, 0),
+    DEFINE_PROP_END_OF_LIST(),
+};

Ditto, it is duplicated.

hw/intc/loongarch_extioi_kvm.c line 234

Quoted 11 lines
hw/intc/loongarch_extioi_kvm.c new file +11 -0
+
+static void kvm_loongarch_extioi_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    KVMLoongArchExtIOIClass *extioi_class = KVM_LOONGARCH_EXTIOI_CLASS(oc);
+
+    extioi_class->parent_realize = dc->realize;
+    dc->realize = kvm_loongarch_extioi_realize;
+    extioi_class->is_created = false;
+    device_class_set_props(dc, extioi_properties);
+    dc->vmsd = &vmstate_kvm_extioi_core;

Do we need reset interface for irqchip_in_kernel mode?

Regards
Bibo Mao

Quoted 101 lines
hw/intc/loongarch_extioi_kvm.c new file +16 -0
+}
+
+static const TypeInfo kvm_loongarch_extioi_info = {
+    .name = TYPE_KVM_LOONGARCH_EXTIOI,
+    .parent = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(KVMLoongArchExtIOI),
+    .class_size = sizeof(KVMLoongArchExtIOIClass),
+    .class_init = kvm_loongarch_extioi_class_init,
+};
+
+static void kvm_loongarch_extioi_register_types(void)
+{
+    type_register_static(&kvm_loongarch_extioi_info);
+}
+
+type_init(kvm_loongarch_extioi_register_types)
hw/intc/meson.build +1 -0
index f55eb1b80b..85174d1af1 100644
@@ -76,3 +76,4 @@ specific_ss.add(when: 'CONFIG_LOONGARCH_IPI_KVM', if_true: files('loongarch_ipi_
  specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c'))
  specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
  specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI_KVM', if_true: files('loongarch_extioi_kvm.c'))
hw/loongarch/Kconfig +1 -0
index f8fcac3e7b..99a523171f 100644
@@ -17,6 +17,7 @@ config LOONGARCH_VIRT
      select LOONGARCH_PCH_MSI
      select LOONGARCH_EXTIOI
      select LOONGARCH_IPI_KVM if KVM
+    select LOONGARCH_EXTIOI_KVM if KVM
      select LS7A_RTC
      select SMBIOS
      select ACPI_PCI
hw/loongarch/virt.c +30 -21
index 3b28e8e671..8ca7c09016 100644
@@ -828,28 +828,37 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
      }
 
      /* Create EXTIOI device */
-    extioi = qdev_new(TYPE_LOONGARCH_EXTIOI);
-    qdev_prop_set_uint32(extioi, "num-cpu", ms->smp.cpus);
-    if (virt_is_veiointc_enabled(lvms)) {
-        qdev_prop_set_bit(extioi, "has-virtualization-extension", true);
-    }
-    sysbus_realize_and_unref(SYS_BUS_DEVICE(extioi), &error_fatal);
-    memory_region_add_subregion(&lvms->system_iocsr, APIC_BASE,
-                    sysbus_mmio_get_region(SYS_BUS_DEVICE(extioi), 0));
-    if (virt_is_veiointc_enabled(lvms)) {
-        memory_region_add_subregion(&lvms->system_iocsr, EXTIOI_VIRT_BASE,
-                    sysbus_mmio_get_region(SYS_BUS_DEVICE(extioi), 1));
-    }
+    if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+        extioi = qdev_new(TYPE_KVM_LOONGARCH_EXTIOI);
+        qdev_prop_set_uint32(extioi, "num-cpu", ms->smp.cpus);
+        if (virt_is_veiointc_enabled(lvms)) {
+            qdev_prop_set_bit(extioi, "has-virtualization-extension", true);
+        }
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(extioi), &error_fatal);
+    } else {
+        extioi = qdev_new(TYPE_LOONGARCH_EXTIOI);
+        qdev_prop_set_uint32(extioi, "num-cpu", ms->smp.cpus);
+        if (virt_is_veiointc_enabled(lvms)) {
+            qdev_prop_set_bit(extioi, "has-virtualization-extension", true);
+        }
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(extioi), &error_fatal);
+        memory_region_add_subregion(&lvms->system_iocsr, APIC_BASE,
+                       sysbus_mmio_get_region(SYS_BUS_DEVICE(extioi), 0));
+        if (virt_is_veiointc_enabled(lvms)) {
+            memory_region_add_subregion(&lvms->system_iocsr, EXTIOI_VIRT_BASE,
+                        sysbus_mmio_get_region(SYS_BUS_DEVICE(extioi), 1));
+        }
 
-    /*
-     * connect ext irq to the cpu irq
-     * cpu_pin[9:2] <= intc_pin[7:0]
-     */
-    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
-        cpudev = DEVICE(qemu_get_cpu(cpu));
-        for (pin = 0; pin < LS3A_INTC_IP; pin++) {
-            qdev_connect_gpio_out(extioi, (cpu * 8 + pin),
-                                  qdev_get_gpio_in(cpudev, pin + 2));
+        /*
+         * connect ext irq to the cpu irq
+         * cpu_pin[9:2] <= intc_pin[7:0]
+         */
+        for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+            cpudev = DEVICE(qemu_get_cpu(cpu));
+            for (pin = 0; pin < LS3A_INTC_IP; pin++) {
+                qdev_connect_gpio_out(extioi, (cpu * 8 + pin),
+                                      qdev_get_gpio_in(cpudev, pin + 2));
+            }
          }
      }
 
#8 lore raw

RFC PATCH v2 4/5 hw/loongarch: Add KVM pch pic device support

In reply to Xianglai Li

Added pch_pic interrupt controller for kvm emulation. The main process is to send the command word for creating an pch_pic device to the kernel,
Delivers the pch pic interrupt controller configuration register base address to the kernel.
When the VM is saved, the ioctl obtains the pch_pic interrupt controller data in the kernel and saves it. When the VM is recovered, the saved data is sent to the kernel.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

 hw/intc/Kconfig                 |   3 +
 hw/intc/loongarch_pch_pic.c     |  40 ++++---
 hw/intc/loongarch_pch_pic_kvm.c | 180 ++++++++++++++++++++++++++++++++
 hw/intc/meson.build             |   1 +
 hw/loongarch/Kconfig            |   1 +
 hw/loongarch/virt.c             |  67 ++++++------
 6 files changed, 249 insertions(+), 43 deletions(-)
 create mode 100644 hw/intc/loongarch_pch_pic_kvm.c
hw/intc/Kconfig +3 -0
index df9352d41d..1169926eec 100644
@@ -105,6 +105,9 @@ config LOONGARCH_PCH_PIC
     bool
     select UNIMP
 
+config LOONGARCH_PCH_PIC_KVM
+    bool
+
 config LOONGARCH_PCH_MSI
     select MSI_NONBROKEN
     bool
hw/intc/loongarch_pch_pic.c +28 -12
index 2d5e65abff..13934be7d9 100644
@@ -16,18 +16,27 @@
 #include "migration/vmstate.h"
 #include "trace.h"
 #include "qapi/error.h"
+#include "sysemu/kvm.h"
 
 static void pch_pic_update_irq(LoongArchPCHPIC *s, uint64_t mask, int level)
 {
     uint64_t val;
     int irq;
+    int kvm_irq;
 
     if (level) {
         val = mask & s->intirr & ~s->int_mask;
         if (val) {
             irq = ctz64(val);
             s->intisr |= MAKE_64BIT_MASK(irq, 1);
-            qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 1);
+            if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+                kvm_irq = (
+                KVM_LOONGARCH_IRQ_TYPE_IOAPIC << KVM_LOONGARCH_IRQ_TYPE_SHIFT)
+                | (0 <<  KVM_LOONGARCH_IRQ_VCPU_SHIFT) | s->htmsi_vector[irq];
+                kvm_set_irq(kvm_state, kvm_irq, !!level);
+            } else {
+                qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 1);
+            }
         }
     } else {
         /*
@@ -38,7 +47,14 @@ static void pch_pic_update_irq(LoongArchPCHPIC *s, uint64_t mask, int level)
         if (val) {
             irq = ctz64(val);
             s->intisr &= ~MAKE_64BIT_MASK(irq, 1);
-            qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 0);
+            if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+                kvm_irq = (
+                KVM_LOONGARCH_IRQ_TYPE_IOAPIC << KVM_LOONGARCH_IRQ_TYPE_SHIFT)
+                | (0 <<  KVM_LOONGARCH_IRQ_VCPU_SHIFT) | s->htmsi_vector[irq];
+                kvm_set_irq(kvm_state, kvm_irq, !!level);
+            } else {
+                qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 0);
+            }
         }
     }
 }
@@ -265,18 +281,18 @@ static uint64_t loongarch_pch_pic_readb(void *opaque, hwaddr addr,
 {
     LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
     uint64_t val = 0;
-    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_OFFSET;
+    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_START;
     int64_t offset_tmp;
 
     switch (offset) {
-    case PCH_PIC_HTMSI_VEC_OFFSET ... PCH_PIC_HTMSI_VEC_END:
-        offset_tmp = offset - PCH_PIC_HTMSI_VEC_OFFSET;
+    case PCH_PIC_HTMSI_VEC_START ... PCH_PIC_HTMSI_VEC_END:
+        offset_tmp = offset - PCH_PIC_HTMSI_VEC_START;
         if (offset_tmp >= 0 && offset_tmp < 64) {
             val = s->htmsi_vector[offset_tmp];
         }
         break;
-    case PCH_PIC_ROUTE_ENTRY_OFFSET ... PCH_PIC_ROUTE_ENTRY_END:
-        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_OFFSET;
+    case PCH_PIC_ROUTE_ENTRY_START ... PCH_PIC_ROUTE_ENTRY_END:
+        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_START;
         if (offset_tmp >= 0 && offset_tmp < 64) {
             val = s->route_entry[offset_tmp];
         }
@@ -294,19 +310,19 @@ static void loongarch_pch_pic_writeb(void *opaque, hwaddr addr,
 {
     LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
     int32_t offset_tmp;
-    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_OFFSET;
+    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_START;
 
     trace_loongarch_pch_pic_writeb(size, addr, data);
 
     switch (offset) {
-    case PCH_PIC_HTMSI_VEC_OFFSET ... PCH_PIC_HTMSI_VEC_END:
-        offset_tmp = offset - PCH_PIC_HTMSI_VEC_OFFSET;
+    case PCH_PIC_HTMSI_VEC_START ... PCH_PIC_HTMSI_VEC_END:
+        offset_tmp = offset - PCH_PIC_HTMSI_VEC_START;
         if (offset_tmp >= 0 && offset_tmp < 64) {
             s->htmsi_vector[offset_tmp] = (uint8_t)(data & 0xff);
         }
         break;
-    case PCH_PIC_ROUTE_ENTRY_OFFSET ... PCH_PIC_ROUTE_ENTRY_END:
-        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_OFFSET;
+    case PCH_PIC_ROUTE_ENTRY_START ... PCH_PIC_ROUTE_ENTRY_END:
+        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_START;
         if (offset_tmp >= 0 && offset_tmp < 64) {
             s->route_entry[offset_tmp] = (uint8_t)(data & 0xff);
         }
hw/intc/loongarch_pch_pic_kvm.c new file +180 -0

180 lines not on this page · raw message

hw/intc/meson.build +1 -0
index 85174d1af1..c20c0a2c05 100644
@@ -77,3 +77,4 @@ specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI_KVM', if_true: files('loongarch_extioi_kvm.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC_KVM', if_true: files('loongarch_pch_pic_kvm.c'))
hw/loongarch/Kconfig +1 -0
index 99a523171f..f909f799ad 100644
@@ -17,6 +17,7 @@ config LOONGARCH_VIRT
     select LOONGARCH_PCH_MSI
     select LOONGARCH_EXTIOI
     select LOONGARCH_IPI_KVM if KVM
+    select LOONGARCH_PCH_PIC_KVM if KVM
     select LOONGARCH_EXTIOI_KVM if KVM
     select LS7A_RTC
     select SMBIOS
hw/loongarch/virt.c +36 -31

76 lines not on this page · raw message

Signature · 1 line
2.39.1
#9 lore raw

Re: [RFC PATCH V2 4/5] hw/loongarch: Add KVM pch pic device support

Quoted 64 lines · Xianglai Li

Added pch_pic interrupt controller for kvm emulation. The main process is to send the command word for creating an pch_pic device to the kernel,
Delivers the pch pic interrupt controller configuration register base address to the kernel.
When the VM is saved, the ioctl obtains the pch_pic interrupt controller data in the kernel and saves it. When the VM is recovered, the saved data is sent to the kernel.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

  hw/intc/Kconfig                 |   3 +
  hw/intc/loongarch_pch_pic.c     |  40 ++++---
  hw/intc/loongarch_pch_pic_kvm.c | 180 ++++++++++++++++++++++++++++++++
  hw/intc/meson.build             |   1 +
  hw/loongarch/Kconfig            |   1 +
  hw/loongarch/virt.c             |  67 ++++++------
  6 files changed, 249 insertions(+), 43 deletions(-)

create mode 100644 hw/intc/loongarch_pch_pic_kvm.c

hw/intc/Kconfig +3 -0
index df9352d41d..1169926eec 100644
@@ -105,6 +105,9 @@ config LOONGARCH_PCH_PIC
      bool
      select UNIMP
 
+config LOONGARCH_PCH_PIC_KVM
+    bool
+
  config LOONGARCH_PCH_MSI
      select MSI_NONBROKEN
      bool
hw/intc/loongarch_pch_pic.c +2 -0
index 2d5e65abff..13934be7d9 100644
@@ -16,18 +16,27 @@
  #include "migration/vmstate.h"
  #include "trace.h"
  #include "qapi/error.h"
+#include "sysemu/kvm.h"
 
  static void pch_pic_update_irq(LoongArchPCHPIC *s, uint64_t mask, int level)
  {
      uint64_t val;
      int irq;
+    int kvm_irq;
 
      if (level) {
          val = mask & s->intirr & ~s->int_mask;
          if (val) {
              irq = ctz64(val);

hw/intc/loongarch_pch_pic.c line 39

Quoted 10 lines
hw/intc/loongarch_pch_pic.c +8 -1
              s->intisr |= MAKE_64BIT_MASK(irq, 1);
-            qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 1);
+            if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+                kvm_irq = (
+                KVM_LOONGARCH_IRQ_TYPE_IOAPIC << KVM_LOONGARCH_IRQ_TYPE_SHIFT)
+                | (0 <<  KVM_LOONGARCH_IRQ_VCPU_SHIFT) | s->htmsi_vector[irq];
+                kvm_set_irq(kvm_state, kvm_irq, !!level);
+            } else {
+                qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 1);
+            }

From my point, modification is unnecessay, since there is separate irq handler if irqchip_in_kernel in file hw/intc/loongarch_pch_pic_kvm.c

Also I do not know why there is so such modification with file hw/intc/loongarch_pch_pic.c, it is irrelative and not used if irqchip_in_kernel is set.

Regards
Bibo Mao

Quoted 357 lines
hw/intc/loongarch_pch_pic.c +18 -11
          }
      } else {
          /*
@@ -38,7 +47,14 @@ static void pch_pic_update_irq(LoongArchPCHPIC *s, uint64_t mask, int level)
          if (val) {
              irq = ctz64(val);
              s->intisr &= ~MAKE_64BIT_MASK(irq, 1);
-            qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 0);
+            if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+                kvm_irq = (
+                KVM_LOONGARCH_IRQ_TYPE_IOAPIC << KVM_LOONGARCH_IRQ_TYPE_SHIFT)
+                | (0 <<  KVM_LOONGARCH_IRQ_VCPU_SHIFT) | s->htmsi_vector[irq];
+                kvm_set_irq(kvm_state, kvm_irq, !!level);
+            } else {
+                qemu_set_irq(s->parent_irq[s->htmsi_vector[irq]], 0);
+            }
          }
      }
  }
@@ -265,18 +281,18 @@ static uint64_t loongarch_pch_pic_readb(void *opaque, hwaddr addr,
  {
      LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
      uint64_t val = 0;
-    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_OFFSET;
+    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_START;
      int64_t offset_tmp;
 
      switch (offset) {
-    case PCH_PIC_HTMSI_VEC_OFFSET ... PCH_PIC_HTMSI_VEC_END:
-        offset_tmp = offset - PCH_PIC_HTMSI_VEC_OFFSET;
+    case PCH_PIC_HTMSI_VEC_START ... PCH_PIC_HTMSI_VEC_END:
+        offset_tmp = offset - PCH_PIC_HTMSI_VEC_START;
          if (offset_tmp >= 0 && offset_tmp < 64) {
              val = s->htmsi_vector[offset_tmp];
          }
          break;
-    case PCH_PIC_ROUTE_ENTRY_OFFSET ... PCH_PIC_ROUTE_ENTRY_END:
-        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_OFFSET;
+    case PCH_PIC_ROUTE_ENTRY_START ... PCH_PIC_ROUTE_ENTRY_END:
+        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_START;
          if (offset_tmp >= 0 && offset_tmp < 64) {
              val = s->route_entry[offset_tmp];
          }
@@ -294,19 +310,19 @@ static void loongarch_pch_pic_writeb(void *opaque, hwaddr addr,
  {
      LoongArchPCHPIC *s = LOONGARCH_PCH_PIC(opaque);
      int32_t offset_tmp;
-    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_OFFSET;
+    uint32_t offset = (addr & 0xfff) + PCH_PIC_ROUTE_ENTRY_START;
 
      trace_loongarch_pch_pic_writeb(size, addr, data);
 
      switch (offset) {
-    case PCH_PIC_HTMSI_VEC_OFFSET ... PCH_PIC_HTMSI_VEC_END:
-        offset_tmp = offset - PCH_PIC_HTMSI_VEC_OFFSET;
+    case PCH_PIC_HTMSI_VEC_START ... PCH_PIC_HTMSI_VEC_END:
+        offset_tmp = offset - PCH_PIC_HTMSI_VEC_START;
          if (offset_tmp >= 0 && offset_tmp < 64) {
              s->htmsi_vector[offset_tmp] = (uint8_t)(data & 0xff);
          }
          break;
-    case PCH_PIC_ROUTE_ENTRY_OFFSET ... PCH_PIC_ROUTE_ENTRY_END:
-        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_OFFSET;
+    case PCH_PIC_ROUTE_ENTRY_START ... PCH_PIC_ROUTE_ENTRY_END:
+        offset_tmp = offset - PCH_PIC_ROUTE_ENTRY_START;
          if (offset_tmp >= 0 && offset_tmp < 64) {
              s->route_entry[offset_tmp] = (uint8_t)(data & 0xff);
          }
hw/intc/loongarch_pch_pic_kvm.c new file +180 -0
new file mode 100644
index 0000000000..9b6a2f6784
@@ -0,0 +1,180 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch kvm pch pic interrupt support
+ *
+ * Copyright (C) 2024 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "hw/qdev-properties.h"
+#include "qemu/typedefs.h"
+#include "hw/intc/loongarch_pch_pic.h"
+#include "hw/sysbus.h"
+#include "linux/kvm.h"
+#include "migration/vmstate.h"
+#include "qapi/error.h"
+#include "sysemu/kvm.h"
+#include "hw/loongarch/virt.h"
+#include "hw/pci-host/ls7a.h"
+#include "qemu/error-report.h"
+
+static void kvm_pch_pic_access_regs(int fd, uint64_t addr,
+                                       void *val, bool is_write)
+{
+        kvm_device_access(fd, KVM_DEV_LOONGARCH_PCH_PIC_GRP_REGS,
+                          addr, val, is_write, &error_abort);
+}
+
+static void kvm_loongarch_pch_pic_save_load(void *opaque, bool is_write)
+{
+    KVMLoongArchPCHPIC *s = (KVMLoongArchPCHPIC *)opaque;
+    KVMLoongArchPCHPICClass *class = KVM_LOONGARCH_PCH_PIC_GET_CLASS(s);
+    int fd = class->dev_fd;
+    int addr, offset;
+
+    kvm_pch_pic_access_regs(fd, PCH_PIC_MASK_START,
+                            (void *)&s->int_mask, is_write);
+    kvm_pch_pic_access_regs(fd, PCH_PIC_HTMSI_EN_START,
+                            (void *)&s->htmsi_en, is_write);
+    kvm_pch_pic_access_regs(fd, PCH_PIC_EDGE_START,
+                            (void *)&s->intedge, is_write);
+    kvm_pch_pic_access_regs(fd, PCH_PIC_AUTO_CTRL0_START,
+                            (void *)&s->auto_crtl0, is_write);
+    kvm_pch_pic_access_regs(fd, PCH_PIC_AUTO_CTRL1_START,
+                            (void *)&s->auto_crtl1, is_write);
+
+    for (addr = PCH_PIC_ROUTE_ENTRY_START;
+         addr < PCH_PIC_ROUTE_ENTRY_END; addr++) {
+        offset = addr - PCH_PIC_ROUTE_ENTRY_START;
+        kvm_pch_pic_access_regs(fd, addr,
+                                (void *)&s->route_entry[offset], is_write);
+    }
+
+    for (addr = PCH_PIC_HTMSI_VEC_START; addr < PCH_PIC_HTMSI_VEC_END; addr++) {
+        offset = addr - PCH_PIC_HTMSI_VEC_START;
+        kvm_pch_pic_access_regs(fd, addr,
+                                (void *)&s->htmsi_vector[offset], is_write);
+    }
+
+    kvm_pch_pic_access_regs(fd, PCH_PIC_INT_IRR_START,
+                            (void *)&s->intirr, is_write);
+    kvm_pch_pic_access_regs(fd, PCH_PIC_INT_ISR_START,
+                            (void *)&s->intisr, is_write);
+    kvm_pch_pic_access_regs(fd, PCH_PIC_POLARITY_START,
+                            (void *)&s->int_polarity, is_write);
+}
+
+static int kvm_loongarch_pch_pic_pre_save(void *opaque)
+{
+    kvm_loongarch_pch_pic_save_load(opaque, false);
+    return 0;
+}
+
+static int kvm_loongarch_pch_pic_post_load(void *opaque, int version_id)
+{
+    kvm_loongarch_pch_pic_save_load(opaque, true);
+    return 0;
+}
+
+static void kvm_pch_pic_handler(void *opaque, int irq, int level)
+{
+    int kvm_irq;
+
+    if (kvm_enabled()) {
+        kvm_irq = \
+            (KVM_LOONGARCH_IRQ_TYPE_IOAPIC << KVM_LOONGARCH_IRQ_TYPE_SHIFT)
+            | (0 <<  KVM_LOONGARCH_IRQ_VCPU_SHIFT) | irq;
+        kvm_set_irq(kvm_state, kvm_irq, !!level);
+    }
+}
+
+static void kvm_loongarch_pch_pic_realize(DeviceState *dev, Error **errp)
+{
+    KVMLoongArchPCHPICClass *pch_pic_class =
+            KVM_LOONGARCH_PCH_PIC_GET_CLASS(dev);
+    struct kvm_create_device cd = {0};
+    uint64_t pch_pic_base = VIRT_PCH_REG_BASE;
+    Error *err = NULL;
+    int ret;
+
+    pch_pic_class->parent_realize(dev, &err);
+    if (err) {
+        error_propagate(errp, err);
+        return;
+    }
+
+    if (!pch_pic_class->is_created) {
+        cd.type = KVM_DEV_TYPE_LOONGARCH_PCH_PIC;
+        ret = kvm_vm_ioctl(kvm_state, KVM_CREATE_DEVICE, &cd);
+        if (ret < 0) {
+            error_setg_errno(errp, errno,
+                             "Creating the KVM pch pic device failed");
+            return;
+        }
+        pch_pic_class->is_created = true;
+        pch_pic_class->dev_fd = cd.fd;
+        fprintf(stdout, "Create LoongArch pch pic irqchip in KVM done!\n");
+
+        ret = kvm_device_access(cd.fd, KVM_DEV_LOONGARCH_PCH_PIC_GRP_CTRL,
+                                KVM_DEV_LOONGARCH_PCH_PIC_CTRL_INIT,
+                                &pch_pic_base, true, NULL);
+        if (ret < 0) {
+            error_report(
+                "KVM PCH_PIC: failed to set the base address of PCH PIC");
+            exit(1);
+        }
+
+        qdev_init_gpio_in(dev, kvm_pch_pic_handler, VIRT_PCH_PIC_IRQ_NUM);
+    }
+}
+
+static const VMStateDescription vmstate_kvm_loongarch_pch_pic = {
+    .name = TYPE_LOONGARCH_PCH_PIC,
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .pre_save = kvm_loongarch_pch_pic_pre_save,
+    .post_load = kvm_loongarch_pch_pic_post_load,
+    .fields = (const VMStateField[]) {
+        VMSTATE_UINT64(int_mask, KVMLoongArchPCHPIC),
+        VMSTATE_UINT64(htmsi_en, KVMLoongArchPCHPIC),
+        VMSTATE_UINT64(intedge, KVMLoongArchPCHPIC),
+        VMSTATE_UINT64(intclr, KVMLoongArchPCHPIC),
+        VMSTATE_UINT64(auto_crtl0, KVMLoongArchPCHPIC),
+        VMSTATE_UINT64(auto_crtl1, KVMLoongArchPCHPIC),
+        VMSTATE_UINT8_ARRAY(route_entry, KVMLoongArchPCHPIC, 64),
+        VMSTATE_UINT8_ARRAY(htmsi_vector, KVMLoongArchPCHPIC, 64),
+        VMSTATE_UINT64(last_intirr, KVMLoongArchPCHPIC),
+        VMSTATE_UINT64(intirr, KVMLoongArchPCHPIC),
+        VMSTATE_UINT64(intisr, KVMLoongArchPCHPIC),
+        VMSTATE_UINT64(int_polarity, KVMLoongArchPCHPIC),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+
+static void kvm_loongarch_pch_pic_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    KVMLoongArchPCHPICClass *pch_pic_class = KVM_LOONGARCH_PCH_PIC_CLASS(oc);
+
+    pch_pic_class->parent_realize = dc->realize;
+    dc->realize = kvm_loongarch_pch_pic_realize;
+    pch_pic_class->is_created = false;
+    dc->vmsd = &vmstate_kvm_loongarch_pch_pic;
+
+}
+
+static const TypeInfo kvm_loongarch_pch_pic_info = {
+    .name = TYPE_KVM_LOONGARCH_PCH_PIC,
+    .parent = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(KVMLoongArchPCHPIC),
+    .class_size = sizeof(KVMLoongArchPCHPICClass),
+    .class_init = kvm_loongarch_pch_pic_class_init,
+};
+
+static void kvm_loongarch_pch_pic_register_types(void)
+{
+    type_register_static(&kvm_loongarch_pch_pic_info);
+}
+
+type_init(kvm_loongarch_pch_pic_register_types)
hw/intc/meson.build +1 -0
index 85174d1af1..c20c0a2c05 100644
@@ -77,3 +77,4 @@ specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_
  specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
  specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c'))
  specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI_KVM', if_true: files('loongarch_extioi_kvm.c'))
+specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC_KVM', if_true: files('loongarch_pch_pic_kvm.c'))
hw/loongarch/Kconfig +1 -0
index 99a523171f..f909f799ad 100644
@@ -17,6 +17,7 @@ config LOONGARCH_VIRT
      select LOONGARCH_PCH_MSI
      select LOONGARCH_EXTIOI
      select LOONGARCH_IPI_KVM if KVM
+    select LOONGARCH_PCH_PIC_KVM if KVM
      select LOONGARCH_EXTIOI_KVM if KVM
      select LS7A_RTC
      select SMBIOS
hw/loongarch/virt.c +36 -31
index 8ca7c09016..db0c08899b 100644
@@ -865,40 +865,45 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
      /* Add Extend I/O Interrupt Controller node */
      fdt_add_eiointc_node(lvms, &cpuintc_phandle, &eiointc_phandle);
 
-    pch_pic = qdev_new(TYPE_LOONGARCH_PCH_PIC);
-    num = VIRT_PCH_PIC_IRQ_NUM;
-    qdev_prop_set_uint32(pch_pic, "pch_pic_irq_num", num);
-    d = SYS_BUS_DEVICE(pch_pic);
-    sysbus_realize_and_unref(d, &error_fatal);
-    memory_region_add_subregion(get_system_memory(), VIRT_IOAPIC_REG_BASE,
-                            sysbus_mmio_get_region(d, 0));
-    memory_region_add_subregion(get_system_memory(),
-                            VIRT_IOAPIC_REG_BASE + PCH_PIC_ROUTE_ENTRY_OFFSET,
+    if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+        pch_pic = qdev_new(TYPE_KVM_LOONGARCH_PCH_PIC);
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(pch_pic), &error_fatal);
+    } else {
+        pch_pic = qdev_new(TYPE_LOONGARCH_PCH_PIC);
+        num = VIRT_PCH_PIC_IRQ_NUM;
+        qdev_prop_set_uint32(pch_pic, "pch_pic_irq_num", num);
+        d = SYS_BUS_DEVICE(pch_pic);
+        sysbus_realize_and_unref(d, &error_fatal);
+        memory_region_add_subregion(get_system_memory(), VIRT_IOAPIC_REG_BASE,
+                                sysbus_mmio_get_region(d, 0));
+        memory_region_add_subregion(get_system_memory(),
+                            VIRT_IOAPIC_REG_BASE + PCH_PIC_ROUTE_ENTRY_START,
                              sysbus_mmio_get_region(d, 1));
-    memory_region_add_subregion(get_system_memory(),
-                            VIRT_IOAPIC_REG_BASE + PCH_PIC_INT_STATUS_LO,
-                            sysbus_mmio_get_region(d, 2));
-
-    /* Connect pch_pic irqs to extioi */
-    for (i = 0; i < num; i++) {
-        qdev_connect_gpio_out(DEVICE(d), i, qdev_get_gpio_in(extioi, i));
-    }
+        memory_region_add_subregion(get_system_memory(),
+                                VIRT_IOAPIC_REG_BASE + PCH_PIC_INT_STATUS_LO,
+                                sysbus_mmio_get_region(d, 2));
 
-    /* Add PCH PIC node */
-    fdt_add_pch_pic_node(lvms, &eiointc_phandle, &pch_pic_phandle);
+        /* Connect pch_pic irqs to extioi */
+        for (i = 0; i < num; i++) {
+            qdev_connect_gpio_out(DEVICE(d), i, qdev_get_gpio_in(extioi, i));
+        }
 
-    pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
-    start   =  num;
-    num = EXTIOI_IRQS - start;
-    qdev_prop_set_uint32(pch_msi, "msi_irq_base", start);
-    qdev_prop_set_uint32(pch_msi, "msi_irq_num", num);
-    d = SYS_BUS_DEVICE(pch_msi);
-    sysbus_realize_and_unref(d, &error_fatal);
-    sysbus_mmio_map(d, 0, VIRT_PCH_MSI_ADDR_LOW);
-    for (i = 0; i < num; i++) {
-        /* Connect pch_msi irqs to extioi */
-        qdev_connect_gpio_out(DEVICE(d), i,
-                              qdev_get_gpio_in(extioi, i + start));
+        /* Add PCH PIC node */
+        fdt_add_pch_pic_node(lvms, &eiointc_phandle, &pch_pic_phandle);
+
+        pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
+        start   =  num;
+        num = EXTIOI_IRQS - start;
+        qdev_prop_set_uint32(pch_msi, "msi_irq_base", start);
+        qdev_prop_set_uint32(pch_msi, "msi_irq_num", num);
+        d = SYS_BUS_DEVICE(pch_msi);
+        sysbus_realize_and_unref(d, &error_fatal);
+        sysbus_mmio_map(d, 0, VIRT_PCH_MSI_ADDR_LOW);
+        for (i = 0; i < num; i++) {
+            /* Connect pch_msi irqs to extioi */
+            qdev_connect_gpio_out(DEVICE(d), i,
+                                  qdev_get_gpio_in(extioi, i + start));
+        }
      }
 
      /* Add PCH MSI node */
#10 lore raw

RFC PATCH v2 5/5 hw/loongarch: Add KVM pch msi device support

In reply to Xianglai Li

Added pch_msi interrupt controller handling during kernel emulation of irq chip.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

 hw/intc/loongarch_pch_msi.c | 42 +++++++++++++++++++++++++++----------
 hw/loongarch/virt.c         | 26 +++++++++++++----------
 target/loongarch/kvm/kvm.c  |  1 -
 3 files changed, 46 insertions(+), 23 deletions(-)
hw/intc/loongarch_pch_msi.c +31 -11
index ecf3ed0267..bab6f852f8 100644
@@ -2,7 +2,7 @@
 /*
  * QEMU Loongson 7A1000 msi interrupt controller.
  *
- * Copyright (C) 2021 Loongson Technology Corporation Limited
+ * Copyright (C) 2024 Loongson Technology Corporation Limited
  */
 
 #include "qemu/osdep.h"
@@ -14,6 +14,8 @@
 #include "hw/misc/unimp.h"
 #include "migration/vmstate.h"
 #include "trace.h"
+#include "sysemu/kvm.h"
+#include "hw/loongarch/virt.h"
 
 static uint64_t loongarch_msi_mem_read(void *opaque, hwaddr addr, unsigned size)
 {
@@ -26,14 +28,24 @@ static void loongarch_msi_mem_write(void *opaque, hwaddr addr,
     LoongArchPCHMSI *s = (LoongArchPCHMSI *)opaque;
     int irq_num;
 
-    /*
-     * vector number is irq number from upper extioi intc
-     * need subtract irq base to get msi vector offset
-     */
-    irq_num = (val & 0xff) - s->irq_base;
-    trace_loongarch_msi_set_irq(irq_num);
-    assert(irq_num < s->irq_num);
-    qemu_set_irq(s->pch_msi_irq[irq_num], 1);
+    MSIMessage msg = {
+        .address = addr,
+        .data = val,
+    };
+
+    if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+        kvm_irqchip_send_msi(kvm_state, msg);
+    } else {
+        /*
+         * vector number is irq number from upper extioi intc
+         * need subtract irq base to get msi vector offset
+         */
+        irq_num = (val & 0xff) - s->irq_base;
+        trace_loongarch_msi_set_irq(irq_num);
+        assert(irq_num < s->irq_num);
+
+        qemu_set_irq(s->pch_msi_irq[irq_num], 1);
+    }
 }
 
 static const MemoryRegionOps loongarch_pch_msi_ops = {
@@ -45,8 +57,16 @@ static const MemoryRegionOps loongarch_pch_msi_ops = {
 static void pch_msi_irq_handler(void *opaque, int irq, int level)
 {
     LoongArchPCHMSI *s = LOONGARCH_PCH_MSI(opaque);
-
-    qemu_set_irq(s->pch_msi_irq[irq], level);
+    MSIMessage msg = {
+        .address = 0,
+        .data = irq,
+    };
+
+    if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+        kvm_irqchip_send_msi(kvm_state, msg);
+    } else {
+        qemu_set_irq(s->pch_msi_irq[irq], level);
+    }
 }
 
 static void loongarch_pch_msi_realize(DeviceState *dev, Error **errp)
hw/loongarch/virt.c +15 -11
index db0c08899b..b42cf7e5af 100644
@@ -887,24 +887,28 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
         for (i = 0; i < num; i++) {
             qdev_connect_gpio_out(DEVICE(d), i, qdev_get_gpio_in(extioi, i));
         }
+    }
 
-        /* Add PCH PIC node */
-        fdt_add_pch_pic_node(lvms, &eiointc_phandle, &pch_pic_phandle);
+    /* Add PCH PIC node */
+    fdt_add_pch_pic_node(lvms, &eiointc_phandle, &pch_pic_phandle);
 
-        pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
-        start   =  num;
-        num = EXTIOI_IRQS - start;
-        qdev_prop_set_uint32(pch_msi, "msi_irq_base", start);
-        qdev_prop_set_uint32(pch_msi, "msi_irq_num", num);
-        d = SYS_BUS_DEVICE(pch_msi);
-        sysbus_realize_and_unref(d, &error_fatal);
-        sysbus_mmio_map(d, 0, VIRT_PCH_MSI_ADDR_LOW);
+    pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
+    num = VIRT_PCH_PIC_IRQ_NUM;
+    start   =  num;
+    num = EXTIOI_IRQS - start;
+    qdev_prop_set_uint32(pch_msi, "msi_irq_base", start);
+    qdev_prop_set_uint32(pch_msi, "msi_irq_num", num);
+    d = SYS_BUS_DEVICE(pch_msi);
+    sysbus_realize_and_unref(d, &error_fatal);
+
+    if (!(kvm_enabled() && kvm_irqchip_in_kernel())) {
+        /* Connect pch_msi irqs to extioi */
         for (i = 0; i < num; i++) {
-            /* Connect pch_msi irqs to extioi */
             qdev_connect_gpio_out(DEVICE(d), i,
                                   qdev_get_gpio_in(extioi, i + start));
         }
     }
+    sysbus_mmio_map(d, 0, VIRT_PCH_MSI_ADDR_LOW);
 
     /* Add PCH MSI node */
     fdt_add_pch_msi_node(lvms, &eiointc_phandle, &pch_msi_phandle);
target/loongarch/kvm/kvm.c +0 -1
index c07dcfd85f..e1be6a6959 100644
@@ -719,7 +719,6 @@ int kvm_arch_get_default_type(MachineState *ms)
 
 int kvm_arch_init(MachineState *ms, KVMState *s)
 {
-    s->kernel_irqchip_allowed = false;
     cap_has_mp_state = kvm_check_extension(s, KVM_CAP_MP_STATE);
     return 0;
 }
Signature · 1 line
2.39.1
#11 lore raw

Re: [RFC PATCH V2 5/5] hw/loongarch: Add KVM pch msi device support

Hi Xianglai,

I do not find any usage about function kvm_irqchip_commit_routes() in your patch-set, do I miss something?

Regards
Bibo Mao

Quoted 151 lines · Xianglai Li

Added pch_msi interrupt controller handling during kernel emulation of irq chip.

---
Cc: Paolo Bonzini <[email protected]>
Cc: Song Gao <[email protected]>
Cc: Jiaxun Yang <[email protected]>
Cc: Huacai Chen <[email protected]>
Cc: "Michael S. Tsirkin" <[email protected]>
Cc: Cornelia Huck <[email protected]>
Cc: [email protected]
Cc: Bibo Mao <[email protected]>
Cc: Xianglai Li <[email protected]>

  hw/intc/loongarch_pch_msi.c | 42 +++++++++++++++++++++++++++----------
  hw/loongarch/virt.c         | 26 +++++++++++++----------
  target/loongarch/kvm/kvm.c  |  1 -
  3 files changed, 46 insertions(+), 23 deletions(-)
hw/intc/loongarch_pch_msi.c +31 -11
index ecf3ed0267..bab6f852f8 100644
@@ -2,7 +2,7 @@
  /*
   * QEMU Loongson 7A1000 msi interrupt controller.
   *
- * Copyright (C) 2021 Loongson Technology Corporation Limited
+ * Copyright (C) 2024 Loongson Technology Corporation Limited
   */
 
  #include "qemu/osdep.h"
@@ -14,6 +14,8 @@
  #include "hw/misc/unimp.h"
  #include "migration/vmstate.h"
  #include "trace.h"
+#include "sysemu/kvm.h"
+#include "hw/loongarch/virt.h"
 
  static uint64_t loongarch_msi_mem_read(void *opaque, hwaddr addr, unsigned size)
  {
@@ -26,14 +28,24 @@ static void loongarch_msi_mem_write(void *opaque, hwaddr addr,
      LoongArchPCHMSI *s = (LoongArchPCHMSI *)opaque;
      int irq_num;
 
-    /*
-     * vector number is irq number from upper extioi intc
-     * need subtract irq base to get msi vector offset
-     */
-    irq_num = (val & 0xff) - s->irq_base;
-    trace_loongarch_msi_set_irq(irq_num);
-    assert(irq_num < s->irq_num);
-    qemu_set_irq(s->pch_msi_irq[irq_num], 1);
+    MSIMessage msg = {
+        .address = addr,
+        .data = val,
+    };
+
+    if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+        kvm_irqchip_send_msi(kvm_state, msg);
+    } else {
+        /*
+         * vector number is irq number from upper extioi intc
+         * need subtract irq base to get msi vector offset
+         */
+        irq_num = (val & 0xff) - s->irq_base;
+        trace_loongarch_msi_set_irq(irq_num);
+        assert(irq_num < s->irq_num);
+
+        qemu_set_irq(s->pch_msi_irq[irq_num], 1);
+    }
  }
 
  static const MemoryRegionOps loongarch_pch_msi_ops = {
@@ -45,8 +57,16 @@ static const MemoryRegionOps loongarch_pch_msi_ops = {
  static void pch_msi_irq_handler(void *opaque, int irq, int level)
  {
      LoongArchPCHMSI *s = LOONGARCH_PCH_MSI(opaque);
-
-    qemu_set_irq(s->pch_msi_irq[irq], level);
+    MSIMessage msg = {
+        .address = 0,
+        .data = irq,
+    };
+
+    if (kvm_enabled() && kvm_irqchip_in_kernel()) {
+        kvm_irqchip_send_msi(kvm_state, msg);
+    } else {
+        qemu_set_irq(s->pch_msi_irq[irq], level);
+    }
  }
 
  static void loongarch_pch_msi_realize(DeviceState *dev, Error **errp)
hw/loongarch/virt.c +15 -11
index db0c08899b..b42cf7e5af 100644
@@ -887,24 +887,28 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
          for (i = 0; i < num; i++) {
              qdev_connect_gpio_out(DEVICE(d), i, qdev_get_gpio_in(extioi, i));
          }
+    }
 
-        /* Add PCH PIC node */
-        fdt_add_pch_pic_node(lvms, &eiointc_phandle, &pch_pic_phandle);
+    /* Add PCH PIC node */
+    fdt_add_pch_pic_node(lvms, &eiointc_phandle, &pch_pic_phandle);
 
-        pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
-        start   =  num;
-        num = EXTIOI_IRQS - start;
-        qdev_prop_set_uint32(pch_msi, "msi_irq_base", start);
-        qdev_prop_set_uint32(pch_msi, "msi_irq_num", num);
-        d = SYS_BUS_DEVICE(pch_msi);
-        sysbus_realize_and_unref(d, &error_fatal);
-        sysbus_mmio_map(d, 0, VIRT_PCH_MSI_ADDR_LOW);
+    pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
+    num = VIRT_PCH_PIC_IRQ_NUM;
+    start   =  num;
+    num = EXTIOI_IRQS - start;
+    qdev_prop_set_uint32(pch_msi, "msi_irq_base", start);
+    qdev_prop_set_uint32(pch_msi, "msi_irq_num", num);
+    d = SYS_BUS_DEVICE(pch_msi);
+    sysbus_realize_and_unref(d, &error_fatal);
+
+    if (!(kvm_enabled() && kvm_irqchip_in_kernel())) {
+        /* Connect pch_msi irqs to extioi */
          for (i = 0; i < num; i++) {
-            /* Connect pch_msi irqs to extioi */
              qdev_connect_gpio_out(DEVICE(d), i,
                                    qdev_get_gpio_in(extioi, i + start));
          }
      }
+    sysbus_mmio_map(d, 0, VIRT_PCH_MSI_ADDR_LOW);
 
      /* Add PCH MSI node */
      fdt_add_pch_msi_node(lvms, &eiointc_phandle, &pch_msi_phandle);
target/loongarch/kvm/kvm.c +0 -1
index c07dcfd85f..e1be6a6959 100644
@@ -719,7 +719,6 @@ int kvm_arch_get_default_type(MachineState *ms)
 
  int kvm_arch_init(MachineState *ms, KVMState *s)
  {
-    s->kernel_irqchip_allowed = false;
      cap_has_mp_state = kvm_check_extension(s, KVM_CAP_MP_STATE);
      return 0;
  }