Skip to contentSkip to conversation

GIC-500 implementation, software + KVM

PATCH In review Pavel Fedin 5 participants 28 messages Jul 2, 2015 – Jul 7, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v4 cover GIC-500 implementation, software + KVM

This is a complete GICv3 implementation, both software emulation and KVM acceleration are supported.

This series is a consolidated and updated patch set, based on:
- GIC-500 implementation:
http://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg01512.html
- vGICv3 implementation:
https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg04496.html
I decided to repost because qemu development has progressed and it became difficult to apply those series to current master.

Changes from previous versions:
- Removed RFC prefix
- Base class separated from the rest, as was requested in RFC v2 review
- Fixed small number of broken comments / code formatting issues, according to the same review
- Removed #if 0 in virt.c
- vGICv3 patch set restructured and more clearly separated into portions, according to old Eric's review
- Removed duplication of low-level vGICv3 code. Common helper routines are used instead.
- Put informative commit messages

Pavel Fedin (6):
Add virt-v3 machine that uses GIC-500
Extract some reusable vGIC code
Set kernel_irqchip_type for the rest of ARM boards which use GIC Make use of kernel_irqchip_type in kvm_arch_irqchip_create() Initial implementation of vGICv3
Enable KVM acceleration for GICv3

Shlomo Pongratz (3):
Implement GIC-500 base class
Implement GIC-500
GICv3 support

 hw/arm/exynos4_boards.c            |    1 +
 hw/arm/realview.c                  |    1 +
 hw/arm/vexpress.c                  |    1 +
 hw/arm/virt.c                      |  141 ++-
 hw/intc/Makefile.objs              |    3 +
 hw/intc/arm_gic_kvm.c              |   84 +-
 hw/intc/arm_gicv3.c                | 2086 ++++++++++++++++++++++++++++++++++++
 hw/intc/arm_gicv3_common.c         |  216 ++++
 hw/intc/arm_gicv3_kvm.c            |  203 ++++
 hw/intc/gicv3_internal.h           |  159 +++
 hw/intc/vgic_common.h              |   43 +
 include/hw/arm/fdt.h               |    2 +-
 include/hw/arm/virt.h              |    6 +-
 include/hw/boards.h                |    1 +
 include/hw/intc/arm_gicv3.h        |   44 +
 include/hw/intc/arm_gicv3_common.h |  116 ++
 include/sysemu/kvm.h               |    3 +-
 kvm-all.c                          |    2 +-
 stubs/kvm.c                        |    2 +-
 target-arm/cpu.h                   |   12 +
 target-arm/cpu64.c                 |  105 ++
 target-arm/kvm.c                   |    8 +-
 22 files changed, 3164 insertions(+), 75 deletions(-)
 create mode 100644 hw/intc/arm_gicv3.c
 create mode 100644 hw/intc/arm_gicv3_common.c
 create mode 100644 hw/intc/arm_gicv3_kvm.c
 create mode 100644 hw/intc/gicv3_internal.h
 create mode 100644 hw/intc/vgic_common.h
 create mode 100644 include/hw/intc/arm_gicv3.h
 create mode 100644 include/hw/intc/arm_gicv3_common.h
Signature · 1 line
1.9.5.msysgit.0
#1 lore raw

PATCH v4 1/9 Implement GIC-500 base class

From: Shlomo Pongratz <[email protected]>

This class is to be used by both software and KVM implementations of GICv3

---
 hw/intc/Makefile.objs              |   1 +
 hw/intc/arm_gicv3_common.c         | 216 +++++++++++++++++++++++++++++++++++++
 hw/intc/gicv3_internal.h           | 159 +++++++++++++++++++++++++++
 include/hw/intc/arm_gicv3.h        |  44 ++++++++
 include/hw/intc/arm_gicv3_common.h | 116 ++++++++++++++++++++
 5 files changed, 536 insertions(+)
 create mode 100644 hw/intc/arm_gicv3_common.c
 create mode 100644 hw/intc/gicv3_internal.h
 create mode 100644 include/hw/intc/arm_gicv3.h
 create mode 100644 include/hw/intc/arm_gicv3_common.h
hw/intc/Makefile.objs +1 -0
index 092d8a8..1317e5a 100644
@@ -12,6 +12,7 @@ common-obj-$(CONFIG_IOAPIC) += ioapic_common.o
 common-obj-$(CONFIG_ARM_GIC) += arm_gic_common.o
 common-obj-$(CONFIG_ARM_GIC) += arm_gic.o
 common-obj-$(CONFIG_ARM_GIC) += arm_gicv2m.o
+common-obj-$(CONFIG_ARM_GIC) += arm_gicv3_common.o
 common-obj-$(CONFIG_OPENPIC) += openpic.o
 
 obj-$(CONFIG_APIC) += apic.o apic_common.o
hw/intc/arm_gicv3_common.c new file +216 -0

216 lines not on this page · raw message

hw/intc/gicv3_internal.h new file +159 -0

159 lines not on this page · raw message

include/hw/intc/arm_gicv3.h new file +44 -0

44 lines not on this page · raw message

include/hw/intc/arm_gicv3_common.h new file +116 -0

116 lines not on this page · raw message

Signature · 1 line
1.9.5.msysgit.0
#2 lore raw

PATCH v4 2/9 Implement GIC-500

In reply to Pavel Fedin

From: Shlomo Pongratz <[email protected]>

Implement GIC-500 from GICv3 family for arm64

This patch is a first step toward 128 cores support for arm64.

At first only 64 cores are supported for two reasons:
First the largest integer type has the size of 64 bits and modifying essential data structures in order to support 128 cores will require the usage of bitops.
Second currently the Linux (kernel) can be configured to support up to 64 cores thus there is no urgency with 128 cores support.

---
 hw/intc/Makefile.objs |    1 +
 hw/intc/arm_gicv3.c   | 2086 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 2087 insertions(+)
 create mode 100644 hw/intc/arm_gicv3.c
hw/intc/Makefile.objs +1 -0
index 1317e5a..6c2b136 100644
@@ -13,6 +13,7 @@ common-obj-$(CONFIG_ARM_GIC) += arm_gic_common.o
 common-obj-$(CONFIG_ARM_GIC) += arm_gic.o
 common-obj-$(CONFIG_ARM_GIC) += arm_gicv2m.o
 common-obj-$(CONFIG_ARM_GIC) += arm_gicv3_common.o
+common-obj-$(CONFIG_ARM_GIC) += arm_gicv3.o
 common-obj-$(CONFIG_OPENPIC) += openpic.o
 
 obj-$(CONFIG_APIC) += apic.o apic_common.o
hw/intc/arm_gicv3.c new file +2086 -0

2,086 lines not on this page · raw message

Signature · 1 line
1.9.5.msysgit.0
#3 lore raw

Re: [Qemu-devel] [PATCH v4 2/9] Implement GIC-500

Quoted 13 lines · Pavel Fedin

From: Shlomo Pongratz <[email protected]>

Implement GIC-500 from GICv3 family for arm64

This patch is a first step toward 128 cores support for arm64.

At first only 64 cores are supported for two reasons:
First the largest integer type has the size of 64 bits and modifying essential data structures in order to support 128 cores will require the usage of bitops.
Second currently the Linux (kernel) can be configured to support up to 64 cores thus there is no urgency with 128 cores support.

Quoted 6 lines
---
 hw/intc/Makefile.objs |    1 +
 hw/intc/arm_gicv3.c   | 2086 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 2087 insertions(+)
 create mode 100644 hw/intc/arm_gicv3.c

Hi. I'm afraid 2000 lines is really far too many to review as a single patch. Please split this one up into more digestible chunks.

thanks
-- PMM

#4 lore raw

Re: [Qemu-devel] [PATCH v4 2/9] Implement GIC-500

Hello!

Quoted 3 lines

Hi. I'm afraid 2000 lines is really far too many to review as a single patch. Please split this one up into more digestible chunks.

Huh... You're fast... But not fair... :( You never told the original author to do this: http://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg01516.html
I am sorry, i really cannot do much here, it's even not my code and to tell the truth i don't have enough time and knowledge to support it. This file is absolutely the same as in Shlomo's RFC v3, i haven't changed any single code line. And, it actually contains only a single added file by now. If you really dislike it, you can simply leave parts 2 and 3 out, because the rest (KVM vGICv3) does not depend on it in any way. I included it only in order to provide the complete solution.

Shlomo? Your turn...

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

#5 lore raw

Re: [Qemu-devel] [PATCH v4 2/9] Implement GIC-500

Quoted 8 lines · Pavel Fedin

Hello!

Quoted 3 lines

Hi. I'm afraid 2000 lines is really far too many to review as a single patch. Please split this one up into more digestible chunks.

Huh... You're fast... But not fair... :( You never told the original
author to do this:

I didn't review the original author's patchset either.

Quoted 5 lines

I am sorry, i really cannot do much here, it's even not my code
and to tell the truth i don't have enough time and knowledge to support it. This file is absolutely the same as in Shlomo's RFC v3, i haven't changed any single code line. And, it actually contains only a single added file by now.

If nobody splits this patch up then it is not going to get reviewed, and so it is not going into master. I don't really mind who does it, but somebody is going to have to...

thanks
-- PMM

#6 lore raw

Re: [Qemu-devel] [PATCH v4 2/9] Implement GIC-500

Quoted 11 lines · Peter Maydell
Quoted 8 lines · Pavel Fedin

Hello!

Quoted 3 lines

Hi. I'm afraid 2000 lines is really far too many to review as a single patch. Please split this one up into more digestible chunks.

Huh... You're fast... But not fair... :( You never told the original
author to do this:

I didn't review the original author's patchset either.

...and in fact I *did* ask Shlomo to split up the patch:
https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg01621.html

thanks
-- PMM

#7 lore raw

PATCH v4 3/9 GICv3 support

In reply to Pavel Fedin

From: Shlomo Pongratz <[email protected]>

Add system instructions used by the Linux (kernel) GICv3 device driver

---
 target-arm/cpu.h   |  12 ++++++
 target-arm/cpu64.c | 105 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 117 insertions(+)
target-arm/cpu.h +12 -0
index 80297b3..c04b54c 100644
@@ -1024,6 +1024,18 @@ void armv7m_nvic_set_pending(void *opaque, int irq);
 int armv7m_nvic_acknowledge_irq(void *opaque);
 void armv7m_nvic_complete_irq(void *opaque, int irq);
 
+void armv8_gicv3_set_sgi(void *opaque, int cpuindex, uint64_t value);
+uint64_t armv8_gicv3_acknowledge_irq(void *opaque, int cpuindex,
+                              MemTxAttrs attrs);
+void armv8_gicv3_complete_irq(void *opaque, int cpuindex, int irq,
+                              MemTxAttrs attrs);
+uint64_t armv8_gicv3_get_priority_mask(void *opaque, int cpuindex);
+void armv8_gicv3_set_priority_mask(void *opaque, int cpuindex, uint32_t mask);
+uint64_t armv8_gicv3_get_sre(void *opaque);
+void armv8_gicv3_set_sre(void *opaque, uint64_t sre);
+uint64_t armv8_gicv3_get_igrpen1(void *opaque, int cpuindex);
+void armv8_gicv3_set_igrpen1(void *opaque, int cpuindex, uint64_t igrpen1);
+
 /* Interface for defining coprocessor registers.
  * Registers are defined in tables of arm_cp_reginfo structs
  * which are passed to define_arm_cp_regs().
target-arm/cpu64.c +105 -0

117 lines not on this page · raw message

Signature · 1 line
1.9.5.msysgit.0
#8 lore raw

PATCH v4 4/9 Add virt-v3 machine that uses GIC-500

In reply to Pavel Fedin

Instead of adding gic-version option, it is much easier to create a new machine type. The problem is mc->max_cpus. I tried to change this value inside property handling code, but it simply did not work, and i still got " Maximum CPUs greater than specified machine type limit" error from qemu. It looks like CPU number limitation is evaluated before machine is instantiated and properties are evaluated. It is possible to make a workaround, of course, but it seems to require much more changes than simply subclassing a machine.

This patch also introduces kernel_irqchip_type member in Machine class. Currently it it used only by virt machine for its internal purposes, however in future it is to be passed to KVM in kvm_irqchip_create(). The variable is defined as int in order to be architecture agnostic, for potential future users.

---
 hw/arm/virt.c         | 141 ++++++++++++++++++++++++++++++++++++++++++--------
 include/hw/arm/fdt.h  |   2 +-
 include/hw/arm/virt.h |   6 ++-
 include/hw/boards.h   |   1 +
 4 files changed, 126 insertions(+), 24 deletions(-)
hw/arm/virt.c +119 -20
index 4846892..5c07d07 100644
@@ -48,6 +48,7 @@
 #include "hw/arm/sysbus-fdt.h"
 #include "hw/platform-bus.h"
 #include "hw/arm/fdt.h"
+#include "linux/kvm.h" /* For KVM_DEV_TYPE_ARM_VGIC_V{2|3} */
 
 /* Number of external interrupt lines to configure the GIC with */
 #define NUM_IRQS 256
@@ -67,6 +68,7 @@ typedef struct VirtBoardInfo {
     uint32_t clock_phandle;
     uint32_t gic_phandle;
     uint32_t v2m_phandle;
+    const char *class_name;
 } VirtBoardInfo;
 
 typedef struct {
@@ -80,6 +82,7 @@ typedef struct {
 } VirtMachineState;
 
 #define TYPE_VIRT_MACHINE   "virt"
+#define TYPE_VIRTV3_MACHINE   "virt-v3"
 #define VIRT_MACHINE(obj) \
     OBJECT_CHECK(VirtMachineState, (obj), TYPE_VIRT_MACHINE)
 #define VIRT_MACHINE_GET_CLASS(obj) \
@@ -106,7 +109,12 @@ static const MemMapEntry a15memmap[] = {
     /* GIC distributor and CPU interfaces sit inside the CPU peripheral space */
     [VIRT_GIC_DIST] =           { 0x08000000, 0x00010000 },
     [VIRT_GIC_CPU] =            { 0x08010000, 0x00010000 },
-    [VIRT_GIC_V2M] =            { 0x08020000, 0x00001000 },
+    [VIRT_GIC_V2M] =            { 0x08020000, 0x00010000 },
+    /* On v3 VIRT_GIC_DIST_MBI and VIRT_ITS_CONTROL take place of
+     * VIRT_GIC_CPU and VIRT_GIC_V2M respectively
+     */
+    [VIRT_ITS_TRANSLATION] =    { 0x08030000, 0x00010000 },
+    [VIRT_LPI] =                { 0x08040000, 0x00800000 },
     [VIRT_UART] =               { 0x09000000, 0x00001000 },
     [VIRT_RTC] =                { 0x09010000, 0x00001000 },
     [VIRT_FW_CFG] =             { 0x09020000, 0x0000000a },
@@ -138,16 +146,25 @@ static VirtBoardInfo machines[] = {
         .cpu_model = "cortex-a53",
         .memmap = a15memmap,
         .irqmap = a15irqmap,
+        .class_name = TYPE_ARM_CPU,
+    },
+    {
+        .cpu_model = "cortex-a53",
+        .memmap = a15memmap,
+        .irqmap = a15irqmap,
+        .class_name = TYPE_AARCH64_CPU,
     },
     {
         .cpu_model = "cortex-a57",
         .memmap = a15memmap,
         .irqmap = a15irqmap,
+        .class_name = TYPE_AARCH64_CPU,
     },
     {
         .cpu_model = "host",
         .memmap = a15memmap,
         .irqmap = a15irqmap,
+        .class_name = TYPE_ARM_CPU,
     },
 };
 
@@ -256,10 +271,13 @@ static void fdt_add_timer_nodes(const VirtBoardInfo *vbi)
      * they are edge-triggered.
      */
     ARMCPU *armcpu;
+    uint32_t max;
     uint32_t irqflags = GIC_FDT_IRQ_FLAGS_EDGE_LO_HI;
 
+    /* Argument is 32 bit but 8 bits are reserved for flags */
+    max = (vbi->smp_cpus >= 24) ? 24 : vbi->smp_cpus;
     irqflags = deposit32(irqflags, GIC_FDT_IRQ_PPI_CPU_START,
-                         GIC_FDT_IRQ_PPI_CPU_WIDTH, (1 << vbi->smp_cpus) - 1);
+                         GIC_FDT_IRQ_PPI_CPU_WIDTH, (1 << max) - 1);
 
     qemu_fdt_add_subnode(vbi->fdt, "/timer");
 
@@ -283,6 +301,18 @@ static void fdt_add_cpu_nodes(const VirtBoardInfo *vbi)
 {
     int cpu;
 
+    /*
+     * From Documentation/devicetree/bindings/arm/cpus.txt
+     *  On ARM v8 64-bit systems value should be set to 2,
+     *  that corresponds to the MPIDR_EL1 register size.
+     *  If MPIDR_EL1[63:32] value is equal to 0 on all CPUs
+     *  in the system, #address-cells can be set to 1, since
+     *  MPIDR_EL1[63:32] bits are not used for CPUs
+     *  identification.
+     *
+     *  Now GIC500 doesn't support affinities 2 & 3 so currently
+     *  #address-cells can stay 1 until future GIC
+     */
     qemu_fdt_add_subnode(vbi->fdt, "/cpus");
     qemu_fdt_setprop_cell(vbi->fdt, "/cpus", "#address-cells", 0x1);
     qemu_fdt_setprop_cell(vbi->fdt, "/cpus", "#size-cells", 0x0);
@@ -319,25 +349,36 @@ static void fdt_add_v2m_gic_node(VirtBoardInfo *vbi)
     qemu_fdt_setprop_cell(vbi->fdt, "/intc/v2m", "phandle", vbi->v2m_phandle);
 }
 
-static void fdt_add_gic_node(VirtBoardInfo *vbi)
+static void fdt_add_gic_node(VirtBoardInfo *vbi, int type)
 {
     vbi->gic_phandle = qemu_fdt_alloc_phandle(vbi->fdt);
     qemu_fdt_setprop_cell(vbi->fdt, "/", "interrupt-parent", vbi->gic_phandle);
 
     qemu_fdt_add_subnode(vbi->fdt, "/intc");
-    /* 'cortex-a15-gic' means 'GIC v2' */
-    qemu_fdt_setprop_string(vbi->fdt, "/intc", "compatible",
-                            "arm,cortex-a15-gic");
     qemu_fdt_setprop_cell(vbi->fdt, "/intc", "#interrupt-cells", 3);
     qemu_fdt_setprop(vbi->fdt, "/intc", "interrupt-controller", NULL, 0);
-    qemu_fdt_setprop_sized_cells(vbi->fdt, "/intc", "reg",
-                                     2, vbi->memmap[VIRT_GIC_DIST].base,
-                                     2, vbi->memmap[VIRT_GIC_DIST].size,
-                                     2, vbi->memmap[VIRT_GIC_CPU].base,
-                                     2, vbi->memmap[VIRT_GIC_CPU].size);
     qemu_fdt_setprop_cell(vbi->fdt, "/intc", "#address-cells", 0x2);
     qemu_fdt_setprop_cell(vbi->fdt, "/intc", "#size-cells", 0x2);
     qemu_fdt_setprop(vbi->fdt, "/intc", "ranges", NULL, 0);
+    if (type == KVM_DEV_TYPE_ARM_VGIC_V3) {
+        qemu_fdt_setprop_string(vbi->fdt, "/intc", "compatible",
+                                "arm,gic-v3");
+        qemu_fdt_setprop_sized_cells(vbi->fdt, "/intc", "reg",
+                                     2, vbi->memmap[VIRT_GIC_DIST].base,
+                                     2, vbi->memmap[VIRT_GIC_DIST].size,
+                                     2, vbi->memmap[VIRT_LPI].base,
+                                     2, vbi->memmap[VIRT_LPI].size);
+    } else {
+        /* 'cortex-a15-gic' means 'GIC v2' */
+        qemu_fdt_setprop_string(vbi->fdt, "/intc", "compatible",
+                                "arm,cortex-a15-gic");
+        qemu_fdt_setprop_sized_cells(vbi->fdt, "/intc", "reg",
+                                      2, vbi->memmap[VIRT_GIC_DIST].base,
+                                      2, vbi->memmap[VIRT_GIC_DIST].size,
+                                      2, vbi->memmap[VIRT_GIC_CPU].base,
+                                      2, vbi->memmap[VIRT_GIC_CPU].size);
+    }
+ 
     qemu_fdt_setprop_cell(vbi->fdt, "/intc", "phandle", vbi->gic_phandle);
 }
 
@@ -360,20 +401,32 @@ static void create_v2m(VirtBoardInfo *vbi, qemu_irq *pic)
     fdt_add_v2m_gic_node(vbi);
 }
 
-static void create_gic(VirtBoardInfo *vbi, qemu_irq *pic)
+static void create_gic(VirtBoardInfo *vbi, qemu_irq *pic, int type)
 {
     /* We create a standalone GIC v2 */
     DeviceState *gicdev;
     SysBusDevice *gicbusdev;
-    const char *gictype = "arm_gic";
+    const char *gictype;
     int i;
 
-    if (kvm_irqchip_in_kernel()) {
+    if (type == KVM_DEV_TYPE_ARM_VGIC_V3) {
+        gictype = "arm_gicv3";
+    } else if (kvm_irqchip_in_kernel()) {
         gictype = "kvm-arm-gic";
+    } else {
+        gictype = "arm_gic";
     }
 
     gicdev = qdev_create(NULL, gictype);
-    qdev_prop_set_uint32(gicdev, "revision", 2);
+ 
+    for (i = 0; i < vbi->smp_cpus; i++) {
+        CPUState *cpu = qemu_get_cpu(i);
+        CPUARMState *env = cpu->env_ptr;
+        env->nvic = gicdev;
+    }
+
+    qdev_prop_set_uint32(gicdev, "revision",
+                         type == KVM_DEV_TYPE_ARM_VGIC_V3 ? 3 : 2);
     qdev_prop_set_uint32(gicdev, "num-cpu", smp_cpus);
     /* Note that the num-irq property counts both internal and external
      * interrupts; there are always 32 of the former (mandated by GIC spec).
@@ -383,6 +436,11 @@ static void create_gic(VirtBoardInfo *vbi, qemu_irq *pic)
     gicbusdev = SYS_BUS_DEVICE(gicdev);
     sysbus_mmio_map(gicbusdev, 0, vbi->memmap[VIRT_GIC_DIST].base);
     sysbus_mmio_map(gicbusdev, 1, vbi->memmap[VIRT_GIC_CPU].base);
+    if (type == KVM_DEV_TYPE_ARM_VGIC_V3) {
+        sysbus_mmio_map(gicbusdev, 2, vbi->memmap[VIRT_ITS_CONTROL].base);
+        sysbus_mmio_map(gicbusdev, 3, vbi->memmap[VIRT_ITS_TRANSLATION].base);
+        sysbus_mmio_map(gicbusdev, 4, vbi->memmap[VIRT_LPI].base);
+    }
 
     /* Wire the outputs from each CPU's generic timer to the
      * appropriate GIC PPI inputs, and the GIC's IRQ output to
@@ -409,9 +467,11 @@ static void create_gic(VirtBoardInfo *vbi, qemu_irq *pic)
         pic[i] = qdev_get_gpio_in(gicdev, i);
     }
 
-    fdt_add_gic_node(vbi);
+    fdt_add_gic_node(vbi, type);
 
-    create_v2m(vbi, pic);
+    if (type == KVM_DEV_TYPE_ARM_VGIC_V2) {
+        create_v2m(vbi, pic);
+    }
 }
 
 static void create_uart(const VirtBoardInfo *vbi, qemu_irq *pic)
@@ -825,7 +885,7 @@ static void machvirt_init(MachineState *machine)
     create_fdt(vbi);
 
     for (n = 0; n < smp_cpus; n++) {
-        ObjectClass *oc = cpu_class_by_name(TYPE_ARM_CPU, cpustr[0]);
+        ObjectClass *oc = cpu_class_by_name(vbi->class_name, cpustr[0]);
         CPUClass *cc = CPU_CLASS(oc);
         Object *cpuobj;
         Error *err = NULL;
@@ -875,7 +935,7 @@ static void machvirt_init(MachineState *machine)
 
     create_flash(vbi);
 
-    create_gic(vbi, pic);
+    create_gic(vbi, pic, machine->kernel_irqchip_type);
 
     create_uart(vbi, pic);
 
@@ -933,7 +993,7 @@ static void virt_set_secure(Object *obj, bool value, Error **errp)
     vms->secure = value;
 }
 
-static void virt_instance_init(Object *obj)
+static void virt_instance_init_common(Object *obj)
 {
     VirtMachineState *vms = VIRT_MACHINE(obj);
 
@@ -947,6 +1007,14 @@ static void virt_instance_init(Object *obj)
                                     NULL);
 }
 
+static void virt_instance_init(Object *obj)
+{
+    MachineState *ms = MACHINE(obj);
+
+    ms->kernel_irqchip_type = KVM_DEV_TYPE_ARM_VGIC_V2;
+    virt_instance_init_common(obj);
+}
+
 static void virt_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
@@ -969,9 +1037,38 @@ static const TypeInfo machvirt_info = {
     .class_init = virt_class_init,
 };
 
+static void virtv3_instance_init(Object *obj)
+{
+    MachineState *ms = MACHINE(obj);
+
+    ms->kernel_irqchip_type = KVM_DEV_TYPE_ARM_VGIC_V3;
+    virt_instance_init_common(obj);
+}
+
+static void virtv3_class_init(ObjectClass *oc, void *data)
+{
+    MachineClass *mc = MACHINE_CLASS(oc);
+
+    mc->name = TYPE_VIRTV3_MACHINE;
+    mc->desc = "ARM Virtual Machine with GICv3",
+    mc->init = machvirt_init;
+    /* With gic3 full implementation (with bitops) rase the lmit to 128 */
+    mc->max_cpus = 64;
+}
+
+static const TypeInfo machvirtv3_info = {
+    .name = TYPE_VIRTV3_MACHINE,
+    .parent = TYPE_VIRT_MACHINE,
+    .instance_size = sizeof(VirtMachineState),
+    .instance_init = virtv3_instance_init,
+    .class_size = sizeof(VirtMachineClass),
+    .class_init = virtv3_class_init,
+};
+
 static void machvirt_machine_init(void)
 {
     type_register_static(&machvirt_info);
+    type_register_static(&machvirtv3_info);
 }
 
 machine_init(machvirt_machine_init);
include/hw/arm/fdt.h +1 -1
index c3d5015..dd794dd 100644
@@ -29,6 +29,6 @@
 #define GIC_FDT_IRQ_FLAGS_LEVEL_LO 8
 
 #define GIC_FDT_IRQ_PPI_CPU_START 8
-#define GIC_FDT_IRQ_PPI_CPU_WIDTH 8
+#define GIC_FDT_IRQ_PPI_CPU_WIDTH 24
 
 #endif
include/hw/arm/virt.h +5 -1
index d22fd8e..852efb9 100644
@@ -46,6 +46,11 @@ enum {
     VIRT_CPUPERIPHS,
     VIRT_GIC_DIST,
     VIRT_GIC_CPU,
+    VIRT_GIC_V2M,
+    VIRT_GIC_DIST_MBI = VIRT_GIC_CPU,
+    VIRT_ITS_CONTROL = VIRT_GIC_V2M,
+    VIRT_ITS_TRANSLATION,
+    VIRT_LPI,
     VIRT_UART,
     VIRT_MMIO,
     VIRT_RTC,
@@ -54,7 +59,6 @@ enum {
     VIRT_PCIE_MMIO,
     VIRT_PCIE_PIO,
     VIRT_PCIE_ECAM,
-    VIRT_GIC_V2M,
     VIRT_PLATFORM_BUS,
 };
 
include/hw/boards.h +1 -0
index 6379901..6e42cf2 100644
@@ -126,6 +126,7 @@ struct MachineState {
     char *accel;
     bool kernel_irqchip_allowed;
     bool kernel_irqchip_required;
+    int kernel_irqchip_type;
     int kvm_shadow_mem;
     char *dtb;
     char *dumpdtb;
Signature · 1 line
1.9.5.msysgit.0
#9 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

Quoted 8 lines · Pavel Fedin

Instead of adding gic-version option, it is much easier to create a new machine type. The problem is mc->max_cpus. I tried to change this value inside property handling code, but it simply did not work, and i still got " Maximum CPUs greater than specified machine type limit" error from qemu. It looks like CPU number limitation is evaluated before machine is instantiated and properties are evaluated. It is possible to make a workaround, of course, but it seems to require much more changes than simply subclassing a machine.

Quoted 7 lines

This patch also introduces kernel_irqchip_type member in Machine class. Currently it it used only by virt machine for its internal purposes, however in future it is to be passed to KVM in kvm_irqchip_create(). The variable is defined as int in order to be architecture agnostic, for potential future users.

As mentioned in previous postings, this should be done as a property on the existing machine type, not a new machine type.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#10 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

Hello!

I already explained this earlier: http://lists.nongnu.org/archive/html/qemu-devel/2015-05/msg04842.html, and i tried to explain this in commit message. Current qemu architecture does not allow doing this in a clean way. I can simply change mc->max_cpus for 'virt' machine to 64 (always), and then check user-supplied value against GIC limitation by myself. And produce error. This will be code duplication. Do you think it is better? Anybody else (Peter, Cristoffer ?), please vote.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

#11 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

Quoted 6 lines · Pavel Fedin

Hello!

I already explained this earlier: http://lists.nongnu.org/archive/html/qemu-devel/2015-05/msg04842.html, and i tried to explain this in commit message. Current qemu architecture does not allow doing this in a clean way. I can simply change mc->max_cpus for 'virt' machine to 64 (always), and then check user-supplied value against GIC limitation by myself. And produce error. This will be code duplication. Do you think it is better? Anybody else (Peter, Cristoffer ?), please vote.

We definitely need a single virt machine, not multiple machines. I don't know enough about the QEMU internals as to how to accomplish this in detail.

-Christoffer

#12 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

Quoted 3 lines

We definitely need a single virt machine, not multiple machines. I don't know enough about the QEMU internals as to how to accomplish this in detail.

Two beat one... Okay, i surrender. :)
I will respin this, hopefully tomorrow. I will change to machine option, and i will likely leave out software GIC emulation. Everyone okay with this? Peter?

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

#13 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

Quoted 7 lines · Pavel Fedin
Quoted 3 lines

We definitely need a single virt machine, not multiple machines. I don't know enough about the QEMU internals as to how to accomplish this in detail.

Two beat one... Okay, i surrender. :)
I will respin this, hopefully tomorrow. I will change to machine option, and i will likely leave out software GIC emulation. Everyone okay with this? Peter?

I think you should leave out the software emulation part for this series and let's concentrate on getting this in first.

Also, you should make sure you have an agreement with Shlomo about taking over his patches before doing so (maybe you are already?).

Thanks,
-Christoffer

#14 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

Hi!

Quoted 2 lines

I think you should leave out the software emulation part for this series and let's concentrate on getting this in first.

Ok.

Quoted 2 lines

Also, you should make sure you have an agreement with Shlomo about taking over his patches before doing so (maybe you are already?).

Shlomo has posted his work here, doesn't this automatically mean that he is sharing it with the community? Additionally, i do not drop his authorship.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

#15 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

Quoted 5 lines · Pavel Fedin

Hi!

Quoted 2 lines

I think you should leave out the software emulation part for this series and let's concentrate on getting this in first.

Quoted 7 lines

Ok.

Quoted 2 lines

Also, you should make sure you have an agreement with Shlomo about taking over his patches before doing so (maybe you are already?).

Shlomo has posted his work here, doesn't this automatically mean that he is sharing it with the community? Additionally, i do not drop his authorship.

Well, it can become a little messy for everyone if Shlomo is working on refactoring and respinning his patches, and meanwhile new versions of his patches go out embedded as part of another patch set. It's not exactly harmful, but usually doesn't happen without prior agreement amongst the involved parties, I would say.

-Christoffer

#16 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

Hi!

Quoted 3 lines

Well, it can become a little messy for everyone if Shlomo is working on refactoring and respinning his patches, and meanwhile new versions of his patches go out embedded as part of another patch set

Exactly for this reason i refactor his code as little as possible, so that he can merge things easily, if he is around. But looks like he become upset and gone, we haven't seen him around for several weeks. Meanwhile i'll try to incorporate SW emulation too, because i also may need it. Just needs more time, and yes, ITS is a separate thing (by the way i even have the code, just without LPIs). So yes, i'll split it up a bit more.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

#17 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

Quoted 7 lines · Pavel Fedin

Hi!

Quoted 3 lines

Well, it can become a little messy for everyone if Shlomo is working on refactoring and respinning his patches, and meanwhile new versions of his patches go out embedded as part of another patch set

Exactly for this reason i refactor his code as little as possible, so that he can merge things easily, if he is around. But looks like he become upset and gone, we haven't seen him around for several weeks.

I don't think you should put words in other people's mouth. He may be on vacation or awaiting approval for submitting this stuff or something like that.

That said, Shlomo, what is your status?

Quoted 2 lines

Meanwhile i'll try to incorporate SW emulation too, because i also may need it. Just needs more time, and yes, ITS is a separate thing (by the way i even have the code, just without LPIs). So yes, i'll split it up a bit more.

I *really* think we should try to *agree* on who does what here, so that we don't have more copies of the same work etc. on the list.

-Christoffer

#18 lore raw

Re: [Qemu-devel] [PATCH v4 4/9] Add virt-v3 machine that uses GIC-500

In reply to Pavel Fedin
Quoted 7 lines · Pavel Fedin

Hello!

I already explained this earlier: http://lists.nongnu.org/archive/html/qemu-devel/2015-05/msg04842.html, and i tried to explain this in commit message. Current qemu architecture does not allow doing this in a clean way. I can simply change mc->max_cpus for 'virt' machine to 64 (always), and
then check user-supplied value against GIC limitation by myself. And
produce error. This will be code duplication. Do you think it is better?
Anybody else (Peter, Cristoffer ?), please vote.

Yes, just increase the declared max cpus to the larger of the two values, and do a manual check at time of use

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#19 lore raw

PATCH v4 5/9 Extract some reusable vGIC code

In reply to Pavel Fedin

These functions are useful also for vGICv3 implementation. Make them accessible from within other modules.

Actually kvm_dist_get() and kvm_dist_put() could also be made reusable, but they would require two extra parameters (s->dev_fd and s->num_cpu) as well as lots of typecasts of 's' to DeviceState * and back to GICState *. This makes the code very ugly so i decided to stop at this point. I tried also an approach with making a base class for all possible GICs, but it would contain only three variables (dev_fd, cpu_num and irq_num), and accessing them through the rest of the code would be again tedious (either ugly casts or qemu-style separate object pointer). So i disliked it too.

---
 hw/intc/arm_gic_kvm.c | 84 ++++++++++++++++++++++++---------------------------
 hw/intc/vgic_common.h | 43 ++++++++++++++++++++++++++
 2 files changed, 82 insertions(+), 45 deletions(-)
 create mode 100644 hw/intc/vgic_common.h
hw/intc/arm_gic_kvm.c +39 -45
index 2cb7d25..d1a4f6a 100644
@@ -23,6 +23,7 @@
 #include "sysemu/kvm.h"
 #include "kvm_arm.h"
 #include "gic_internal.h"
+#include "vgic_common.h"
 
 //#define DEBUG_GIC_KVM
 
@@ -52,7 +53,7 @@ typedef struct KVMARMGICClass {
     void (*parent_reset)(DeviceState *dev);
 } KVMARMGICClass;
 
-static void kvm_arm_gic_set_irq(void *opaque, int irq, int level)
+void kvm_arm_gic_set_irq(uint32_t num_irq, int irq, int level)
 {
     /* Meaning of the 'irq' parameter:
      *  [0..N-1] : external interrupts
@@ -63,10 +64,9 @@ static void kvm_arm_gic_set_irq(void *opaque, int irq, int level)
      * has separate fields in the irq number for type,
      * CPU number and interrupt number.
      */
-    GICState *s = (GICState *)opaque;
     int kvm_irq, irqtype, cpu;
 
-    if (irq < (s->num_irq - GIC_INTERNAL)) {
+    if (irq < (num_irq - GIC_INTERNAL)) {
         /* External interrupt. The kernel numbers these like the GIC
          * hardware, with external interrupt IDs starting after the
          * internal ones.
@@ -77,7 +77,7 @@ static void kvm_arm_gic_set_irq(void *opaque, int irq, int level)
     } else {
         /* Internal interrupt: decode into (cpu, interrupt id) */
         irqtype = KVM_ARM_IRQ_TYPE_PPI;
-        irq -= (s->num_irq - GIC_INTERNAL);
+        irq -= (num_irq - GIC_INTERNAL);
         cpu = irq / GIC_INTERNAL;
         irq %= GIC_INTERNAL;
     }
@@ -87,12 +87,19 @@ static void kvm_arm_gic_set_irq(void *opaque, int irq, int level)
     kvm_set_irq(kvm_state, kvm_irq, !!level);
 }
 
+static void kvm_arm_gicv2_set_irq(void *opaque, int irq, int level)
+{
+    GICState *s = (GICState *)opaque;
+
+    kvm_arm_gic_set_irq(s->num_irq, irq, level);
+}
+
 static bool kvm_arm_gic_can_save_restore(GICState *s)
 {
     return s->dev_fd >= 0;
 }
 
-static bool kvm_gic_supports_attr(GICState *s, int group, int attrnum)
+bool kvm_gic_supports_attr(int dev_fd, int group, int attrnum)
 {
     struct kvm_device_attr attr = {
         .group = group,
@@ -100,14 +107,14 @@ static bool kvm_gic_supports_attr(GICState *s, int group, int attrnum)
         .flags = 0,
     };
 
-    if (s->dev_fd == -1) {
+    if (dev_fd == -1) {
         return false;
     }
 
-    return kvm_device_ioctl(s->dev_fd, KVM_HAS_DEVICE_ATTR, &attr) == 0;
+    return kvm_device_ioctl(dev_fd, KVM_HAS_DEVICE_ATTR, &attr) == 0;
 }
 
-static void kvm_gic_access(GICState *s, int group, int offset,
+void kvm_gic_access(int dev_fd, int group, int offset,
                                    int cpu, uint32_t *val, bool write)
 {
     struct kvm_device_attr attr;
@@ -130,7 +137,7 @@ static void kvm_gic_access(GICState *s, int group, int offset,
         type = KVM_GET_DEVICE_ATTR;
     }
 
-    err = kvm_device_ioctl(s->dev_fd, type, &attr);
+    err = kvm_device_ioctl(dev_fd, type, &attr);
     if (err < 0) {
         fprintf(stderr, "KVM_{SET/GET}_DEVICE_ATTR failed: %s\n",
                 strerror(-err));
@@ -138,20 +145,6 @@ static void kvm_gic_access(GICState *s, int group, int offset,
     }
 }
 
-static void kvm_gicd_access(GICState *s, int offset, int cpu,
-                            uint32_t *val, bool write)
-{
-    kvm_gic_access(s, KVM_DEV_ARM_VGIC_GRP_DIST_REGS,
-                   offset, cpu, val, write);
-}
-
-static void kvm_gicc_access(GICState *s, int offset, int cpu,
-                            uint32_t *val, bool write)
-{
-    kvm_gic_access(s, KVM_DEV_ARM_VGIC_GRP_CPU_REGS,
-                   offset, cpu, val, write);
-}
-
 #define for_each_irq_reg(_ctr, _max_irq, _field_width) \
     for (_ctr = 0; _ctr < ((_max_irq) / (32 / (_field_width))); _ctr++)
 
@@ -291,7 +284,7 @@ static void kvm_dist_get(GICState *s, uint32_t offset, int width,
         irq = i * regsz;
         cpu = 0;
         while ((cpu < s->num_cpu && irq < GIC_INTERNAL) || cpu == 0) {
-            kvm_gicd_access(s, offset, cpu, &reg, false);
+            kvm_gicd_access(s->dev_fd, offset, cpu, &reg, false);
             for (j = 0; j < regsz; j++) {
                 field = extract32(reg, j * width, width);
                 translate_fn(s, irq + j, cpu, &field, false);
@@ -324,7 +317,7 @@ static void kvm_dist_put(GICState *s, uint32_t offset, int width,
                 translate_fn(s, irq + j, cpu, &field, true);
                 reg = deposit32(reg, j * width, width, field);
             }
-            kvm_gicd_access(s, offset, cpu, &reg, true);
+            kvm_gicd_access(s->dev_fd, offset, cpu, &reg, true);
 
             cpu++;
         }
@@ -355,10 +348,10 @@ static void kvm_arm_gic_put(GICState *s)
 
     /* s->ctlr -> GICD_CTLR */
     reg = s->ctlr;
-    kvm_gicd_access(s, 0x0, 0, &reg, true);
+    kvm_gicd_access(s->dev_fd, 0x0, 0, &reg, true);
 
     /* Sanity checking on GICD_TYPER and s->num_irq, s->num_cpu */
-    kvm_gicd_access(s, 0x4, 0, &reg, false);
+    kvm_gicd_access(s->dev_fd, 0x4, 0, &reg, false);
     num_irq = ((reg & 0x1f) + 1) * 32;
     num_cpu = ((reg & 0xe0) >> 5) + 1;
 
@@ -416,24 +409,24 @@ static void kvm_arm_gic_put(GICState *s)
     for (cpu = 0; cpu < s->num_cpu; cpu++) {
         /* s->cpu_ctlr[cpu] -> GICC_CTLR */
         reg = s->cpu_ctlr[cpu];
-        kvm_gicc_access(s, 0x00, cpu, &reg, true);
+        kvm_gicc_access(s->dev_fd, 0x00, cpu, &reg, true);
 
         /* s->priority_mask[cpu] -> GICC_PMR */
         reg = (s->priority_mask[cpu] & 0xff);
-        kvm_gicc_access(s, 0x04, cpu, &reg, true);
+        kvm_gicc_access(s->dev_fd, 0x04, cpu, &reg, true);
 
         /* s->bpr[cpu] -> GICC_BPR */
         reg = (s->bpr[cpu] & 0x7);
-        kvm_gicc_access(s, 0x08, cpu, &reg, true);
+        kvm_gicc_access(s->dev_fd, 0x08, cpu, &reg, true);
 
         /* s->abpr[cpu] -> GICC_ABPR */
         reg = (s->abpr[cpu] & 0x7);
-        kvm_gicc_access(s, 0x1c, cpu, &reg, true);
+        kvm_gicc_access(s->dev_fd, 0x1c, cpu, &reg, true);
 
         /* s->apr[n][cpu] -> GICC_APRn */
         for (i = 0; i < 4; i++) {
             reg = s->apr[i][cpu];
-            kvm_gicc_access(s, 0xd0 + i * 4, cpu, &reg, true);
+            kvm_gicc_access(s->dev_fd, 0xd0 + i * 4, cpu, &reg, true);
         }
     }
 }
@@ -454,11 +447,11 @@ static void kvm_arm_gic_get(GICState *s)
      */
 
     /* GICD_CTLR -> s->ctlr */
-    kvm_gicd_access(s, 0x0, 0, &reg, false);
+    kvm_gicd_access(s->dev_fd, 0x0, 0, &reg, false);
     s->ctlr = reg;
 
     /* Sanity checking on GICD_TYPER -> s->num_irq, s->num_cpu */
-    kvm_gicd_access(s, 0x4, 0, &reg, false);
+    kvm_gicd_access(s->dev_fd, 0x4, 0, &reg, false);
     s->num_irq = ((reg & 0x1f) + 1) * 32;
     s->num_cpu = ((reg & 0xe0) >> 5) + 1;
 
@@ -469,7 +462,7 @@ static void kvm_arm_gic_get(GICState *s)
     }
 
     /* GICD_IIDR -> ? */
-    kvm_gicd_access(s, 0x8, 0, &reg, false);
+    kvm_gicd_access(s->dev_fd, 0x8, 0, &reg, false);
 
     /* Clear all the IRQ settings */
     for (i = 0; i < s->num_irq; i++) {
@@ -507,24 +500,24 @@ static void kvm_arm_gic_get(GICState *s)
 
     for (cpu = 0; cpu < s->num_cpu; cpu++) {
         /* GICC_CTLR -> s->cpu_ctlr[cpu] */
-        kvm_gicc_access(s, 0x00, cpu, &reg, false);
+        kvm_gicc_access(s->dev_fd, 0x00, cpu, &reg, false);
         s->cpu_ctlr[cpu] = reg;
 
         /* GICC_PMR -> s->priority_mask[cpu] */
-        kvm_gicc_access(s, 0x04, cpu, &reg, false);
+        kvm_gicc_access(s->dev_fd, 0x04, cpu, &reg, false);
         s->priority_mask[cpu] = (reg & 0xff);
 
         /* GICC_BPR -> s->bpr[cpu] */
-        kvm_gicc_access(s, 0x08, cpu, &reg, false);
+        kvm_gicc_access(s->dev_fd, 0x08, cpu, &reg, false);
         s->bpr[cpu] = (reg & 0x7);
 
         /* GICC_ABPR -> s->abpr[cpu] */
-        kvm_gicc_access(s, 0x1c, cpu, &reg, false);
+        kvm_gicc_access(s->dev_fd, 0x1c, cpu, &reg, false);
         s->abpr[cpu] = (reg & 0x7);
 
         /* GICC_APRn -> s->apr[n][cpu] */
         for (i = 0; i < 4; i++) {
-            kvm_gicc_access(s, 0xd0 + i * 4, cpu, &reg, false);
+            kvm_gicc_access(s->dev_fd, 0xd0 + i * 4, cpu, &reg, false);
             s->apr[i][cpu] = reg;
         }
     }
@@ -569,7 +562,7 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
      *   ...
      */
     i += (GIC_INTERNAL * s->num_cpu);
-    qdev_init_gpio_in(dev, kvm_arm_gic_set_irq, i);
+    qdev_init_gpio_in(dev, kvm_arm_gicv2_set_irq, i);
     /* We never use our outbound IRQ/FIQ lines but provide them so that
      * we maintain the same interface as the non-KVM GIC.
      */
@@ -590,15 +583,16 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    if (kvm_gic_supports_attr(s, KVM_DEV_ARM_VGIC_GRP_NR_IRQS, 0)) {
+    if (kvm_gic_supports_attr(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS, 0)) {
         uint32_t numirqs = s->num_irq;
-        kvm_gic_access(s, KVM_DEV_ARM_VGIC_GRP_NR_IRQS, 0, 0, &numirqs, 1);
+        kvm_gic_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS, 0, 0,
+                       &numirqs, 1);
     }
 
     /* Tell the kernel to complete VGIC initialization now */
-    if (kvm_gic_supports_attr(s, KVM_DEV_ARM_VGIC_GRP_CTRL,
+    if (kvm_gic_supports_attr(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
                               KVM_DEV_ARM_VGIC_CTRL_INIT)) {
-        kvm_gic_access(s, KVM_DEV_ARM_VGIC_GRP_CTRL,
+        kvm_gic_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
                           KVM_DEV_ARM_VGIC_CTRL_INIT, 0, 0, 1);
     }
 
hw/intc/vgic_common.h new file +43 -0

43 lines not on this page · raw message

Signature · 1 line
1.9.5.msysgit.0
#20 lore raw

PATCH v4 6/9 Set kernel_irqchip_type for the rest of ARM boards which use GIC

In reply to Pavel Fedin

Just in case, the value is set for absolutely all board models which include GIC in some form. I am not sure whether all these models can use KVM acceleration, but it definitely would not hurt.

---
 hw/arm/exynos4_boards.c | 1 +
 hw/arm/realview.c       | 1 +
 hw/arm/vexpress.c       | 1 +
 3 files changed, 3 insertions(+)
hw/arm/exynos4_boards.c +1 -0
index d644db1..d4136bc 100644
@@ -104,6 +104,7 @@ static Exynos4210State *exynos4_boards_init_common(MachineState *machine,
                 exynos4_machines[board_type].max_cpus);
     }
 
+    machine->kernel_irqchip_type = KVM_DEV_TYPE_ARM_VGIC_V2;
     exynos4_board_binfo.ram_size = exynos4_board_ram_size[board_type];
     exynos4_board_binfo.board_id = exynos4_board_id[board_type];
     exynos4_board_binfo.smp_bootreg_addr =
hw/arm/realview.c +1 -0
index ef2788d..f670d9f 100644
@@ -74,6 +74,7 @@ static void realview_init(MachineState *machine,
     ram_addr_t ram_size = machine->ram_size;
     hwaddr periphbase = 0;
 
+    machine->kernel_irqchip_type = KVM_DEV_TYPE_ARM_VGIC_V2;
     switch (board_type) {
     case BOARD_EB:
         break;
hw/arm/vexpress.c +1 -0
index da21788..0675a00 100644
@@ -556,6 +556,7 @@ static void vexpress_common_init(MachineState *machine)
     const hwaddr *map = daughterboard->motherboard_map;
     int i;
 
+    machine->kernel_irqchip_type = KVM_DEV_TYPE_ARM_VGIC_V2;
     daughterboard->init(vms, machine->ram_size, machine->cpu_model, pic);
 
     /*
Signature · 1 line
1.9.5.msysgit.0
#21 lore raw

PATCH v4 7/9 Make use of kernel_irqchip_type in kvm_arch_irqchip_create()

In reply to Pavel Fedin

This function is actually used only for capability verification, but now it will check for the correct GIC type.

---
 include/sysemu/kvm.h | 3 ++-
 kvm-all.c            | 2 +-
 stubs/kvm.c          | 2 +-
 target-arm/kvm.c     | 8 ++++++--
 4 files changed, 10 insertions(+), 5 deletions(-)
include/sysemu/kvm.h +2 -1
index f459fbd..b9009c8 100644
@@ -426,6 +426,7 @@ void kvm_init_irq_routing(KVMState *s);
 /**
  * kvm_arch_irqchip_create:
  * @KVMState: The KVMState pointer
+ * @type: irqchip type, architecture-specific
  *
  * Allow architectures to create an in-kernel irq chip themselves.
  *
@@ -433,7 +434,7 @@ void kvm_init_irq_routing(KVMState *s);
  *            0: irq chip was not created
  *          > 0: irq chip was created
  */
-int kvm_arch_irqchip_create(KVMState *s);
+int kvm_arch_irqchip_create(KVMState *s, int type);
 
 /**
  * kvm_set_one_reg - set a register value in KVM via KVM_SET_ONE_REG ioctl
kvm-all.c +1 -1
index 6bb2716..fcee52c 100644
@@ -1351,7 +1351,7 @@ static int kvm_irqchip_create(MachineState *machine, KVMState *s)
 
     /* First probe and see if there's a arch-specific hook to create the
      * in-kernel irqchip for us */
-    ret = kvm_arch_irqchip_create(s);
+    ret = kvm_arch_irqchip_create(s, machine->kernel_irqchip_type);
     if (ret < 0) {
         return ret;
     } else if (ret == 0) {
stubs/kvm.c +1 -1
index e7c60b6..a8505ff 100644
@@ -1,7 +1,7 @@
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
 
-int kvm_arch_irqchip_create(KVMState *s)
+int kvm_arch_irqchip_create(KVMState *s, int type)
 {
     return 0;
 }
target-arm/kvm.c +6 -2
index 548bfd7..fa1073f 100644
@@ -579,7 +579,7 @@ void kvm_arch_init_irq_routing(KVMState *s)
 {
 }
 
-int kvm_arch_irqchip_create(KVMState *s)
+int kvm_arch_irqchip_create(KVMState *s, int type)
 {
     int ret;
 
@@ -587,11 +587,15 @@ int kvm_arch_irqchip_create(KVMState *s)
      * let the device do this when it initializes itself, otherwise we
      * fall back to the old API */
 
-    ret = kvm_create_device(s, KVM_DEV_TYPE_ARM_VGIC_V2, true);
+    ret = kvm_create_device(s, type, true);
     if (ret == 0) {
         return 1;
     }
 
+    /* Fallback will create VGIC v2 */
+    if (type != KVM_DEV_TYPE_ARM_VGIC_V2) {
+        return ret;
+    }
     return 0;
 }
 
Signature · 1 line
1.9.5.msysgit.0
#22 lore raw

PATCH v4 8/9 Initial implementation of vGICv3

In reply to Pavel Fedin

Get/put routines are missing, live migration is not possible.

---
 hw/intc/Makefile.objs   |   1 +
 hw/intc/arm_gicv3_kvm.c | 203 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 204 insertions(+)
 create mode 100644 hw/intc/arm_gicv3_kvm.c
hw/intc/Makefile.objs +1 -0
index 6c2b136..22c8814 100644
@@ -18,6 +18,7 @@ common-obj-$(CONFIG_OPENPIC) += openpic.o
 
 obj-$(CONFIG_APIC) += apic.o apic_common.o
 obj-$(CONFIG_ARM_GIC_KVM) += arm_gic_kvm.o
+obj-$(CONFIG_ARM_GIC_KVM) += arm_gicv3_kvm.o
 obj-$(CONFIG_STELLARIS) += armv7m_nvic.o
 obj-$(CONFIG_EXYNOS4) += exynos4210_gic.o exynos4210_combiner.o
 obj-$(CONFIG_GRLIB) += grlib_irqmp.o
hw/intc/arm_gicv3_kvm.c new file +203 -0

203 lines not on this page · raw message

Signature · 1 line
1.9.5.msysgit.0
#23 lore raw

PATCH v4 9/9 Enable KVM acceleration for GICv3

In reply to Pavel Fedin

Instantiate "kvm-arm-gicv3" class for GICv3 with KVM acceleration

---
 hw/arm/virt.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
hw/arm/virt.c +2 -4
index 5c07d07..100cb17 100644
@@ -410,11 +410,9 @@ static void create_gic(VirtBoardInfo *vbi, qemu_irq *pic, int type)
     int i;
 
     if (type == KVM_DEV_TYPE_ARM_VGIC_V3) {
-        gictype = "arm_gicv3";
-    } else if (kvm_irqchip_in_kernel()) {
-        gictype = "kvm-arm-gic";
+        gictype = kvm_irqchip_in_kernel() ? "kvm-arm-gicv3" : "arm_gicv3";
     } else {
-        gictype = "arm_gic";
+        gictype = kvm_irqchip_in_kernel() ? "kvm-arm-gic" : "arm_gic";
     }
 
     gicdev = qdev_create(NULL, gictype);
Signature · 1 line
1.9.5.msysgit.0
#24 lore raw
In reply to Pavel Fedin

Hi Pavel,

Quoted 2 lines · Pavel Fedin

This is a complete GICv3 implementation, both software emulation and KVM acceleration are supported.

do you plan to resend a version without TCG code, just using shared base class?

Eric

Quoted 65 lines

This series is a consolidated and updated patch set, based on:
- GIC-500 implementation:
http://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg01512.html
- vGICv3 implementation:
https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg04496.html
I decided to repost because qemu development has progressed and it became difficult to apply those series to current master.

Changes from previous versions:
- Removed RFC prefix
- Base class separated from the rest, as was requested in RFC v2 review
- Fixed small number of broken comments / code formatting issues, according to the same review
- Removed #if 0 in virt.c
- vGICv3 patch set restructured and more clearly separated into portions, according to old Eric's review
- Removed duplication of low-level vGICv3 code. Common helper routines are used instead.
- Put informative commit messages

Pavel Fedin (6):
Add virt-v3 machine that uses GIC-500
Extract some reusable vGIC code
Set kernel_irqchip_type for the rest of ARM boards which use GIC Make use of kernel_irqchip_type in kvm_arch_irqchip_create() Initial implementation of vGICv3
Enable KVM acceleration for GICv3

Shlomo Pongratz (3):
Implement GIC-500 base class
Implement GIC-500
GICv3 support

 hw/arm/exynos4_boards.c            |    1 +
 hw/arm/realview.c                  |    1 +
 hw/arm/vexpress.c                  |    1 +
 hw/arm/virt.c                      |  141 ++-
 hw/intc/Makefile.objs              |    3 +
 hw/intc/arm_gic_kvm.c              |   84 +-
 hw/intc/arm_gicv3.c                | 2086 ++++++++++++++++++++++++++++++++++++
 hw/intc/arm_gicv3_common.c         |  216 ++++
 hw/intc/arm_gicv3_kvm.c            |  203 ++++
 hw/intc/gicv3_internal.h           |  159 +++
 hw/intc/vgic_common.h              |   43 +
 include/hw/arm/fdt.h               |    2 +-
 include/hw/arm/virt.h              |    6 +-
 include/hw/boards.h                |    1 +
 include/hw/intc/arm_gicv3.h        |   44 +
 include/hw/intc/arm_gicv3_common.h |  116 ++
 include/sysemu/kvm.h               |    3 +-
 kvm-all.c                          |    2 +-
 stubs/kvm.c                        |    2 +-
 target-arm/cpu.h                   |   12 +
 target-arm/cpu64.c                 |  105 ++
 target-arm/kvm.c                   |    8 +-
 22 files changed, 3164 insertions(+), 75 deletions(-)
 create mode 100644 hw/intc/arm_gicv3.c
 create mode 100644 hw/intc/arm_gicv3_common.c
 create mode 100644 hw/intc/arm_gicv3_kvm.c
 create mode 100644 hw/intc/gicv3_internal.h
 create mode 100644 hw/intc/vgic_common.h
 create mode 100644 include/hw/intc/arm_gicv3.h
 create mode 100644 include/hw/intc/arm_gicv3_common.h
#25 lore raw
Quoted 2 lines

do you plan to resend a version without TCG code, just using shared base class?

Done that, you missed it:
http://lists.nongnu.org/archive/html/qemu-devel/2015-07/msg00830.html

Sorry for confusion with version numbers, i have to idea how to do this correctly when
combining/splitting different series. I set that one to v2 because i consider it to be a continuation of my old RFC which was v1.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

#26 lore raw
Quoted 5 lines · Pavel Fedin
Quoted 2 lines

do you plan to resend a version without TCG code, just using shared base class?

Done that, you missed it:
http://lists.nongnu.org/archive/html/qemu-devel/2015-07/msg00830.html

Hum OK thanks. I will review that one then ;-)

Eric

Quoted 11 lines

Sorry for confusion with version numbers, i have to idea how to do this correctly when
combining/splitting different series. I set that one to v2 because i consider it to be a continuation of my old RFC which was v1.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

#27 lore raw
Quoted 3 lines
Quoted 1 line

http://lists.nongnu.org/archive/html/qemu-devel/2015-07/msg00830.html

Hum OK thanks. I will review that one then ;-)

By this time i've released v3 with some important fixes. You're going to get it because i finally
learned to correctly specify --to and -cc multiple times to git send-email :)

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia