Skip to contentSkip to conversation

Add riscv kvm accel support

PATCH Superseded Yifei Jiang 3 participants 22 messages Dec 20, 2021 – Jan 10, 2022

Subsystems (5)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover Add riscv kvm accel support

This series adds both riscv32 and riscv64 kvm support, and implements migration based on riscv.

Because of RISC-V KVM has been merged into the Linux master, so this series are changed from RFC to patch.

Several steps to use this:
1. Build emulation

$ ./configure --target-list=riscv64-softmmu
$ make -j$(nproc)

2. Build kernel

3. Build QEMU VM
Cross built in riscv toolchain.

$ PKG_CONFIG_LIBDIR=<toolchain pkgconfig path>
$ export PKG_CONFIG_SYSROOT_DIR=<toolchain sysroot path>
$ ./configure --target-list=riscv64-softmmu --enable-kvm \
--cross-prefix=riscv64-linux-gnu- --disable-libiscsi --disable-glusterfs \
--disable-libusb --disable-usb-redir --audio-drv-list= --disable-opengl \
--disable-libxml2
$ make -j$(nproc)

4. Start emulation

$ ./qemu-system-riscv64 -M virt -m 4096M -cpu rv64,x-h=true -nographic \
        -name guest=riscv-hyp,debug-threads=on \
        -smp 4 \
        -bios ./fw_jump.bin \
        -kernel ./Image \
        -drive file=./hyp.img,format=raw,id=hd0 \
        -device virtio-blk-device,drive=hd0 \
        -append "root=/dev/vda rw console=ttyS0 earlycon=sbi"

5. Start kvm-acceled QEMU VM in emulation

$ ./qemu-system-riscv64 -M virt,accel=kvm -m 1024M -cpu host -nographic \
        -name guest=riscv-guset \
        -smp 2 \
        -bios none \
        -kernel ./Image \
        -drive file=./guest.img,format=raw,id=hd0 \
        -device virtio-blk-device,drive=hd0 \
        -append "root=/dev/vda rw console=ttyS0 earlycon=sbi"

Changes since patch v2
- Create a macro for get and put timer csr.
- Remove M-mode PLIC contexts when kvm is enabled.
- Add get timer frequency.
- Move cpu_host_load to vmstate_kvmtimer.

Changes since patch v1
- Rebase on recent commit a216e7cf119c91ffdf5931834a1a030ebea40d70
- Sync-up headers with Linux-5.16-rc4.
- Fixbug in kvm_arch_init_vcpu.
- Create a macro for get and put regs csr.
- Start kernel directly when kvm_enabled.
- Use riscv_cpu_set_irq to inject KVM interrupts.
- Use the Semihosting Console API for RISC-V kvm handle sbi.
- Update vmstate_riscv_cpu version id.
Placing kvm_timer into a subsection.

Changes since RFC v6
- Rebase on recent commit 8627edfb3f1fca24a96a0954148885c3241c10f8
- Sync-up headers with Linux-5.16-rc1

Changes since RFC v5
- Rebase on QEMU v6.1.0-rc1 and kvm-riscv linux v19.
- Move kvm interrupt setting to riscv_cpu_update_mip().
- Replace __u64 with uint64_t.

Changes since RFC v4
- Rebase on QEMU v6.0.0-rc2 and kvm-riscv linux v17.
- Remove time scaling support as software solution is incomplete. Because it will cause unacceptable performance degradation. and We will post a better solution.
- Revise according to Alistair's review comments.
- Remove compile time XLEN checks in kvm_riscv_reg_id
- Surround TYPE_RISCV_CPU_HOST definition by CONFIG_KVM and share it between RV32 and RV64.
- Add kvm-stub.c for reduce unnecessary compilation checks.
- Add riscv_setup_direct_kernel() to direct boot kernel for KVM.

Changes since RFC v3
- Rebase on QEMU v5.2.0-rc2 and kvm-riscv linux v15.
- Add time scaling support(New patches 13, 14 and 15).
- Fix the bug that guest vm can't reboot.

Changes since RFC v2
- Fix checkpatch error at target/riscv/sbi_ecall_interface.h.
- Add riscv migration support.

Changes since RFC v1
- Add separate SBI ecall interface header.
- Add riscv32 kvm accel support.

Yifei Jiang (12):
update-linux-headers: Add asm-riscv/kvm.h
target/riscv: Add target/riscv/kvm.c to place the public kvm interface target/riscv: Implement function kvm_arch_init_vcpu target/riscv: Implement kvm_arch_get_registers target/riscv: Implement kvm_arch_put_registers target/riscv: Support start kernel directly by KVM target/riscv: Support setting external interrupt by KVM target/riscv: Handle KVM_EXIT_RISCV_SBI exit
target/riscv: Add host cpu type
target/riscv: Add kvm_riscv_get/put_regs_timer target/riscv: Implement virtual time adjusting with vm state changing target/riscv: Support virtual time context synchronization

 hw/intc/sifive_plic.c              |   8 +-
 hw/riscv/boot.c                    |  16 +-
 hw/riscv/virt.c                    |  87 +++--
 include/hw/riscv/boot.h            |   1 +
 linux-headers/asm-riscv/kvm.h      | 128 +++++++
 meson.build                        |   2 +
 target/riscv/cpu.c                 |  29 +-
 target/riscv/cpu.h                 |  11 +
 target/riscv/kvm-stub.c            |  30 ++
 target/riscv/kvm.c                 | 533 +++++++++++++++++++++++++++++
 target/riscv/kvm_riscv.h           |  25 ++
 target/riscv/machine.c             |  30 ++
 target/riscv/meson.build           |   1 +
 target/riscv/sbi_ecall_interface.h |  72 ++++
 14 files changed, 944 insertions(+), 29 deletions(-)
 create mode 100644 linux-headers/asm-riscv/kvm.h
 create mode 100644 target/riscv/kvm-stub.c
 create mode 100644 target/riscv/kvm.c
 create mode 100644 target/riscv/kvm_riscv.h
 create mode 100644 target/riscv/sbi_ecall_interface.h
Signature · 1 line
2.19.1
#1 lore raw

PATCH v3 1/12 update-linux-headers: Add asm-riscv/kvm.h

Add asm-riscv/kvm.h for RISC-V KVM, and update linux/kvm.h

---
 linux-headers/asm-riscv/kvm.h | 128 ++++++++++++++++++++++++++++++++++
 1 file changed, 128 insertions(+)
 create mode 100644 linux-headers/asm-riscv/kvm.h
linux-headers/asm-riscv/kvm.h new file +128 -0
new file mode 100644
index 0000000000..f808ad1ce5
@@ -0,0 +1,128 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2019 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ *     Anup Patel <[email protected]>
+ */
+
+#ifndef __LINUX_KVM_RISCV_H
+#define __LINUX_KVM_RISCV_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+#include <asm/ptrace.h>
+
+#define __KVM_HAVE_READONLY_MEM
+
+#define KVM_COALESCED_MMIO_PAGE_OFFSET 1
+
+#define KVM_INTERRUPT_SET	-1U
+#define KVM_INTERRUPT_UNSET	-2U
+
+/* for KVM_GET_REGS and KVM_SET_REGS */
+struct kvm_regs {
+};
+
+/* for KVM_GET_FPU and KVM_SET_FPU */
+struct kvm_fpu {
+};
+
+/* KVM Debug exit structure */
+struct kvm_debug_exit_arch {
+};
+
+/* for KVM_SET_GUEST_DEBUG */
+struct kvm_guest_debug_arch {
+};
+
+/* definition of registers in kvm_run */
+struct kvm_sync_regs {
+};
+
+/* for KVM_GET_SREGS and KVM_SET_SREGS */
+struct kvm_sregs {
+};
+
+/* CONFIG registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+struct kvm_riscv_config {
+	unsigned long isa;
+};
+
+/* CORE registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+struct kvm_riscv_core {
+	struct user_regs_struct regs;
+	unsigned long mode;
+};
+
+/* Possible privilege modes for kvm_riscv_core */
+#define KVM_RISCV_MODE_S	1
+#define KVM_RISCV_MODE_U	0
+
+/* CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+struct kvm_riscv_csr {
+	unsigned long sstatus;
+	unsigned long sie;
+	unsigned long stvec;
+	unsigned long sscratch;
+	unsigned long sepc;
+	unsigned long scause;
+	unsigned long stval;
+	unsigned long sip;
+	unsigned long satp;
+	unsigned long scounteren;
+};
+
+/* TIMER registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+struct kvm_riscv_timer {
+	__u64 frequency;
+	__u64 time;
+	__u64 compare;
+	__u64 state;
+};
+
+/* Possible states for kvm_riscv_timer */
+#define KVM_RISCV_TIMER_STATE_OFF	0
+#define KVM_RISCV_TIMER_STATE_ON	1
+
+#define KVM_REG_SIZE(id)		\
+	(1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
+
+/* If you need to interpret the index values, here is the key: */
+#define KVM_REG_RISCV_TYPE_MASK		0x00000000FF000000
+#define KVM_REG_RISCV_TYPE_SHIFT	24
+
+/* Config registers are mapped as type 1 */
+#define KVM_REG_RISCV_CONFIG		(0x01 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_CONFIG_REG(name)	\
+	(offsetof(struct kvm_riscv_config, name) / sizeof(unsigned long))
+
+/* Core registers are mapped as type 2 */
+#define KVM_REG_RISCV_CORE		(0x02 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_CORE_REG(name)	\
+		(offsetof(struct kvm_riscv_core, name) / sizeof(unsigned long))
+
+/* Control and status registers are mapped as type 3 */
+#define KVM_REG_RISCV_CSR		(0x03 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_CSR_REG(name)	\
+		(offsetof(struct kvm_riscv_csr, name) / sizeof(unsigned long))
+
+/* Timer registers are mapped as type 4 */
+#define KVM_REG_RISCV_TIMER		(0x04 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_TIMER_REG(name)	\
+		(offsetof(struct kvm_riscv_timer, name) / sizeof(__u64))
+
+/* F extension registers are mapped as type 5 */
+#define KVM_REG_RISCV_FP_F		(0x05 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_FP_F_REG(name)	\
+		(offsetof(struct __riscv_f_ext_state, name) / sizeof(__u32))
+
+/* D extension registers are mapped as type 6 */
+#define KVM_REG_RISCV_FP_D		(0x06 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_FP_D_REG(name)	\
+		(offsetof(struct __riscv_d_ext_state, name) / sizeof(__u64))
+
+#endif
+
+#endif /* __LINUX_KVM_RISCV_H */
Signature · 1 line
2.19.1
#2 lore raw

PATCH v3 2/12 target/riscv: Add target/riscv/kvm.c to place the public kvm interface

In reply to Yifei Jiang

Add target/riscv/kvm.c to place kvm_arch_* function needed by kvm/kvm-all.c. Meanwhile, add kvm support in meson.build file.

---
 meson.build              |   2 +
 target/riscv/kvm.c       | 133 +++++++++++++++++++++++++++++++++++++++
 target/riscv/meson.build |   1 +
 3 files changed, 136 insertions(+)
 create mode 100644 target/riscv/kvm.c
meson.build +2 -0
index 2495360fd0..e74633b42c 100644
@@ -85,6 +85,8 @@ elif cpu in ['ppc', 'ppc64']
   kvm_targets = ['ppc-softmmu', 'ppc64-softmmu']
 elif cpu in ['mips', 'mips64']
   kvm_targets = ['mips-softmmu', 'mipsel-softmmu', 'mips64-softmmu', 'mips64el-softmmu']
+elif cpu in ['riscv']
+  kvm_targets = ['riscv32-softmmu', 'riscv64-softmmu']
 else
   kvm_targets = []
 endif
target/riscv/kvm.c new file +133 -0

133 lines not on this page · raw message

target/riscv/meson.build +1 -0
index d5e0bc93ea..2faf08a941 100644
@@ -19,6 +19,7 @@ riscv_ss.add(files(
   'bitmanip_helper.c',
   'translate.c',
 ))
+riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'))
 
 riscv_softmmu_ss = ss.source_set()
 riscv_softmmu_ss.add(files(
Signature · 1 line
2.19.1
#3 lore raw

PATCH v3 3/12 target/riscv: Implement function kvm_arch_init_vcpu

In reply to Yifei Jiang

Get isa info from kvm while kvm init.

---
 target/riscv/kvm.c | 32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)
target/riscv/kvm.c +31 -1
index 687dd4b621..ccf3753048 100644
@@ -38,6 +38,23 @@
 #include "qemu/log.h"
 #include "hw/loader.h"
 
+static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx)
+{
+    uint64_t id = KVM_REG_RISCV | type | idx;
+
+    switch (riscv_cpu_mxl(env)) {
+    case MXL_RV32:
+        id |= KVM_REG_SIZE_U32;
+        break;
+    case MXL_RV64:
+        id |= KVM_REG_SIZE_U64;
+        break;
+    default:
+        g_assert_not_reached();
+    }
+    return id;
+}
+
 const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
     KVM_CAP_LAST_INFO
 };
@@ -79,7 +96,20 @@ void kvm_arch_init_irq_routing(KVMState *s)
 
 int kvm_arch_init_vcpu(CPUState *cs)
 {
-    return 0;
+    int ret = 0;
+    target_ulong isa;
+    RISCVCPU *cpu = RISCV_CPU(cs);
+    CPURISCVState *env = &cpu->env;
+    uint64_t id;
+
+    id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CONFIG, KVM_REG_RISCV_CONFIG_REG(isa));
+    ret = kvm_get_one_reg(cs, id, &isa);
+    if (ret) {
+        return ret;
+    }
+    env->misa_ext = isa;
+
+    return ret;
 }
 
 int kvm_arch_msi_data_to_gsi(uint32_t data)
Signature · 1 line
2.19.1
#4 lore raw

PATCH v3 4/12 target/riscv: Implement kvm_arch_get_registers

In reply to Yifei Jiang

Get GPR CSR and FP registers from kvm by KVM_GET_ONE_REG ioctl.

---
 target/riscv/kvm.c | 112 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 111 insertions(+), 1 deletion(-)
target/riscv/kvm.c +111 -1
index ccf3753048..6d4df0ef6d 100644
@@ -55,13 +55,123 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx
     return id;
 }
 
+#define RISCV_CORE_REG(env, name)  kvm_riscv_reg_id(env, KVM_REG_RISCV_CORE, \
+                 KVM_REG_RISCV_CORE_REG(name))
+
+#define RISCV_CSR_REG(env, name)  kvm_riscv_reg_id(env, KVM_REG_RISCV_CSR, \
+                 KVM_REG_RISCV_CSR_REG(name))
+
+#define RISCV_FP_F_REG(env, idx)  kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_F, idx)
+
+#define RISCV_FP_D_REG(env, idx)  kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_D, idx)
+
+#define KVM_RISCV_GET_CSR(cs, env, csr, reg) \
+    do { \
+        int ret = kvm_get_one_reg(cs, RISCV_CSR_REG(env, csr), &reg); \
+        if (ret) { \
+            return ret; \
+        } \
+    } while(0)
+
+static int kvm_riscv_get_regs_core(CPUState *cs)
+{
+    int ret = 0;
+    int i;
+    target_ulong reg;
+    CPURISCVState *env = &RISCV_CPU(cs)->env;
+
+    ret = kvm_get_one_reg(cs, RISCV_CORE_REG(env, regs.pc), &reg);
+    if (ret) {
+        return ret;
+    }
+    env->pc = reg;
+
+    for (i = 1; i < 32; i++) {
+        uint64_t id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CORE, i);
+        ret = kvm_get_one_reg(cs, id, &reg);
+        if (ret) {
+            return ret;
+        }
+        env->gpr[i] = reg;
+    }
+
+    return ret;
+}
+
+static int kvm_riscv_get_regs_csr(CPUState *cs)
+{
+    int ret = 0;
+    CPURISCVState *env = &RISCV_CPU(cs)->env;
+
+    KVM_RISCV_GET_CSR(cs, env, sstatus, env->mstatus);
+    KVM_RISCV_GET_CSR(cs, env, sie, env->mie);
+    KVM_RISCV_GET_CSR(cs, env, stvec, env->stvec);
+    KVM_RISCV_GET_CSR(cs, env, sscratch, env->sscratch);
+    KVM_RISCV_GET_CSR(cs, env, sepc, env->sepc);
+    KVM_RISCV_GET_CSR(cs, env, scause, env->scause);
+    KVM_RISCV_GET_CSR(cs, env, stval, env->stval);
+    KVM_RISCV_GET_CSR(cs, env, sip, env->mip);
+    KVM_RISCV_GET_CSR(cs, env, satp, env->satp);
+    return ret;
+}
+
+static int kvm_riscv_get_regs_fp(CPUState *cs)
+{
+    int ret = 0;
+    int i;
+    CPURISCVState *env = &RISCV_CPU(cs)->env;
+
+    if (riscv_has_ext(env, RVD)) {
+        uint64_t reg;
+        for (i = 0; i < 32; i++) {
+            ret = kvm_get_one_reg(cs, RISCV_FP_D_REG(env, i), &reg);
+            if (ret) {
+                return ret;
+            }
+            env->fpr[i] = reg;
+        }
+        return ret;
+    }
+
+    if (riscv_has_ext(env, RVF)) {
+        uint32_t reg;
+        for (i = 0; i < 32; i++) {
+            ret = kvm_get_one_reg(cs, RISCV_FP_F_REG(env, i), &reg);
+            if (ret) {
+                return ret;
+            }
+            env->fpr[i] = reg;
+        }
+        return ret;
+    }
+
+    return ret;
+}
+
 const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
     KVM_CAP_LAST_INFO
 };
 
 int kvm_arch_get_registers(CPUState *cs)
 {
-    return 0;
+    int ret = 0;
+
+    ret = kvm_riscv_get_regs_core(cs);
+    if (ret) {
+        return ret;
+    }
+
+    ret = kvm_riscv_get_regs_csr(cs);
+    if (ret) {
+        return ret;
+    }
+
+    ret = kvm_riscv_get_regs_fp(cs);
+    if (ret) {
+        return ret;
+    }
+
+    return ret;
 }
 
 int kvm_arch_put_registers(CPUState *cs, int level)
Signature · 1 line
2.19.1
#5 lore raw

PATCH v3 5/12 target/riscv: Implement kvm_arch_put_registers

In reply to Yifei Jiang

Put GPR CSR and FP registers to kvm by KVM_SET_ONE_REG ioctl

---
 target/riscv/kvm.c | 104 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 103 insertions(+), 1 deletion(-)
target/riscv/kvm.c +103 -1
index 6d4df0ef6d..e695b91dc7 100644
@@ -73,6 +73,14 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx
         } \
     } while(0)
 
+#define KVM_RISCV_SET_CSR(cs, env, csr, reg) \
+    do { \
+        int ret = kvm_set_one_reg(cs, RISCV_CSR_REG(env, csr), &reg); \
+        if (ret) { \
+            return ret; \
+        } \
+    } while(0)
+
 static int kvm_riscv_get_regs_core(CPUState *cs)
 {
     int ret = 0;
@@ -98,6 +106,31 @@ static int kvm_riscv_get_regs_core(CPUState *cs)
     return ret;
 }
 
+static int kvm_riscv_put_regs_core(CPUState *cs)
+{
+    int ret = 0;
+    int i;
+    target_ulong reg;
+    CPURISCVState *env = &RISCV_CPU(cs)->env;
+
+    reg = env->pc;
+    ret = kvm_set_one_reg(cs, RISCV_CORE_REG(env, regs.pc), &reg);
+    if (ret) {
+        return ret;
+    }
+
+    for (i = 1; i < 32; i++) {
+        uint64_t id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CORE, i);
+        reg = env->gpr[i];
+        ret = kvm_set_one_reg(cs, id, &reg);
+        if (ret) {
+            return ret;
+        }
+    }
+
+    return ret;
+}
+
 static int kvm_riscv_get_regs_csr(CPUState *cs)
 {
     int ret = 0;
@@ -115,6 +148,24 @@ static int kvm_riscv_get_regs_csr(CPUState *cs)
     return ret;
 }
 
+static int kvm_riscv_put_regs_csr(CPUState *cs)
+{
+    int ret = 0;
+    CPURISCVState *env = &RISCV_CPU(cs)->env;
+
+    KVM_RISCV_SET_CSR(cs, env, sstatus, env->mstatus);
+    KVM_RISCV_SET_CSR(cs, env, sie, env->mie);
+    KVM_RISCV_SET_CSR(cs, env, stvec, env->stvec);
+    KVM_RISCV_SET_CSR(cs, env, sscratch, env->sscratch);
+    KVM_RISCV_SET_CSR(cs, env, sepc, env->sepc);
+    KVM_RISCV_SET_CSR(cs, env, scause, env->scause);
+    KVM_RISCV_SET_CSR(cs, env, stval, env->stval);
+    KVM_RISCV_SET_CSR(cs, env, sip, env->mip);
+    KVM_RISCV_SET_CSR(cs, env, satp, env->satp);
+
+    return ret;
+}
+
 static int kvm_riscv_get_regs_fp(CPUState *cs)
 {
     int ret = 0;
@@ -148,6 +199,40 @@ static int kvm_riscv_get_regs_fp(CPUState *cs)
     return ret;
 }
 
+static int kvm_riscv_put_regs_fp(CPUState *cs)
+{
+    int ret = 0;
+    int i;
+    CPURISCVState *env = &RISCV_CPU(cs)->env;
+
+    if (riscv_has_ext(env, RVD)) {
+        uint64_t reg;
+        for (i = 0; i < 32; i++) {
+            reg = env->fpr[i];
+            ret = kvm_set_one_reg(cs, RISCV_FP_D_REG(env, i), &reg);
+            if (ret) {
+                return ret;
+            }
+        }
+        return ret;
+    }
+
+    if (riscv_has_ext(env, RVF)) {
+        uint32_t reg;
+        for (i = 0; i < 32; i++) {
+            reg = env->fpr[i];
+            ret = kvm_set_one_reg(cs, RISCV_FP_F_REG(env, i), &reg);
+            if (ret) {
+                return ret;
+            }
+        }
+        return ret;
+    }
+
+    return ret;
+}
+
+
 const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
     KVM_CAP_LAST_INFO
 };
@@ -176,7 +261,24 @@ int kvm_arch_get_registers(CPUState *cs)
 
 int kvm_arch_put_registers(CPUState *cs, int level)
 {
-    return 0;
+    int ret = 0;
+
+    ret = kvm_riscv_put_regs_core(cs);
+    if (ret) {
+        return ret;
+    }
+
+    ret = kvm_riscv_put_regs_csr(cs);
+    if (ret) {
+        return ret;
+    }
+
+    ret = kvm_riscv_put_regs_fp(cs);
+    if (ret) {
+        return ret;
+    }
+
+    return ret;
 }
 
 int kvm_arch_release_virq_post(int virq)
Signature · 1 line
2.19.1
#6 lore raw

PATCH v3 6/12 target/riscv: Support start kernel directly by KVM

In reply to Yifei Jiang

Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. Add kvm_riscv.h to place riscv specific interface.

In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.

---
 hw/intc/sifive_plic.c    |  8 +++-
 hw/riscv/boot.c          | 16 +++++++-
 hw/riscv/virt.c          | 87 ++++++++++++++++++++++++++++------------
 include/hw/riscv/boot.h  |  1 +
 target/riscv/cpu.c       |  8 ++++
 target/riscv/cpu.h       |  3 ++
 target/riscv/kvm-stub.c  | 25 ++++++++++++
 target/riscv/kvm.c       | 14 +++++++
 target/riscv/kvm_riscv.h | 24 +++++++++++
 target/riscv/meson.build |  2 +-
 10 files changed, 159 insertions(+), 29 deletions(-)
 create mode 100644 target/riscv/kvm-stub.c
 create mode 100644 target/riscv/kvm_riscv.h
hw/intc/sifive_plic.c +6 -2
index 877e76877c..1b2b4cc25e 100644
@@ -30,6 +30,7 @@
 #include "target/riscv/cpu.h"
 #include "migration/vmstate.h"
 #include "hw/irq.h"
+#include "sysemu/kvm.h"
 
 #define RISCV_DEBUG_PLIC 0
 
@@ -555,8 +556,11 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,
 
         qdev_connect_gpio_out(dev, i,
                               qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT));
-        qdev_connect_gpio_out(dev, num_harts + i,
-                              qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
+
+        if (!kvm_enabled()) {
+            qdev_connect_gpio_out(dev, num_harts + i,
+                                  qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
+        }
     }
 
     return dev;
hw/riscv/boot.c +15 -1
index 519fa455a1..ccff662d89 100644
@@ -30,6 +30,7 @@
 #include "elf.h"
 #include "sysemu/device_tree.h"
 #include "sysemu/qtest.h"
+#include "sysemu/kvm.h"
 
 #include <libfdt.h>
 
@@ -51,7 +52,9 @@ char *riscv_plic_hart_config_string(int hart_count)
         CPUState *cs = qemu_get_cpu(i);
         CPURISCVState *env = &RISCV_CPU(cs)->env;
 
-        if (riscv_has_ext(env, RVS)) {
+        if (kvm_enabled()) {
+            vals[i] = "S";
+        } else if (riscv_has_ext(env, RVS)) {
             vals[i] = "MS";
         } else {
             vals[i] = "M";
@@ -317,3 +320,14 @@ void riscv_setup_rom_reset_vec(MachineState *machine, RISCVHartArrayState *harts
 
     return;
 }
+
+void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr fdt_addr)
+{
+    CPUState *cs;
+
+    for (cs = first_cpu; cs; cs = CPU_NEXT(cs)) {
+        RISCVCPU *riscv_cpu = RISCV_CPU(cs);
+        riscv_cpu->env.kernel_addr = kernel_addr;
+        riscv_cpu->env.fdt_addr = fdt_addr;
+    }
+}
hw/riscv/virt.c +62 -25

133 lines not on this page · raw message

include/hw/riscv/boot.h +1 -0
index baff11dd8a..5834c234aa 100644
@@ -58,5 +58,6 @@ void riscv_rom_copy_firmware_info(MachineState *machine, hwaddr rom_base,
                                   hwaddr rom_size,
                                   uint32_t reset_vec_size,
                                   uint64_t kernel_entry);
+void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr fdt_addr);
 
 #endif /* RISCV_BOOT_H */
target/riscv/cpu.c +8 -0
index f812998123..1c944872a3 100644
@@ -29,6 +29,8 @@
 #include "hw/qdev-properties.h"
 #include "migration/vmstate.h"
 #include "fpu/softfloat-helpers.h"
+#include "sysemu/kvm.h"
+#include "kvm_riscv.h"
 
 /* RISC-V CPU definitions */
 
@@ -380,6 +382,12 @@ static void riscv_cpu_reset(DeviceState *dev)
     cs->exception_index = RISCV_EXCP_NONE;
     env->load_res = -1;
     set_default_nan_mode(1, &env->fp_status);
+
+#ifndef CONFIG_USER_ONLY
+    if (kvm_enabled()) {
+        kvm_riscv_reset_vcpu(cpu);
+    }
+#endif
 }
 
 static void riscv_cpu_disas_set_info(CPUState *s, disassemble_info *info)
target/riscv/cpu.h +3 -0
index 0760c0af93..2807eb1bcb 100644
@@ -255,6 +255,9 @@ struct CPURISCVState {
 
     /* Fields from here on are preserved across CPU reset. */
     QEMUTimer *timer; /* Internal timer */
+
+    hwaddr kernel_addr;
+    hwaddr fdt_addr;
 };
 
 OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,
target/riscv/kvm-stub.c new file +25 -0
new file mode 100644
index 0000000000..39b96fe3f4
@@ -0,0 +1,25 @@
+/*
+ * QEMU KVM RISC-V specific function stubs
+ *
+ * Copyright (c) 2020 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "kvm_riscv.h"
+
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
+{
+    abort();
+}
target/riscv/kvm.c +14 -0
index e695b91dc7..db6d8a5b6e 100644
@@ -37,6 +37,7 @@
 #include "hw/irq.h"
 #include "qemu/log.h"
 #include "hw/loader.h"
+#include "kvm_riscv.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx)
 {
@@ -369,6 +370,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
     return 0;
 }
 
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
+{
+    CPURISCVState *env = &cpu->env;
+
+    if (!kvm_enabled()) {
+        return;
+    }
+    env->pc = cpu->env.kernel_addr;
+    env->gpr[10] = kvm_arch_vcpu_id(CPU(cpu)); /* a0 */
+    env->gpr[11] = cpu->env.fdt_addr;          /* a1 */
+    env->satp = 0;
+}
+
 bool kvm_arch_cpu_check_are_resettable(void)
 {
     return true;
target/riscv/kvm_riscv.h new file +24 -0
new file mode 100644
index 0000000000..f38c82bf59
@@ -0,0 +1,24 @@
+/*
+ * QEMU KVM support -- RISC-V specific functions.
+ *
+ * Copyright (c) 2020 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef QEMU_KVM_RISCV_H
+#define QEMU_KVM_RISCV_H
+
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
+
+#endif
target/riscv/meson.build +1 -1
index 2faf08a941..fe41cc5805 100644
@@ -19,7 +19,7 @@ riscv_ss.add(files(
   'bitmanip_helper.c',
   'translate.c',
 ))
-riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'))
+riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c'))
 
 riscv_softmmu_ss = ss.source_set()
 riscv_softmmu_ss.add(files(
Signature · 1 line
2.19.1
maintainer
#7 lore raw

Re: [PATCH v3 06/12] target/riscv: Support start kernel directly by KVM

Quoted 38 lines · Yifei Jiang

Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. Add kvm_riscv.h to place riscv specific interface.

In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.

---
 hw/intc/sifive_plic.c    |  8 +++-
 hw/riscv/boot.c          | 16 +++++++-
 hw/riscv/virt.c          | 87 ++++++++++++++++++++++++++++------------
 include/hw/riscv/boot.h  |  1 +
 target/riscv/cpu.c       |  8 ++++
 target/riscv/cpu.h       |  3 ++
 target/riscv/kvm-stub.c  | 25 ++++++++++++
 target/riscv/kvm.c       | 14 +++++++
 target/riscv/kvm_riscv.h | 24 +++++++++++
 target/riscv/meson.build |  2 +-
 10 files changed, 159 insertions(+), 29 deletions(-)
 create mode 100644 target/riscv/kvm-stub.c
 create mode 100644 target/riscv/kvm_riscv.h
hw/intc/sifive_plic.c +1 -0
index 877e76877c..1b2b4cc25e 100644
@@ -30,6 +30,7 @@
 #include "target/riscv/cpu.h"
 #include "migration/vmstate.h"
 #include "hw/irq.h"
+#include "sysemu/kvm.h"
 
 #define RISCV_DEBUG_PLIC 0
 

hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, · line 563

Quoted 10 lines
hw/intc/sifive_plic.c +5 -2
@@ -555,8 +556,11 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,
 
         qdev_connect_gpio_out(dev, i,
                               qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT));
-        qdev_connect_gpio_out(dev, num_harts + i,
-                              qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
+
+        if (!kvm_enabled()) {
+            qdev_connect_gpio_out(dev, num_harts + i,
+                                  qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
+        }

We should have any KVM enabled check in interrupt controller emulators instead I suggest to re-write the for-loop in sifive_plic_create() as follows:

SiFivePLICState *plic;

(....)

plic = SIFIVE_PLIC(dev);

for (i = 0; i < plic->num_addrs; i++) {
    CPUState *cpu = qemu_get_cpu(plic->addr_config[i].hartid);
   if (plic->addr_mode[i] == PLICMode_S) {
       qdev_connect_gpio_out(dev, i,
            qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT));
   }
   if (plic->addr_mode[i] == PLICMode_M) {
        qdev_connect_gpio_out(dev, num_harts + i,
            qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
    }
}
Quoted 54 lines
hw/intc/sifive_plic.c +0 -0
     }
 
     return dev;
hw/riscv/boot.c +15 -1
index 519fa455a1..ccff662d89 100644
@@ -30,6 +30,7 @@
 #include "elf.h"
 #include "sysemu/device_tree.h"
 #include "sysemu/qtest.h"
+#include "sysemu/kvm.h"
 
 #include <libfdt.h>
 
@@ -51,7 +52,9 @@ char *riscv_plic_hart_config_string(int hart_count)
         CPUState *cs = qemu_get_cpu(i);
         CPURISCVState *env = &RISCV_CPU(cs)->env;
 
-        if (riscv_has_ext(env, RVS)) {
+        if (kvm_enabled()) {
+            vals[i] = "S";
+        } else if (riscv_has_ext(env, RVS)) {
             vals[i] = "MS";
         } else {
             vals[i] = "M";
@@ -317,3 +320,14 @@ void riscv_setup_rom_reset_vec(MachineState *machine, RISCVHartArrayState *harts
 
     return;
 }
+
+void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr fdt_addr)
+{
+    CPUState *cs;
+
+    for (cs = first_cpu; cs; cs = CPU_NEXT(cs)) {
+        RISCVCPU *riscv_cpu = RISCV_CPU(cs);
+        riscv_cpu->env.kernel_addr = kernel_addr;
+        riscv_cpu->env.fdt_addr = fdt_addr;
+    }
+}
hw/riscv/virt.c +1 -0
index 3af074148e..cc1a03f284 100644
@@ -38,6 +38,7 @@
 #include "chardev/char.h"
 #include "sysemu/device_tree.h"
 #include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
 #include "hw/pci/pci.h"
 #include "hw/pci-host/gpex.h"
 #include "hw/display/ramfb.h"

hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 58

Quoted 8 lines
hw/riscv/virt.c +4 -0
@@ -50,7 +51,11 @@ static const MemMapEntry virt_memmap[] = {
     [VIRT_CLINT] =       {  0x2000000,       0x10000 },
     [VIRT_ACLINT_SSWI] = {  0x2F00000,        0x4000 },
     [VIRT_PCIE_PIO] =    {  0x3000000,       0x10000 },
+#if defined(CONFIG_KVM)
+    [VIRT_PLIC] =        {  0xc000000, VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 1) },
+#else
     [VIRT_PLIC] =        {  0xc000000, VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 2) },
+#endif

Please drop this change because the same QEMU binary should be usable with and without KVM enabled.

Quoted 281 lines
hw/riscv/virt.c +57 -25
     [VIRT_UART0] =       { 0x10000000,         0x100 },
     [VIRT_VIRTIO] =      { 0x10001000,        0x1000 },
     [VIRT_FW_CFG] =      { 0x10100000,          0x18 },
@@ -372,13 +377,22 @@ static void create_fdt_socket_plic(RISCVVirtState *s,
         "sifive,plic-1.0.0", "riscv,plic0"
     };
 
-    plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4);
+    if (kvm_enabled()) {
+        plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2);
+    } else {
+        plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4);
+    }
 
     for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
-        plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]);
-        plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT);
-        plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]);
-        plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT);
+        if (kvm_enabled()) {
+            plic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
+            plic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
+        } else {
+            plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]);
+            plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT);
+            plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]);
+            plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT);
+        }
     }
 
     plic_phandles[socket] = (*phandle)++;
@@ -436,10 +450,12 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,
 
         create_fdt_socket_memory(s, memmap, socket);
 
-        if (s->have_aclint) {
-            create_fdt_socket_aclint(s, memmap, socket, intc_phandles);
-        } else {
-            create_fdt_socket_clint(s, memmap, socket, intc_phandles);
+        if (!kvm_enabled()) {
+            if (s->have_aclint) {
+                create_fdt_socket_aclint(s, memmap, socket, intc_phandles);
+            } else {
+                create_fdt_socket_clint(s, memmap, socket, intc_phandles);
+            }
         }
 
         create_fdt_socket_plic(s, memmap, socket, phandle,
@@ -801,23 +817,25 @@ static void virt_machine_init(MachineState *machine)
                                 hart_count, &error_abort);
         sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort);
 
-        /* Per-socket CLINT */
-        riscv_aclint_swi_create(
-            memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size,
-            base_hartid, hart_count, false);
-        riscv_aclint_mtimer_create(
-            memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size +
-                RISCV_ACLINT_SWI_SIZE,
-            RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid, hart_count,
-            RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
-            RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true);
-
-        /* Per-socket ACLINT SSWI */
-        if (s->have_aclint) {
+        if (!kvm_enabled()) {
+            /* Per-socket CLINT */
             riscv_aclint_swi_create(
-                memmap[VIRT_ACLINT_SSWI].base +
-                    i * memmap[VIRT_ACLINT_SSWI].size,
-                base_hartid, hart_count, true);
+                memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size,
+                base_hartid, hart_count, false);
+            riscv_aclint_mtimer_create(
+                memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size +
+                    RISCV_ACLINT_SWI_SIZE,
+                RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid, hart_count,
+                RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
+                RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true);
+
+            /* Per-socket ACLINT SSWI */
+            if (s->have_aclint) {
+                riscv_aclint_swi_create(
+                    memmap[VIRT_ACLINT_SSWI].base +
+                        i * memmap[VIRT_ACLINT_SSWI].size,
+                    base_hartid, hart_count, true);
+            }
         }
 
         /* Per-socket PLIC hart topology configuration string */
@@ -884,6 +902,16 @@ static void virt_machine_init(MachineState *machine)
     memory_region_add_subregion(system_memory, memmap[VIRT_MROM].base,
                                 mask_rom);
 
+    /*
+     * Only direct boot kernel is currently supported for KVM VM,
+     * so the "-bios" parameter is ignored and treated like "-bios none"
+     * when KVM is enabled.
+     */
+    if (kvm_enabled()) {
+        g_free(machine->firmware);
+        machine->firmware = g_strdup("none");
+    }
+
     if (riscv_is_32bit(&s->soc[0])) {
         firmware_end_addr = riscv_find_and_load_firmware(machine,
                                     RISCV32_BIOS_BIN, start_addr, NULL);
@@ -941,6 +969,15 @@ static void virt_machine_init(MachineState *machine)
                               virt_memmap[VIRT_MROM].size, kernel_entry,
                               fdt_load_addr, machine->fdt);
 
+    /*
+     * Only direct boot kernel is currently supported for KVM VM,
+     * So here setup kernel start address and fdt address.
+     * TODO:Support firmware loading and integrate to TCG start
+     */
+    if (kvm_enabled()) {
+        riscv_setup_direct_kernel(kernel_entry, fdt_load_addr);
+    }
+
     /* SiFive Test MMIO device */
     sifive_test_create(memmap[VIRT_TEST].base);
 
include/hw/riscv/boot.h +1 -0
index baff11dd8a..5834c234aa 100644
@@ -58,5 +58,6 @@ void riscv_rom_copy_firmware_info(MachineState *machine, hwaddr rom_base,
                                   hwaddr rom_size,
                                   uint32_t reset_vec_size,
                                   uint64_t kernel_entry);
+void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr fdt_addr);
 
 #endif /* RISCV_BOOT_H */
target/riscv/cpu.c +8 -0
index f812998123..1c944872a3 100644
@@ -29,6 +29,8 @@
 #include "hw/qdev-properties.h"
 #include "migration/vmstate.h"
 #include "fpu/softfloat-helpers.h"
+#include "sysemu/kvm.h"
+#include "kvm_riscv.h"
 
 /* RISC-V CPU definitions */
 
@@ -380,6 +382,12 @@ static void riscv_cpu_reset(DeviceState *dev)
     cs->exception_index = RISCV_EXCP_NONE;
     env->load_res = -1;
     set_default_nan_mode(1, &env->fp_status);
+
+#ifndef CONFIG_USER_ONLY
+    if (kvm_enabled()) {
+        kvm_riscv_reset_vcpu(cpu);
+    }
+#endif
 }
 
 static void riscv_cpu_disas_set_info(CPUState *s, disassemble_info *info)
target/riscv/cpu.h +3 -0
index 0760c0af93..2807eb1bcb 100644
@@ -255,6 +255,9 @@ struct CPURISCVState {
 
     /* Fields from here on are preserved across CPU reset. */
     QEMUTimer *timer; /* Internal timer */
+
+    hwaddr kernel_addr;
+    hwaddr fdt_addr;
 };
 
 OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,
target/riscv/kvm-stub.c new file +25 -0
new file mode 100644
index 0000000000..39b96fe3f4
@@ -0,0 +1,25 @@
+/*
+ * QEMU KVM RISC-V specific function stubs
+ *
+ * Copyright (c) 2020 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "kvm_riscv.h"
+
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
+{
+    abort();
+}
target/riscv/kvm.c +14 -0
index e695b91dc7..db6d8a5b6e 100644
@@ -37,6 +37,7 @@
 #include "hw/irq.h"
 #include "qemu/log.h"
 #include "hw/loader.h"
+#include "kvm_riscv.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx)
 {
@@ -369,6 +370,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
     return 0;
 }
 
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
+{
+    CPURISCVState *env = &cpu->env;
+
+    if (!kvm_enabled()) {
+        return;
+    }
+    env->pc = cpu->env.kernel_addr;
+    env->gpr[10] = kvm_arch_vcpu_id(CPU(cpu)); /* a0 */
+    env->gpr[11] = cpu->env.fdt_addr;          /* a1 */
+    env->satp = 0;
+}
+
 bool kvm_arch_cpu_check_are_resettable(void)
 {
     return true;
target/riscv/kvm_riscv.h new file +24 -0
new file mode 100644
index 0000000000..f38c82bf59
@@ -0,0 +1,24 @@
+/*
+ * QEMU KVM support -- RISC-V specific functions.
+ *
+ * Copyright (c) 2020 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef QEMU_KVM_RISCV_H
+#define QEMU_KVM_RISCV_H
+
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
+
+#endif
target/riscv/meson.build +1 -2
index 2faf08a941..fe41cc5805 100644
@@ -19,7 +19,7 @@ riscv_ss.add(files(
   'bitmanip_helper.c',
   'translate.c',
 ))
-riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'))
+riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c'))
 
 riscv_softmmu_ss = ss.source_set()
 riscv_softmmu_ss.add(files(
--
Quoted 2 lines

2.19.1

Regards,
Anup

maintainer
#8 lore raw

Re: [PATCH v3 06/12] target/riscv: Support start kernel directly by KVM

Quoted 41 lines · Anup Patel
Quoted 39 lines · Yifei Jiang

Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. Add kvm_riscv.h to place riscv specific interface.

In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.

---
 hw/intc/sifive_plic.c    |  8 +++-
 hw/riscv/boot.c          | 16 +++++++-
 hw/riscv/virt.c          | 87 ++++++++++++++++++++++++++++------------
 include/hw/riscv/boot.h  |  1 +
 target/riscv/cpu.c       |  8 ++++
 target/riscv/cpu.h       |  3 ++
 target/riscv/kvm-stub.c  | 25 ++++++++++++
 target/riscv/kvm.c       | 14 +++++++
 target/riscv/kvm_riscv.h | 24 +++++++++++
 target/riscv/meson.build |  2 +-
 10 files changed, 159 insertions(+), 29 deletions(-)
 create mode 100644 target/riscv/kvm-stub.c
 create mode 100644 target/riscv/kvm_riscv.h
hw/intc/sifive_plic.c +1 -0
index 877e76877c..1b2b4cc25e 100644
@@ -30,6 +30,7 @@
 #include "target/riscv/cpu.h"
 #include "migration/vmstate.h"
 #include "hw/irq.h"
+#include "sysemu/kvm.h"
 
 #define RISCV_DEBUG_PLIC 0
 
@@ -555,8 +556,11 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,
 
Quoted 12 lines

hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, · line 563

Quoted 9 lines
hw/intc/sifive_plic.c +5 -2
         qdev_connect_gpio_out(dev, i,
                               qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT));
-        qdev_connect_gpio_out(dev, num_harts + i,
-                              qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
+
+        if (!kvm_enabled()) {
+            qdev_connect_gpio_out(dev, num_harts + i,
+                                  qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
+        }

We should have any KVM enabled check in interrupt controller emulators instead I suggest to re-write the for-loop in sifive_plic_create() as follows:

Typo correction:
*"We should not have any KVM enabled check in ...."

Regards,
Anup

Quoted 371 lines

SiFivePLICState *plic;

(....)

plic = SIFIVE_PLIC(dev);

for (i = 0; i < plic->num_addrs; i++) {
    CPUState *cpu = qemu_get_cpu(plic->addr_config[i].hartid);
   if (plic->addr_mode[i] == PLICMode_S) {
       qdev_connect_gpio_out(dev, i,
            qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT));
   }
   if (plic->addr_mode[i] == PLICMode_M) {
        qdev_connect_gpio_out(dev, num_harts + i,
            qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
    }
}
Quoted 54 lines
hw/intc/sifive_plic.c +0 -0
     }
 
     return dev;
hw/riscv/boot.c +15 -1
index 519fa455a1..ccff662d89 100644
@@ -30,6 +30,7 @@
 #include "elf.h"
 #include "sysemu/device_tree.h"
 #include "sysemu/qtest.h"
+#include "sysemu/kvm.h"
 
 #include <libfdt.h>
 
@@ -51,7 +52,9 @@ char *riscv_plic_hart_config_string(int hart_count)
         CPUState *cs = qemu_get_cpu(i);
         CPURISCVState *env = &RISCV_CPU(cs)->env;
 
-        if (riscv_has_ext(env, RVS)) {
+        if (kvm_enabled()) {
+            vals[i] = "S";
+        } else if (riscv_has_ext(env, RVS)) {
             vals[i] = "MS";
         } else {
             vals[i] = "M";
@@ -317,3 +320,14 @@ void riscv_setup_rom_reset_vec(MachineState *machine, RISCVHartArrayState *harts
 
     return;
 }
+
+void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr fdt_addr)
+{
+    CPUState *cs;
+
+    for (cs = first_cpu; cs; cs = CPU_NEXT(cs)) {
+        RISCVCPU *riscv_cpu = RISCV_CPU(cs);
+        riscv_cpu->env.kernel_addr = kernel_addr;
+        riscv_cpu->env.fdt_addr = fdt_addr;
+    }
+}
hw/riscv/virt.c +1 -0
index 3af074148e..cc1a03f284 100644
@@ -38,6 +38,7 @@
 #include "chardev/char.h"
 #include "sysemu/device_tree.h"
 #include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
 #include "hw/pci/pci.h"
 #include "hw/pci-host/gpex.h"
 #include "hw/display/ramfb.h"

hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 58

Quoted 8 lines
hw/riscv/virt.c +4 -0
@@ -50,7 +51,11 @@ static const MemMapEntry virt_memmap[] = {
     [VIRT_CLINT] =       {  0x2000000,       0x10000 },
     [VIRT_ACLINT_SSWI] = {  0x2F00000,        0x4000 },
     [VIRT_PCIE_PIO] =    {  0x3000000,       0x10000 },
+#if defined(CONFIG_KVM)
+    [VIRT_PLIC] =        {  0xc000000, VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 1) },
+#else
     [VIRT_PLIC] =        {  0xc000000, VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 2) },
+#endif

Please drop this change because the same QEMU binary should be usable with and without KVM enabled.

Quoted 281 lines
hw/riscv/virt.c +57 -25
     [VIRT_UART0] =       { 0x10000000,         0x100 },
     [VIRT_VIRTIO] =      { 0x10001000,        0x1000 },
     [VIRT_FW_CFG] =      { 0x10100000,          0x18 },
@@ -372,13 +377,22 @@ static void create_fdt_socket_plic(RISCVVirtState *s,
         "sifive,plic-1.0.0", "riscv,plic0"
     };
 
-    plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4);
+    if (kvm_enabled()) {
+        plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2);
+    } else {
+        plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4);
+    }
 
     for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
-        plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]);
-        plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT);
-        plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]);
-        plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT);
+        if (kvm_enabled()) {
+            plic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
+            plic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
+        } else {
+            plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]);
+            plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT);
+            plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]);
+            plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT);
+        }
     }
 
     plic_phandles[socket] = (*phandle)++;
@@ -436,10 +450,12 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,
 
         create_fdt_socket_memory(s, memmap, socket);
 
-        if (s->have_aclint) {
-            create_fdt_socket_aclint(s, memmap, socket, intc_phandles);
-        } else {
-            create_fdt_socket_clint(s, memmap, socket, intc_phandles);
+        if (!kvm_enabled()) {
+            if (s->have_aclint) {
+                create_fdt_socket_aclint(s, memmap, socket, intc_phandles);
+            } else {
+                create_fdt_socket_clint(s, memmap, socket, intc_phandles);
+            }
         }
 
         create_fdt_socket_plic(s, memmap, socket, phandle,
@@ -801,23 +817,25 @@ static void virt_machine_init(MachineState *machine)
                                 hart_count, &error_abort);
         sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort);
 
-        /* Per-socket CLINT */
-        riscv_aclint_swi_create(
-            memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size,
-            base_hartid, hart_count, false);
-        riscv_aclint_mtimer_create(
-            memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size +
-                RISCV_ACLINT_SWI_SIZE,
-            RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid, hart_count,
-            RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
-            RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true);
-
-        /* Per-socket ACLINT SSWI */
-        if (s->have_aclint) {
+        if (!kvm_enabled()) {
+            /* Per-socket CLINT */
             riscv_aclint_swi_create(
-                memmap[VIRT_ACLINT_SSWI].base +
-                    i * memmap[VIRT_ACLINT_SSWI].size,
-                base_hartid, hart_count, true);
+                memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size,
+                base_hartid, hart_count, false);
+            riscv_aclint_mtimer_create(
+                memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size +
+                    RISCV_ACLINT_SWI_SIZE,
+                RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid, hart_count,
+                RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
+                RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true);
+
+            /* Per-socket ACLINT SSWI */
+            if (s->have_aclint) {
+                riscv_aclint_swi_create(
+                    memmap[VIRT_ACLINT_SSWI].base +
+                        i * memmap[VIRT_ACLINT_SSWI].size,
+                    base_hartid, hart_count, true);
+            }
         }
 
         /* Per-socket PLIC hart topology configuration string */
@@ -884,6 +902,16 @@ static void virt_machine_init(MachineState *machine)
     memory_region_add_subregion(system_memory, memmap[VIRT_MROM].base,
                                 mask_rom);
 
+    /*
+     * Only direct boot kernel is currently supported for KVM VM,
+     * so the "-bios" parameter is ignored and treated like "-bios none"
+     * when KVM is enabled.
+     */
+    if (kvm_enabled()) {
+        g_free(machine->firmware);
+        machine->firmware = g_strdup("none");
+    }
+
     if (riscv_is_32bit(&s->soc[0])) {
         firmware_end_addr = riscv_find_and_load_firmware(machine,
                                     RISCV32_BIOS_BIN, start_addr, NULL);
@@ -941,6 +969,15 @@ static void virt_machine_init(MachineState *machine)
                               virt_memmap[VIRT_MROM].size, kernel_entry,
                               fdt_load_addr, machine->fdt);
 
+    /*
+     * Only direct boot kernel is currently supported for KVM VM,
+     * So here setup kernel start address and fdt address.
+     * TODO:Support firmware loading and integrate to TCG start
+     */
+    if (kvm_enabled()) {
+        riscv_setup_direct_kernel(kernel_entry, fdt_load_addr);
+    }
+
     /* SiFive Test MMIO device */
     sifive_test_create(memmap[VIRT_TEST].base);
 
include/hw/riscv/boot.h +1 -0
index baff11dd8a..5834c234aa 100644
@@ -58,5 +58,6 @@ void riscv_rom_copy_firmware_info(MachineState *machine, hwaddr rom_base,
                                   hwaddr rom_size,
                                   uint32_t reset_vec_size,
                                   uint64_t kernel_entry);
+void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr fdt_addr);
 
 #endif /* RISCV_BOOT_H */
target/riscv/cpu.c +8 -0
index f812998123..1c944872a3 100644
@@ -29,6 +29,8 @@
 #include "hw/qdev-properties.h"
 #include "migration/vmstate.h"
 #include "fpu/softfloat-helpers.h"
+#include "sysemu/kvm.h"
+#include "kvm_riscv.h"
 
 /* RISC-V CPU definitions */
 
@@ -380,6 +382,12 @@ static void riscv_cpu_reset(DeviceState *dev)
     cs->exception_index = RISCV_EXCP_NONE;
     env->load_res = -1;
     set_default_nan_mode(1, &env->fp_status);
+
+#ifndef CONFIG_USER_ONLY
+    if (kvm_enabled()) {
+        kvm_riscv_reset_vcpu(cpu);
+    }
+#endif
 }
 
 static void riscv_cpu_disas_set_info(CPUState *s, disassemble_info *info)
target/riscv/cpu.h +3 -0
index 0760c0af93..2807eb1bcb 100644
@@ -255,6 +255,9 @@ struct CPURISCVState {
 
     /* Fields from here on are preserved across CPU reset. */
     QEMUTimer *timer; /* Internal timer */
+
+    hwaddr kernel_addr;
+    hwaddr fdt_addr;
 };
 
 OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,
target/riscv/kvm-stub.c new file +25 -0
new file mode 100644
index 0000000000..39b96fe3f4
@@ -0,0 +1,25 @@
+/*
+ * QEMU KVM RISC-V specific function stubs
+ *
+ * Copyright (c) 2020 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "kvm_riscv.h"
+
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
+{
+    abort();
+}
target/riscv/kvm.c +14 -0
index e695b91dc7..db6d8a5b6e 100644
@@ -37,6 +37,7 @@
 #include "hw/irq.h"
 #include "qemu/log.h"
 #include "hw/loader.h"
+#include "kvm_riscv.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx)
 {
@@ -369,6 +370,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
     return 0;
 }
 
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
+{
+    CPURISCVState *env = &cpu->env;
+
+    if (!kvm_enabled()) {
+        return;
+    }
+    env->pc = cpu->env.kernel_addr;
+    env->gpr[10] = kvm_arch_vcpu_id(CPU(cpu)); /* a0 */
+    env->gpr[11] = cpu->env.fdt_addr;          /* a1 */
+    env->satp = 0;
+}
+
 bool kvm_arch_cpu_check_are_resettable(void)
 {
     return true;
target/riscv/kvm_riscv.h new file +24 -0
new file mode 100644
index 0000000000..f38c82bf59
@@ -0,0 +1,24 @@
+/*
+ * QEMU KVM support -- RISC-V specific functions.
+ *
+ * Copyright (c) 2020 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef QEMU_KVM_RISCV_H
+#define QEMU_KVM_RISCV_H
+
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
+
+#endif
target/riscv/meson.build +1 -2
index 2faf08a941..fe41cc5805 100644
@@ -19,7 +19,7 @@ riscv_ss.add(files(
   'bitmanip_helper.c',
   'translate.c',
 ))
-riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'))
+riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c'))
 
 riscv_softmmu_ss = ss.source_set()
 riscv_softmmu_ss.add(files(
--
Quoted 2 lines

2.19.1

Regards,
Anup

author
#9 lore raw

RE: [PATCH v3 06/12] target/riscv: Support start kernel directly by KVM

Quoted 74 lines

-----Original Message-----
From: Anup Patel [mailto:anup at brainfault.org]
Sent: Thursday, December 23, 2021 2:04 PM
To: Jiangyifei <[email protected]>
Cc: QEMU Developers <[email protected]>; open list:RISC-V <[email protected]>; kvm-riscv at lists.infradead.org; KVM General <[email protected]>; libvir-list at redhat.com; Anup Patel <[email protected]>; Palmer Dabbelt <[email protected]>; Alistair Francis <[email protected]>; Bin Meng <[email protected]>; Fanliang (EulerOS) <[email protected]>; Wubin (H) <[email protected]>; Wanghaibin (D) <[email protected]>; wanbo (G) <[email protected]>; limingwang (A) <[email protected]>
Subject: Re: [PATCH v3 06/12] target/riscv: Support start kernel directly by KVM

Quoted 2 lines · Anup Patel

On Mon, Dec 20, 2021 at 6:39 PM Yifei Jiang <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines

Get kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. Add kvm_riscv.h to place riscv specific interface.

In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.

Quoted 7 lines
Quoted 7 lines
---
 hw/intc/sifive_plic.c    |  8 +++-
 hw/riscv/boot.c          | 16 +++++++-
 hw/riscv/virt.c          | 87

++++++++++++++++++++++++++++------------

Quoted 21 lines
Quoted 21 lines
 include/hw/riscv/boot.h  |  1 +
 target/riscv/cpu.c       |  8 ++++
 target/riscv/cpu.h       |  3 ++
 target/riscv/kvm-stub.c  | 25 ++++++++++++
 target/riscv/kvm.c       | 14 +++++++
 target/riscv/kvm_riscv.h | 24 +++++++++++  target/riscv/meson.build
|  2 +-
 10 files changed, 159 insertions(+), 29 deletions(-)  create mode

100644 target/riscv/kvm-stub.c create mode 100644 target/riscv/kvm_riscv.h

hw/intc/sifive_plic.c index renamed from hw/intc/sifive_plic.c +0 -0

877e76877c..1b2b4cc25e 100644

hw/intc/sifive_plic.c +1 -0
@@ -30,6 +30,7 @@
 #include "target/riscv/cpu.h"
 #include "migration/vmstate.h"
 #include "hw/irq.h"
+#include "sysemu/kvm.h"
 
Quoted 7 lines

hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, · line 559

Quoted 7 lines
hw/intc/sifive_plic.c +0 -0
 #define RISCV_DEBUG_PLIC 0
 
@@ -555,8 +556,11 @@ DeviceState *sifive_plic_create(hwaddr addr,
 char *hart_config,
 
         qdev_connect_gpio_out(dev, i,
                               qdev_get_gpio_in(DEVICE(cpu),

IRQ_S_EXT));

Quoted 2 lines

hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, · line 560

Quoted 2 lines
hw/intc/sifive_plic.c +0 -2
-        qdev_connect_gpio_out(dev, num_harts + i,
-                              qdev_get_gpio_in(DEVICE(cpu),

IRQ_M_EXT));

Quoted 4 lines

hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,

Quoted 4 lines
hw/intc/sifive_plic.c +4 -0
+
+        if (!kvm_enabled()) {
+            qdev_connect_gpio_out(dev, num_harts + i,
+                                  qdev_get_gpio_in(DEVICE(cpu),

IRQ_M_EXT));

Quoted 2 lines
Quoted 1 line
hw/intc/sifive_plic.c +1 -0
+        }
Quoted 28 lines
Quoted 2 lines

We should have any KVM enabled check in interrupt controller emulators instead I suggest to re-write the for-loop in sifive_plic_create() as follows:

Typo correction:
*"We should not have any KVM enabled check in ...."

Regards,
Anup

Quoted 19 lines

SiFivePLICState *plic;

(....)

plic = SIFIVE_PLIC(dev);

for (i = 0; i < plic->num_addrs; i++) {
    CPUState *cpu = qemu_get_cpu(plic->addr_config[i].hartid);
   if (plic->addr_mode[i] == PLICMode_S) {
       qdev_connect_gpio_out(dev, i,
            qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT));
   }
   if (plic->addr_mode[i] == PLICMode_M) {
        qdev_connect_gpio_out(dev, num_harts + i,
            qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
    }
}

Yes, this looks better. It will be re-write in the next series as suggested.

Quoted 61 lines
Quoted 52 lines
Quoted 52 lines
hw/intc/sifive_plic.c +0 -0
     }
 
     return dev;
hw/riscv/boot.c index renamed from hw/riscv/boot.c +0 -0

519fa455a1..ccff662d89 100644

hw/riscv/boot.c +15 -1
@@ -30,6 +30,7 @@
 #include "elf.h"
 #include "sysemu/device_tree.h"
 #include "sysemu/qtest.h"
+#include "sysemu/kvm.h"
 
 #include <libfdt.h>
 
@@ -51,7 +52,9 @@ char *riscv_plic_hart_config_string(int hart_count)
         CPUState *cs = qemu_get_cpu(i);
         CPURISCVState *env = &RISCV_CPU(cs)->env;
 
-        if (riscv_has_ext(env, RVS)) {
+        if (kvm_enabled()) {
+            vals[i] = "S";
+        } else if (riscv_has_ext(env, RVS)) {
             vals[i] = "MS";
         } else {
             vals[i] = "M";
@@ -317,3 +320,14 @@ void riscv_setup_rom_reset_vec(MachineState
 *machine, RISCVHartArrayState *harts
 
     return;
 }
+
+void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr fdt_addr)
+{
+    CPUState *cs;
+
+    for (cs = first_cpu; cs; cs = CPU_NEXT(cs)) {
+        RISCVCPU *riscv_cpu = RISCV_CPU(cs);
+        riscv_cpu->env.kernel_addr = kernel_addr;
+        riscv_cpu->env.fdt_addr = fdt_addr;
+    }
+}
hw/riscv/virt.c index renamed from hw/riscv/virt.c +0 -0

3af074148e..cc1a03f284 100644

hw/riscv/virt.c +1 -0
@@ -38,6 +38,7 @@
 #include "chardev/char.h"
 #include "sysemu/device_tree.h"
 #include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
 #include "hw/pci/pci.h"
Quoted 8 lines

hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 55

Quoted 8 lines
hw/riscv/virt.c +2 -0
 #include "hw/pci-host/gpex.h"
 #include "hw/display/ramfb.h"
@@ -50,7 +51,11 @@ static const MemMapEntry virt_memmap[] = {
     [VIRT_CLINT] =       {  0x2000000,       0x10000 },
     [VIRT_ACLINT_SSWI] = {  0x2F00000,        0x4000 },
     [VIRT_PCIE_PIO] =    {  0x3000000,       0x10000 },
+#if defined(CONFIG_KVM)
+    [VIRT_PLIC] =        {  0xc000000,

VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 1) },

Quoted 8 lines
Quoted 2 lines

hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 57

Quoted 2 lines
hw/riscv/virt.c +1 -0
+#else
     [VIRT_PLIC] =        {  0xc000000,

VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 2) },

Quoted 5 lines

hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 58

Quoted 1 line
hw/riscv/virt.c +1 -0
+#endif

Please drop this change because the same QEMU binary should be usable with and without KVM enabled.

It will be dropped in the next series.

Yifei

Quoted 315 lines
Quoted 4 lines

hw/riscv/virt.c static void

Quoted 4 lines
hw/riscv/virt.c +0 -0
     [VIRT_UART0] =       { 0x10000000,         0x100 },
     [VIRT_VIRTIO] =      { 0x10001000,        0x1000 },
     [VIRT_FW_CFG] =      { 0x10100000,          0x18 },
@@ -372,13 +377,22 @@ static void

create_fdt_socket_plic(RISCVVirtState *s,

Quoted 26 lines
Quoted 26 lines
hw/riscv/virt.c +14 -5
         "sifive,plic-1.0.0", "riscv,plic0"
     };
 
-    plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4);
+    if (kvm_enabled()) {
+        plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2);
+    } else {
+        plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4);
+    }
 
     for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) {
-        plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]);
-        plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT);
-        plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]);
-        plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT);
+        if (kvm_enabled()) {
+            plic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
+            plic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
+        } else {
+            plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandles[cpu]);
+            plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT);
+            plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandles[cpu]);
+            plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT);
+        }
     }
 
Quoted 8 lines

hw/riscv/virt.c

Quoted 8 lines
hw/riscv/virt.c +0 -0
     plic_phandles[socket] = (*phandle)++; @@ -436,10 +450,12 @@

static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,

         create_fdt_socket_memory(s, memmap, socket);
hw/riscv/virt.c +0 -2
-        if (s->have_aclint) {
-            create_fdt_socket_aclint(s, memmap, socket,

intc_phandles);

Quoted 2 lines

hw/riscv/virt.c

Quoted 2 lines
hw/riscv/virt.c +0 -2
-        } else {
-            create_fdt_socket_clint(s, memmap, socket,

intc_phandles);

Quoted 3 lines

hw/riscv/virt.c static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,

Quoted 3 lines
hw/riscv/virt.c +3 -0
+        if (!kvm_enabled()) {
+            if (s->have_aclint) {
+                create_fdt_socket_aclint(s, memmap, socket,

intc_phandles);

Quoted 2 lines

hw/riscv/virt.c

Quoted 2 lines
hw/riscv/virt.c +2 -0
+            } else {
+                create_fdt_socket_clint(s, memmap, socket,

intc_phandles);

Quoted 11 lines

hw/riscv/virt.c

Quoted 11 lines
hw/riscv/virt.c +1 -4
+            }
         }
 
         create_fdt_socket_plic(s, memmap, socket, phandle, @@
-801,23 +817,25 @@ static void virt_machine_init(MachineState *machine)
                                 hart_count, &error_abort);
         sysbus_realize(SYS_BUS_DEVICE(&s->soc[i]), &error_abort);
 
-        /* Per-socket CLINT */
-        riscv_aclint_swi_create(
-            memmap[VIRT_CLINT].base + i *

memmap[VIRT_CLINT].size,

Quoted 3 lines

hw/riscv/virt.c static void virt_machine_init(MachineState *machine)

Quoted 3 lines
hw/riscv/virt.c +0 -3
-            base_hartid, hart_count, false);
-        riscv_aclint_mtimer_create(
-            memmap[VIRT_CLINT].base + i *

memmap[VIRT_CLINT].size +

Quoted 2 lines

hw/riscv/virt.c

Quoted 2 lines
hw/riscv/virt.c +0 -2
-                RISCV_ACLINT_SWI_SIZE,
-            RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid,

hart_count,

Quoted 1 line
Quoted 1 line

- RISCV_ACLINT_DEFAULT_MTIMECMP,

RISCV_ACLINT_DEFAULT_MTIME,

Quoted 11 lines

hw/riscv/virt.c static void virt_machine_init(MachineState *machine)

Quoted 11 lines
hw/riscv/virt.c +3 -7
-            RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true);
-
-        /* Per-socket ACLINT SSWI */
-        if (s->have_aclint) {
+        if (!kvm_enabled()) {
+            /* Per-socket CLINT */
             riscv_aclint_swi_create(
-                memmap[VIRT_ACLINT_SSWI].base +
-                    i * memmap[VIRT_ACLINT_SSWI].size,
-                base_hartid, hart_count, true);
+                memmap[VIRT_CLINT].base + i *

memmap[VIRT_CLINT].size,

Quoted 3 lines

hw/riscv/virt.c static void virt_machine_init(MachineState *machine)

Quoted 3 lines
hw/riscv/virt.c +3 -0
+                base_hartid, hart_count, false);
+            riscv_aclint_mtimer_create(
+                memmap[VIRT_CLINT].base + i *

memmap[VIRT_CLINT].size +

Quoted 2 lines

hw/riscv/virt.c

Quoted 2 lines
hw/riscv/virt.c +2 -0
+                    RISCV_ACLINT_SWI_SIZE,
+                RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid,

hart_count,

Quoted 1 line
Quoted 1 line

+ RISCV_ACLINT_DEFAULT_MTIMECMP,

RISCV_ACLINT_DEFAULT_MTIME,

Quoted 11 lines
Quoted 11 lines
hw/riscv/virt.c +9 -0
+                RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true);
+
+            /* Per-socket ACLINT SSWI */
+            if (s->have_aclint) {
+                riscv_aclint_swi_create(
+                    memmap[VIRT_ACLINT_SSWI].base +
+                        i * memmap[VIRT_ACLINT_SSWI].size,
+                    base_hartid, hart_count, true);
+            }
         }
Quoted 3 lines
Quoted 3 lines
         /* Per-socket PLIC hart topology configuration string */ @@

-884,6 +902,16 @@ static void virt_machine_init(MachineState *machine)
memory_region_add_subregion(system_memory,

memmap[VIRT_MROM].base,

Quoted 15 lines
Quoted 5 lines
                                 mask_rom);
hw/riscv/virt.c +3 -0
+    /*
+     * Only direct boot kernel is currently supported for KVM VM,
+     * so the "-bios" parameter is ignored and treated like "-bios none"

hw/riscv/virt.c static void virt_machine_init(MachineState *machine)

Quoted 10 lines
hw/riscv/virt.c +7 -0
+     * when KVM is enabled.
+     */
+    if (kvm_enabled()) {
+        g_free(machine->firmware);
+        machine->firmware = g_strdup("none");
+    }
+
     if (riscv_is_32bit(&s->soc[0])) {
         firmware_end_addr = riscv_find_and_load_firmware(machine,
                                     RISCV32_BIOS_BIN,

start_addr,

Quoted 1 line
Quoted 1 line

NULL); @@ -941,6 +969,15 @@ static void

virt_machine_init(MachineState *machine)

Quoted 1 line
Quoted 1 line

virt_memmap[VIRT_MROM].size,

kernel_entry,

Quoted 14 lines
Quoted 14 lines
                               fdt_load_addr, machine->fdt);
hw/riscv/virt.c +9 -0
+    /*
+     * Only direct boot kernel is currently supported for KVM VM,
+     * So here setup kernel start address and fdt address.
+     * TODO:Support firmware loading and integrate to TCG start
+     */
+    if (kvm_enabled()) {
+        riscv_setup_direct_kernel(kernel_entry, fdt_load_addr);
+    }
+
     /* SiFive Test MMIO device */
     sifive_test_create(memmap[VIRT_TEST].base);
Quoted 5 lines

include/hw/riscv/boot.h void riscv_rom_copy_firmware_info(MachineState

Quoted 5 lines
include/hw/riscv/boot.h index renamed from include/hw/riscv/boot.h +0 -0

baff11dd8a..5834c234aa 100644

include/hw/riscv/boot.h +0 -0
@@ -58,5 +58,6 @@ void riscv_rom_copy_firmware_info(MachineState

*machine, hwaddr rom_base,

Quoted 168 lines
Quoted 163 lines
include/hw/riscv/boot.h +2 -0
                                   hwaddr rom_size,
                                   uint32_t reset_vec_size,
                                   uint64_t kernel_entry);
+void riscv_setup_direct_kernel(hwaddr kernel_addr, hwaddr
+fdt_addr);
 
 #endif /* RISCV_BOOT_H */
target/riscv/cpu.c index renamed from target/riscv/cpu.c +0 -0

f812998123..1c944872a3 100644

target/riscv/cpu.c +8 -0
@@ -29,6 +29,8 @@
 #include "hw/qdev-properties.h"
 #include "migration/vmstate.h"
 #include "fpu/softfloat-helpers.h"
+#include "sysemu/kvm.h"
+#include "kvm_riscv.h"
 
 /* RISC-V CPU definitions */
 
@@ -380,6 +382,12 @@ static void riscv_cpu_reset(DeviceState *dev)
     cs->exception_index = RISCV_EXCP_NONE;
     env->load_res = -1;
     set_default_nan_mode(1, &env->fp_status);
+
+#ifndef CONFIG_USER_ONLY
+    if (kvm_enabled()) {
+        kvm_riscv_reset_vcpu(cpu);
+    }
+#endif
 }
 
 static void riscv_cpu_disas_set_info(CPUState *s, disassemble_info

*info) diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 0760c0af93..2807eb1bcb 100644

target/riscv/cpu.h +3 -0
@@ -255,6 +255,9 @@ struct CPURISCVState {
 
     /* Fields from here on are preserved across CPU reset. */
     QEMUTimer *timer; /* Internal timer */
+
+    hwaddr kernel_addr;
+    hwaddr fdt_addr;
 };
 
 OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, diff --git

a/target/riscv/kvm-stub.c b/target/riscv/kvm-stub.c new file mode 100644 index 0000000000..39b96fe3f4

target/riscv/kvm-stub.c new file +30 -0
@@ -0,0 +1,25 @@
+/*
+ * QEMU KVM RISC-V specific function stubs
+ *
+ * Copyright (c) 2020 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+modify it
+ * under the terms and conditions of the GNU General Public
+License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but
+WITHOUT
+ * ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public
+License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public
+License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "kvm_riscv.h"
+
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu) {
+    abort();
+}
target/riscv/kvm.c index renamed from target/riscv/kvm.c +0 -0

e695b91dc7..db6d8a5b6e 100644

target/riscv/kvm.c +1 -0
@@ -37,6 +37,7 @@
 #include "hw/irq.h"
 #include "qemu/log.h"
 #include "hw/loader.h"
+#include "kvm_riscv.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type,

uint64_t idx) { @@ -369,6 +370,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)

     return 0;

}

target/riscv/kvm.c +12 -0
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu) {
+    CPURISCVState *env = &cpu->env;
+
+    if (!kvm_enabled()) {
+        return;
+    }
+    env->pc = cpu->env.kernel_addr;
+    env->gpr[10] = kvm_arch_vcpu_id(CPU(cpu)); /* a0 */
+    env->gpr[11] = cpu->env.fdt_addr;          /* a1 */
+    env->satp = 0;
+}
+
 bool kvm_arch_cpu_check_are_resettable(void)
 {
     return true;
target/riscv/kvm_riscv.h new renamed from target/riscv/kvm_riscv.h +0 -0

file mode 100644 index 0000000000..f38c82bf59

target/riscv/kvm_riscv.h new file +30 -0
@@ -0,0 +1,24 @@
+/*
+ * QEMU KVM support -- RISC-V specific functions.
+ *
+ * Copyright (c) 2020 Huawei Technologies Co., Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+modify it
+ * under the terms and conditions of the GNU General Public
+License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but
+WITHOUT
+ * ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public
+License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public
+License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef QEMU_KVM_RISCV_H
+#define QEMU_KVM_RISCV_H
+
+void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
+
+#endif
target/riscv/meson.build +2 -2
index 2faf08a941..fe41cc5805 100644
@@ -19,7 +19,7 @@ riscv_ss.add(files(
   'bitmanip_helper.c',
   'translate.c',
 ))
-riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'))
+riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false:
+files('kvm-stub.c'))
 
 riscv_softmmu_ss = ss.source_set()
 riscv_softmmu_ss.add(files(
--
Quoted 2 lines

2.19.1

Regards,
Anup

#10 lore raw

PATCH v3 7/12 target/riscv: Support setting external interrupt by KVM

In reply to Yifei Jiang

When KVM is enabled, set the S-mode external interrupt through kvm_riscv_set_irq function.

---
 target/riscv/cpu.c       |  6 +++++-
 target/riscv/kvm-stub.c  |  5 +++++
 target/riscv/kvm.c       | 17 +++++++++++++++++
 target/riscv/kvm_riscv.h |  1 +
 4 files changed, 28 insertions(+), 1 deletion(-)
target/riscv/cpu.c +5 -1
index 1c944872a3..3fc3a9c45b 100644
@@ -603,7 +603,11 @@ static void riscv_cpu_set_irq(void *opaque, int irq, int level)
     case IRQ_S_EXT:
     case IRQ_VS_EXT:
     case IRQ_M_EXT:
-        riscv_cpu_update_mip(cpu, 1 << irq, BOOL_TO_MASK(level));
+        if (kvm_enabled()) {
+            kvm_riscv_set_irq(cpu, irq, level);
+        } else {
+            riscv_cpu_update_mip(cpu, 1 << irq, BOOL_TO_MASK(level));
+        }
         break;
     default:
         g_assert_not_reached();
target/riscv/kvm-stub.c +5 -0
index 39b96fe3f4..4e8fc31a21 100644
@@ -23,3 +23,8 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
 {
     abort();
 }
+
+void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level)
+{
+    abort();
+}
target/riscv/kvm.c +17 -0
index db6d8a5b6e..0027f11f45 100644
@@ -383,6 +383,23 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
     env->satp = 0;
 }
 
+void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level)
+{
+    int ret;
+    unsigned virq = level ? KVM_INTERRUPT_SET : KVM_INTERRUPT_UNSET;
+
+    if (irq != IRQ_S_EXT) {
+        perror("kvm riscv set irq != IRQ_S_EXT\n");
+        abort();
+    }
+
+    ret = kvm_vcpu_ioctl(CPU(cpu), KVM_INTERRUPT, &virq);
+    if (ret < 0) {
+        perror("Set irq failed");
+        abort();
+    }
+}
+
 bool kvm_arch_cpu_check_are_resettable(void)
 {
     return true;
target/riscv/kvm_riscv.h +1 -0
index f38c82bf59..ed281bdce0 100644
@@ -20,5 +20,6 @@
 #define QEMU_KVM_RISCV_H
 
 void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
+void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
 
 #endif
Signature · 1 line
2.19.1
maintainer Reviewed-by on v3 7/12
#11 lore raw

Re: [PATCH v3 07/12] target/riscv: Support setting external interrupt by KVM

Quoted 7 lines · Yifei Jiang

When KVM is enabled, set the S-mode external interrupt through kvm_riscv_set_irq function.

Looks good to me.

Regards,
Anup

Quoted 79 lines
---
 target/riscv/cpu.c       |  6 +++++-
 target/riscv/kvm-stub.c  |  5 +++++
 target/riscv/kvm.c       | 17 +++++++++++++++++
 target/riscv/kvm_riscv.h |  1 +
 4 files changed, 28 insertions(+), 1 deletion(-)
target/riscv/cpu.c +5 -1
index 1c944872a3..3fc3a9c45b 100644
@@ -603,7 +603,11 @@ static void riscv_cpu_set_irq(void *opaque, int irq, int level)
     case IRQ_S_EXT:
     case IRQ_VS_EXT:
     case IRQ_M_EXT:
-        riscv_cpu_update_mip(cpu, 1 << irq, BOOL_TO_MASK(level));
+        if (kvm_enabled()) {
+            kvm_riscv_set_irq(cpu, irq, level);
+        } else {
+            riscv_cpu_update_mip(cpu, 1 << irq, BOOL_TO_MASK(level));
+        }
         break;
     default:
         g_assert_not_reached();
target/riscv/kvm-stub.c +5 -0
index 39b96fe3f4..4e8fc31a21 100644
@@ -23,3 +23,8 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
 {
     abort();
 }
+
+void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level)
+{
+    abort();
+}
target/riscv/kvm.c +17 -0
index db6d8a5b6e..0027f11f45 100644
@@ -383,6 +383,23 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
     env->satp = 0;
 }
 
+void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level)
+{
+    int ret;
+    unsigned virq = level ? KVM_INTERRUPT_SET : KVM_INTERRUPT_UNSET;
+
+    if (irq != IRQ_S_EXT) {
+        perror("kvm riscv set irq != IRQ_S_EXT\n");
+        abort();
+    }
+
+    ret = kvm_vcpu_ioctl(CPU(cpu), KVM_INTERRUPT, &virq);
+    if (ret < 0) {
+        perror("Set irq failed");
+        abort();
+    }
+}
+
 bool kvm_arch_cpu_check_are_resettable(void)
 {
     return true;
target/riscv/kvm_riscv.h +1 -1
index f38c82bf59..ed281bdce0 100644
@@ -20,5 +20,6 @@
 #define QEMU_KVM_RISCV_H
 
 void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
+void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
 
 #endif
--

2.19.1

#12 lore raw

PATCH v3 8/12 target/riscv: Handle KVM_EXIT_RISCV_SBI exit

In reply to Yifei Jiang

Use char-fe to handle console sbi call, which implement early console io while apply 'earlycon=sbi' into kernel parameters.

---
 target/riscv/kvm.c                 | 43 +++++++++++++++++-
 target/riscv/sbi_ecall_interface.h | 72 ++++++++++++++++++++++++++++++
 2 files changed, 114 insertions(+), 1 deletion(-)
 create mode 100644 target/riscv/sbi_ecall_interface.h
target/riscv/kvm.c +42 -1
index 0027f11f45..4d08669c81 100644
@@ -38,6 +38,9 @@
 #include "qemu/log.h"
 #include "hw/loader.h"
 #include "kvm_riscv.h"
+#include "sbi_ecall_interface.h"
+#include "chardev/char-fe.h"
+#include "semihosting/console.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx)
 {
@@ -365,9 +368,47 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cs)
     return true;
 }
 
+static int kvm_riscv_handle_sbi(CPUState *cs, struct kvm_run *run)
+{
+    int ret = 0;
+    unsigned char ch;
+    switch (run->riscv_sbi.extension_id) {
+    case SBI_EXT_0_1_CONSOLE_PUTCHAR:
+        ch = run->riscv_sbi.args[0];
+        qemu_semihosting_log_out((const char *)&ch, sizeof(ch));
+        break;
+    case SBI_EXT_0_1_CONSOLE_GETCHAR:
+        ret = qemu_chr_fe_read_all(serial_hd(0)->be, &ch, sizeof(ch));
+        if (ret == sizeof(ch)) {
+            run->riscv_sbi.args[0] = ch;
+        } else {
+            run->riscv_sbi.args[0] = -1;
+        }
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s: un-handled SBI EXIT, specific reasons is %lu\n",
+                      __func__, run->riscv_sbi.extension_id);
+        ret = -1;
+        break;
+    }
+    return ret;
+}
+
 int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
 {
-    return 0;
+    int ret = 0;
+    switch (run->exit_reason) {
+    case KVM_EXIT_RISCV_SBI:
+        ret = kvm_riscv_handle_sbi(cs, run);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s: un-handled exit reason %d\n",
+                      __func__, run->exit_reason);
+        ret = -1;
+        break;
+    }
+    return ret;
 }
 
 void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
target/riscv/sbi_ecall_interface.h new file +72 -0

72 lines not on this page · raw message

Signature · 1 line
2.19.1
#13 lore raw

Re: [PATCH v3 08/12] target/riscv: Handle KVM_EXIT_RISCV_SBI exit

Quoted 29 lines · Yifei Jiang via

Use char-fe to handle console sbi call, which implement early console io while apply 'earlycon=sbi' into kernel parameters.

---
 target/riscv/kvm.c                 | 43 +++++++++++++++++-
 target/riscv/sbi_ecall_interface.h | 72 ++++++++++++++++++++++++++++++
 2 files changed, 114 insertions(+), 1 deletion(-)
 create mode 100644 target/riscv/sbi_ecall_interface.h
target/riscv/kvm.c +3 -0
index 0027f11f45..4d08669c81 100644
@@ -38,6 +38,9 @@
 #include "qemu/log.h"
 #include "hw/loader.h"
 #include "kvm_riscv.h"
+#include "sbi_ecall_interface.h"
+#include "chardev/char-fe.h"
+#include "semihosting/console.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx)
 {
@@ -365,9 +368,47 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cs)
     return true;

target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs) · line 378

Quoted 10 lines
target/riscv/kvm.c +8 -0
 }
 
+static int kvm_riscv_handle_sbi(CPUState *cs, struct kvm_run *run)
+{
+    int ret = 0;
+    unsigned char ch;
+    switch (run->riscv_sbi.extension_id) {
+    case SBI_EXT_0_1_CONSOLE_PUTCHAR:
+        ch = run->riscv_sbi.args[0];
+        qemu_semihosting_log_out((const char *)&ch, sizeof(ch));

Hmmm... We print to the semihosting

target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs) · line 381

Quoted 3 lines
target/riscv/kvm.c +3 -0
+        break;
+    case SBI_EXT_0_1_CONSOLE_GETCHAR:
+        ret = qemu_chr_fe_read_all(serial_hd(0)->be, &ch, sizeof(ch));

but then read from the first serial device.

That seems a little strange. Would it be better to just print to serial as well?

Alistair

Quoted 116 lines
target/riscv/kvm.c +28 -1
+        if (ret == sizeof(ch)) {
+            run->riscv_sbi.args[0] = ch;
+        } else {
+            run->riscv_sbi.args[0] = -1;
+        }
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s: un-handled SBI EXIT, specific reasons is %lu\n",
+                      __func__, run->riscv_sbi.extension_id);
+        ret = -1;
+        break;
+    }
+    return ret;
+}
+
 int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
 {
-    return 0;
+    int ret = 0;
+    switch (run->exit_reason) {
+    case KVM_EXIT_RISCV_SBI:
+        ret = kvm_riscv_handle_sbi(cs, run);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s: un-handled exit reason %d\n",
+                      __func__, run->exit_reason);
+        ret = -1;
+        break;
+    }
+    return ret;
 }
 
 void kvm_riscv_reset_vcpu(RISCVCPU *cpu)
target/riscv/sbi_ecall_interface.h new file +72 -1
new file mode 100644
index 0000000000..fb1a3fa8f2
@@ -0,0 +1,72 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2019 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ *   Anup Patel <[email protected]>
+ */
+
+#ifndef __SBI_ECALL_INTERFACE_H__
+#define __SBI_ECALL_INTERFACE_H__
+
+/* clang-format off */
+
+/* SBI Extension IDs */
+#define SBI_EXT_0_1_SET_TIMER           0x0
+#define SBI_EXT_0_1_CONSOLE_PUTCHAR     0x1
+#define SBI_EXT_0_1_CONSOLE_GETCHAR     0x2
+#define SBI_EXT_0_1_CLEAR_IPI           0x3
+#define SBI_EXT_0_1_SEND_IPI            0x4
+#define SBI_EXT_0_1_REMOTE_FENCE_I      0x5
+#define SBI_EXT_0_1_REMOTE_SFENCE_VMA   0x6
+#define SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID 0x7
+#define SBI_EXT_0_1_SHUTDOWN            0x8
+#define SBI_EXT_BASE                    0x10
+#define SBI_EXT_TIME                    0x54494D45
+#define SBI_EXT_IPI                     0x735049
+#define SBI_EXT_RFENCE                  0x52464E43
+#define SBI_EXT_HSM                     0x48534D
+
+/* SBI function IDs for BASE extension*/
+#define SBI_EXT_BASE_GET_SPEC_VERSION   0x0
+#define SBI_EXT_BASE_GET_IMP_ID         0x1
+#define SBI_EXT_BASE_GET_IMP_VERSION    0x2
+#define SBI_EXT_BASE_PROBE_EXT          0x3
+#define SBI_EXT_BASE_GET_MVENDORID      0x4
+#define SBI_EXT_BASE_GET_MARCHID        0x5
+#define SBI_EXT_BASE_GET_MIMPID         0x6
+
+/* SBI function IDs for TIME extension*/
+#define SBI_EXT_TIME_SET_TIMER          0x0
+
+/* SBI function IDs for IPI extension*/
+#define SBI_EXT_IPI_SEND_IPI            0x0
+
+/* SBI function IDs for RFENCE extension*/
+#define SBI_EXT_RFENCE_REMOTE_FENCE_I       0x0
+#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA    0x1
+#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID  0x2
+#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA   0x3
+#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID 0x4
+#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA   0x5
+#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID 0x6
+
+/* SBI function IDs for HSM extension */
+#define SBI_EXT_HSM_HART_START          0x0
+#define SBI_EXT_HSM_HART_STOP           0x1
+#define SBI_EXT_HSM_HART_GET_STATUS     0x2
+
+#define SBI_HSM_HART_STATUS_STARTED     0x0
+#define SBI_HSM_HART_STATUS_STOPPED     0x1
+#define SBI_HSM_HART_STATUS_START_PENDING   0x2
+#define SBI_HSM_HART_STATUS_STOP_PENDING    0x3
+
+#define SBI_SPEC_VERSION_MAJOR_OFFSET   24
+#define SBI_SPEC_VERSION_MAJOR_MASK     0x7f
+#define SBI_SPEC_VERSION_MINOR_MASK     0xffffff
+#define SBI_EXT_VENDOR_START            0x09000000
+#define SBI_EXT_VENDOR_END              0x09FFFFFF
+/* clang-format on */
+
+#endif
--

2.19.1

author
#14 lore raw

RE: [PATCH v3 08/12] target/riscv: Handle KVM_EXIT_RISCV_SBI exit

Quoted 65 lines

-----Original Message-----
From: Alistair Francis [mailto:alistair23 at gmail.com]
Sent: Thursday, January 6, 2022 6:04 AM
To: Jiangyifei <[email protected]>
Cc: qemu-devel at nongnu.org Developers <[email protected]>; open list:RISC-V <[email protected]>; kvm-riscv at lists.infradead.org; open list:Overall <[email protected]>; libvir-list at redhat.com; Anup Patel <[email protected]>; Palmer Dabbelt <[email protected]>; Alistair Francis <[email protected]>; Bin Meng <[email protected]>; Fanliang (EulerOS) <[email protected]>; Wubin (H) <[email protected]>; Wanghaibin (D) <[email protected]>; wanbo (G) <[email protected]>; limingwang (A) <[email protected]>
Subject: Re: [PATCH v3 08/12] target/riscv: Handle KVM_EXIT_RISCV_SBI exit

Quoted 32 lines · Yifei Jiang via

Use char-fe to handle console sbi call, which implement early console io while apply 'earlycon=sbi' into kernel parameters.

---
 target/riscv/kvm.c                 | 43 +++++++++++++++++-
 target/riscv/sbi_ecall_interface.h | 72
++++++++++++++++++++++++++++++
 2 files changed, 114 insertions(+), 1 deletion(-)  create mode 100644

target/riscv/sbi_ecall_interface.h

target/riscv/kvm.c index renamed from target/riscv/kvm.c +0 -0

0027f11f45..4d08669c81 100644

target/riscv/kvm.c +3 -0
@@ -38,6 +38,9 @@
 #include "qemu/log.h"
 #include "hw/loader.h"
 #include "kvm_riscv.h"
+#include "sbi_ecall_interface.h"
+#include "chardev/char-fe.h"
+#include "semihosting/console.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type,

uint64_t idx) { @@ -365,9 +368,47 @@ bool
kvm_arch_stop_on_emulation_error(CPUState *cs)

     return true;

}

target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs) · line 378

Quoted 7 lines
target/riscv/kvm.c +7 -0
+static int kvm_riscv_handle_sbi(CPUState *cs, struct kvm_run *run) {
+    int ret = 0;
+    unsigned char ch;
+    switch (run->riscv_sbi.extension_id) {
+    case SBI_EXT_0_1_CONSOLE_PUTCHAR:
+        ch = run->riscv_sbi.args[0];
+        qemu_semihosting_log_out((const char *)&ch, sizeof(ch));

Hmmm... We print to the semihosting

target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs)

Quoted 4 lines
target/riscv/kvm.c +4 -0
+        break;
+    case SBI_EXT_0_1_CONSOLE_GETCHAR:
+        ret = qemu_chr_fe_read_all(serial_hd(0)->be, &ch,
+ sizeof(ch));
Quoted 6 lines

but then read from the first serial device.

That seems a little strange. Would it be better to just print to serial as well?

Alistair

Hm...I also think so. It will be modified in the next series.

Yifei

Quoted 117 lines

target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs)

Quoted 9 lines
target/riscv/kvm.c +9 -0
+        if (ret == sizeof(ch)) {
+            run->riscv_sbi.args[0] = ch;
+        } else {
+            run->riscv_sbi.args[0] = -1;
+        }
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s: un-handled SBI EXIT, specific reasons

is %lu\n",

Quoted 107 lines
target/riscv/kvm.c +19 -1
+                      __func__, run->riscv_sbi.extension_id);
+        ret = -1;
+        break;
+    }
+    return ret;
+}
+
 int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)  {
-    return 0;
+    int ret = 0;
+    switch (run->exit_reason) {
+    case KVM_EXIT_RISCV_SBI:
+        ret = kvm_riscv_handle_sbi(cs, run);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s: un-handled exit reason %d\n",
+                      __func__, run->exit_reason);
+        ret = -1;
+        break;
+    }
+    return ret;
 }
 
 void kvm_riscv_reset_vcpu(RISCVCPU *cpu) diff --git

a/target/riscv/sbi_ecall_interface.h
b/target/riscv/sbi_ecall_interface.h
new file mode 100644
index 0000000000..fb1a3fa8f2

target/riscv/sbi_ecall_interface.h new file +72 -1
@@ -0,0 +1,72 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2019 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ *   Anup Patel <[email protected]>
+ */
+
+#ifndef __SBI_ECALL_INTERFACE_H__
+#define __SBI_ECALL_INTERFACE_H__
+
+/* clang-format off */
+
+/* SBI Extension IDs */
+#define SBI_EXT_0_1_SET_TIMER           0x0
+#define SBI_EXT_0_1_CONSOLE_PUTCHAR     0x1
+#define SBI_EXT_0_1_CONSOLE_GETCHAR     0x2
+#define SBI_EXT_0_1_CLEAR_IPI           0x3
+#define SBI_EXT_0_1_SEND_IPI            0x4
+#define SBI_EXT_0_1_REMOTE_FENCE_I      0x5
+#define SBI_EXT_0_1_REMOTE_SFENCE_VMA   0x6
+#define SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID 0x7
+#define SBI_EXT_0_1_SHUTDOWN            0x8
+#define SBI_EXT_BASE                    0x10
+#define SBI_EXT_TIME                    0x54494D45
+#define SBI_EXT_IPI                     0x735049
+#define SBI_EXT_RFENCE                  0x52464E43
+#define SBI_EXT_HSM                     0x48534D
+
+/* SBI function IDs for BASE extension*/
+#define SBI_EXT_BASE_GET_SPEC_VERSION   0x0
+#define SBI_EXT_BASE_GET_IMP_ID         0x1
+#define SBI_EXT_BASE_GET_IMP_VERSION    0x2
+#define SBI_EXT_BASE_PROBE_EXT          0x3
+#define SBI_EXT_BASE_GET_MVENDORID      0x4
+#define SBI_EXT_BASE_GET_MARCHID        0x5
+#define SBI_EXT_BASE_GET_MIMPID         0x6
+
+/* SBI function IDs for TIME extension*/
+#define SBI_EXT_TIME_SET_TIMER          0x0
+
+/* SBI function IDs for IPI extension*/
+#define SBI_EXT_IPI_SEND_IPI            0x0
+
+/* SBI function IDs for RFENCE extension*/
+#define SBI_EXT_RFENCE_REMOTE_FENCE_I       0x0
+#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA    0x1
+#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID  0x2
+#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA   0x3
+#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID 0x4
+#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA   0x5
+#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID 0x6
+
+/* SBI function IDs for HSM extension */
+#define SBI_EXT_HSM_HART_START          0x0
+#define SBI_EXT_HSM_HART_STOP           0x1
+#define SBI_EXT_HSM_HART_GET_STATUS     0x2
+
+#define SBI_HSM_HART_STATUS_STARTED     0x0
+#define SBI_HSM_HART_STATUS_STOPPED     0x1
+#define SBI_HSM_HART_STATUS_START_PENDING   0x2
+#define SBI_HSM_HART_STATUS_STOP_PENDING    0x3
+
+#define SBI_SPEC_VERSION_MAJOR_OFFSET   24
+#define SBI_SPEC_VERSION_MAJOR_MASK     0x7f
+#define SBI_SPEC_VERSION_MINOR_MASK     0xffffff
+#define SBI_EXT_VENDOR_START            0x09000000
+#define SBI_EXT_VENDOR_END              0x09FFFFFF
+/* clang-format on */
+
+#endif
--

2.19.1

#15 lore raw

PATCH v3 9/12 target/riscv: Add host cpu type

In reply to Yifei Jiang

'host' type cpu is set isa to RV32 or RV64 simply, more isa info will obtain from KVM in kvm_arch_init_vcpu()

---
 target/riscv/cpu.c | 15 +++++++++++++++
 target/riscv/cpu.h |  1 +
 2 files changed, 16 insertions(+)
target/riscv/cpu.c +15 -0
index 3fc3a9c45b..c6d979594b 100644
@@ -213,6 +213,18 @@ static void rv32_imafcu_nommu_cpu_init(Object *obj)
 }
 #endif
 
+#if defined(CONFIG_KVM)
+static void riscv_host_cpu_init(Object *obj)
+{
+    CPURISCVState *env = &RISCV_CPU(obj)->env;
+#if defined(TARGET_RISCV32)
+    set_misa(env, MXL_RV32, 0);
+#elif defined(TARGET_RISCV64)
+    set_misa(env, MXL_RV64, 0);
+#endif
+}
+#endif
+
 static ObjectClass *riscv_cpu_class_by_name(const char *cpu_model)
 {
     ObjectClass *oc;
@@ -814,6 +826,9 @@ static const TypeInfo riscv_cpu_type_infos[] = {
         .class_init = riscv_cpu_class_init,
     },
     DEFINE_CPU(TYPE_RISCV_CPU_ANY,              riscv_any_cpu_init),
+#if defined(CONFIG_KVM)
+    DEFINE_CPU(TYPE_RISCV_CPU_HOST,             riscv_host_cpu_init),
+#endif
 #if defined(TARGET_RISCV32)
     DEFINE_CPU(TYPE_RISCV_CPU_BASE32,           rv32_base_cpu_init),
     DEFINE_CPU(TYPE_RISCV_CPU_IBEX,             rv32_ibex_cpu_init),
target/riscv/cpu.h +1 -0
index 2807eb1bcb..e7dba35acb 100644
@@ -45,6 +45,7 @@
 #define TYPE_RISCV_CPU_SIFIVE_E51       RISCV_CPU_TYPE_NAME("sifive-e51")
 #define TYPE_RISCV_CPU_SIFIVE_U34       RISCV_CPU_TYPE_NAME("sifive-u34")
 #define TYPE_RISCV_CPU_SIFIVE_U54       RISCV_CPU_TYPE_NAME("sifive-u54")
+#define TYPE_RISCV_CPU_HOST             RISCV_CPU_TYPE_NAME("host")
 
 #if defined(TARGET_RISCV32)
 # define TYPE_RISCV_CPU_BASE            TYPE_RISCV_CPU_BASE32
Signature · 1 line
2.19.1
#16 lore raw

PATCH v3 10/12 target/riscv: Add kvm_riscv_get/put_regs_timer

In reply to Yifei Jiang

Add kvm_riscv_get/put_regs_timer to synchronize virtual time context from KVM.

To set register of RISCV_TIMER_REG(state) will occur a error from KVM on kvm_timer_state == 0. It's better to adapt in KVM, but it doesn't matter that adaping in QEMU.

---
 target/riscv/cpu.h |  7 +++++
 target/riscv/kvm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 79 insertions(+)
target/riscv/cpu.h +7 -0
index e7dba35acb..c892a2c8b7 100644
@@ -259,6 +259,13 @@ struct CPURISCVState {
 
     hwaddr kernel_addr;
     hwaddr fdt_addr;
+
+    /* kvm timer */
+    bool kvm_timer_dirty;
+    uint64_t kvm_timer_time;
+    uint64_t kvm_timer_compare;
+    uint64_t kvm_timer_state;
+    uint64_t kvm_timer_frequency;
 };
 
 OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,
target/riscv/kvm.c +72 -0

96 lines not on this page · raw message

Signature · 1 line
2.19.1
maintainer Reviewed-by on v3 10/12
#17 lore raw

Re: [PATCH v3 10/12] target/riscv: Add kvm_riscv_get/put_regs_timer

Quoted 10 lines · Yifei Jiang

Add kvm_riscv_get/put_regs_timer to synchronize virtual time context from KVM.

To set register of RISCV_TIMER_REG(state) will occur a error from KVM on kvm_timer_state == 0. It's better to adapt in KVM, but it doesn't matter that adaping in QEMU.

Looks good to me.

Regards,
Anup

Quoted 130 lines
---
 target/riscv/cpu.h |  7 +++++
 target/riscv/kvm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 79 insertions(+)
target/riscv/cpu.h +7 -0
index e7dba35acb..c892a2c8b7 100644
@@ -259,6 +259,13 @@ struct CPURISCVState {
 
     hwaddr kernel_addr;
     hwaddr fdt_addr;
+
+    /* kvm timer */
+    bool kvm_timer_dirty;
+    uint64_t kvm_timer_time;
+    uint64_t kvm_timer_compare;
+    uint64_t kvm_timer_state;
+    uint64_t kvm_timer_frequency;
 };
 
 OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,
target/riscv/kvm.c +72 -1
index 4d08669c81..3c20ec5ad3 100644
@@ -41,6 +41,7 @@
 #include "sbi_ecall_interface.h"
 #include "chardev/char-fe.h"
 #include "semihosting/console.h"
+#include "migration/migration.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx)
 {
@@ -65,6 +66,9 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx
 #define RISCV_CSR_REG(env, name)  kvm_riscv_reg_id(env, KVM_REG_RISCV_CSR, \
                  KVM_REG_RISCV_CSR_REG(name))
 
+#define RISCV_TIMER_REG(env, name)  kvm_riscv_reg_id(env, KVM_REG_RISCV_TIMER, \
+                 KVM_REG_RISCV_TIMER_REG(name))
+
 #define RISCV_FP_F_REG(env, idx)  kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_F, idx)
 
 #define RISCV_FP_D_REG(env, idx)  kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_D, idx)
@@ -85,6 +89,22 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx
         } \
     } while(0)
 
+#define KVM_RISCV_GET_TIMER(cs, env, name, reg) \
+    do { \
+        int ret = kvm_get_one_reg(cs, RISCV_TIMER_REG(env, name), &reg); \
+        if (ret) { \
+            abort(); \
+        } \
+    } while(0)
+
+#define KVM_RISCV_SET_TIMER(cs, env, name, reg) \
+    do { \
+        int ret = kvm_set_one_reg(cs, RISCV_TIMER_REG(env, time), &reg); \
+        if (ret) { \
+            abort(); \
+        } \
+    } while (0)
+
 static int kvm_riscv_get_regs_core(CPUState *cs)
 {
     int ret = 0;
@@ -236,6 +256,58 @@ static int kvm_riscv_put_regs_fp(CPUState *cs)
     return ret;
 }
 
+static void kvm_riscv_get_regs_timer(CPUState *cs)
+{
+    CPURISCVState *env = &RISCV_CPU(cs)->env;
+
+    if (env->kvm_timer_dirty) {
+        return;
+    }
+
+    KVM_RISCV_GET_TIMER(cs, env, time, env->kvm_timer_time);
+    KVM_RISCV_GET_TIMER(cs, env, compare, env->kvm_timer_compare);
+    KVM_RISCV_GET_TIMER(cs, env, state, env->kvm_timer_state);
+    KVM_RISCV_GET_TIMER(cs, env, frequency, env->kvm_timer_frequency);
+
+    env->kvm_timer_dirty = true;
+}
+
+static void kvm_riscv_put_regs_timer(CPUState *cs)
+{
+    uint64_t reg;
+    CPURISCVState *env = &RISCV_CPU(cs)->env;
+
+    if (!env->kvm_timer_dirty) {
+        return;
+    }
+
+    KVM_RISCV_SET_TIMER(cs, env, time, env->kvm_timer_time);
+    KVM_RISCV_SET_TIMER(cs, env, compare, env->kvm_timer_compare);
+
+    /*
+     * To set register of RISCV_TIMER_REG(state) will occur a error from KVM
+     * on env->kvm_timer_state == 0, It's better to adapt in KVM, but it
+     * doesn't matter that adaping in QEMU now.
+     * TODO If KVM changes, adapt here.
+     */
+    if (env->kvm_timer_state) {
+        KVM_RISCV_SET_TIMER(cs, env, state, env->kvm_timer_state);
+    }
+
+    /*
+     * For now, migration will not work between Hosts with different timer
+     * frequency. Therefore, we should check whether they are the same here
+     * during the migration.
+     */
+    if (migration_is_running(migrate_get_current()->state)) {
+        KVM_RISCV_GET_TIMER(cs, env, frequency, reg);
+        if (reg != env->kvm_timer_frequency) {
+            error_report("Dst Hosts timer frequency != Src Hosts");
+        }
+    }
+
+    env->kvm_timer_dirty = false;
+}
 
 const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
     KVM_CAP_LAST_INFO
--

2.19.1

#18 lore raw

PATCH v3 11/12 target/riscv: Implement virtual time adjusting with vm state changing

In reply to Yifei Jiang

We hope that virtual time adjusts with vm state changing. When a vm is stopped, guest virtual time should stop counting and kvm_timer should be stopped. When the vm is resumed, guest virtual time should continue to count and kvm_timer should be restored.

---
 target/riscv/kvm.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
target/riscv/kvm.c +14 -0
index 3c20ec5ad3..6c0306bd2b 100644
@@ -42,6 +42,7 @@
 #include "chardev/char-fe.h"
 #include "semihosting/console.h"
 #include "migration/migration.h"
+#include "sysemu/runstate.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx)
 {
@@ -378,6 +379,17 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)
     return cpu->cpu_index;
 }
 
+static void kvm_riscv_vm_state_change(void *opaque, bool running, RunState state)
+{
+    CPUState *cs = opaque;
+
+    if (running) {
+        kvm_riscv_put_regs_timer(cs);
+    } else {
+        kvm_riscv_get_regs_timer(cs);
+    }
+}
+
 void kvm_arch_init_irq_routing(KVMState *s)
 {
 }
@@ -390,6 +402,8 @@ int kvm_arch_init_vcpu(CPUState *cs)
     CPURISCVState *env = &cpu->env;
     uint64_t id;
 
+    qemu_add_vm_change_state_handler(kvm_riscv_vm_state_change, cs);
+
     id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CONFIG, KVM_REG_RISCV_CONFIG_REG(isa));
     ret = kvm_get_one_reg(cs, id, &isa);
     if (ret) {
Signature · 1 line
2.19.1
Reviewed-by on v3 11/12
#19 lore raw

Re: [PATCH v3 11/12] target/riscv: Implement virtual time adjusting with vm state changing

Quoted 9 lines · Yifei Jiang via

We hope that virtual time adjusts with vm state changing. When a vm is stopped, guest virtual time should stop counting and kvm_timer should be stopped. When the vm is resumed, guest virtual time should continue to count and kvm_timer should be restored.

Alistair

Quoted 47 lines
---
 target/riscv/kvm.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
target/riscv/kvm.c +14 -1
index 3c20ec5ad3..6c0306bd2b 100644
@@ -42,6 +42,7 @@
 #include "chardev/char-fe.h"
 #include "semihosting/console.h"
 #include "migration/migration.h"
+#include "sysemu/runstate.h"
 
 static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx)
 {
@@ -378,6 +379,17 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)
     return cpu->cpu_index;
 }
 
+static void kvm_riscv_vm_state_change(void *opaque, bool running, RunState state)
+{
+    CPUState *cs = opaque;
+
+    if (running) {
+        kvm_riscv_put_regs_timer(cs);
+    } else {
+        kvm_riscv_get_regs_timer(cs);
+    }
+}
+
 void kvm_arch_init_irq_routing(KVMState *s)
 {
 }
@@ -390,6 +402,8 @@ int kvm_arch_init_vcpu(CPUState *cs)
     CPURISCVState *env = &cpu->env;
     uint64_t id;
 
+    qemu_add_vm_change_state_handler(kvm_riscv_vm_state_change, cs);
+
     id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CONFIG, KVM_REG_RISCV_CONFIG_REG(isa));
     ret = kvm_get_one_reg(cs, id, &isa);
     if (ret) {
--

2.19.1

#20 lore raw

PATCH v3 12/12 target/riscv: Support virtual time context synchronization

In reply to Yifei Jiang

Add virtual time context description to vmstate_kvmtimer. After cpu being loaded, virtual time context is updated to KVM.

---
 target/riscv/machine.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
target/riscv/machine.c +30 -0
index ad8248ebfd..95eb82792a 100644
@@ -164,6 +164,35 @@ static const VMStateDescription vmstate_pointermasking = {
     }
 };
 
+static bool kvmtimer_needed(void *opaque)
+{
+    return kvm_enabled();
+}
+
+static int cpu_post_load(void *opaque, int version_id)
+{
+    RISCVCPU *cpu = opaque;
+    CPURISCVState *env = &cpu->env;
+
+    env->kvm_timer_dirty = true;
+    return 0;
+}
+
+static const VMStateDescription vmstate_kvmtimer = {
+    .name = "cpu/kvmtimer",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = kvmtimer_needed,
+    .post_load = cpu_post_load,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(env.kvm_timer_time, RISCVCPU),
+        VMSTATE_UINT64(env.kvm_timer_compare, RISCVCPU),
+        VMSTATE_UINT64(env.kvm_timer_state, RISCVCPU),
+
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 const VMStateDescription vmstate_riscv_cpu = {
     .name = "cpu",
     .version_id = 3,
@@ -218,6 +247,7 @@ const VMStateDescription vmstate_riscv_cpu = {
         &vmstate_hyper,
         &vmstate_vector,
         &vmstate_pointermasking,
+        &vmstate_kvmtimer,
         NULL
     }
 };
Signature · 1 line
2.19.1
Reviewed-by on v3 12/12
#21 lore raw

Re: [PATCH v3 12/12] target/riscv: Support virtual time context synchronization

Quoted 7 lines · Yifei Jiang via

Add virtual time context description to vmstate_kvmtimer. After cpu being loaded, virtual time context is updated to KVM.

Alistair

Quoted 56 lines
---
 target/riscv/machine.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
target/riscv/machine.c +30 -1
index ad8248ebfd..95eb82792a 100644
@@ -164,6 +164,35 @@ static const VMStateDescription vmstate_pointermasking = {
     }
 };
 
+static bool kvmtimer_needed(void *opaque)
+{
+    return kvm_enabled();
+}
+
+static int cpu_post_load(void *opaque, int version_id)
+{
+    RISCVCPU *cpu = opaque;
+    CPURISCVState *env = &cpu->env;
+
+    env->kvm_timer_dirty = true;
+    return 0;
+}
+
+static const VMStateDescription vmstate_kvmtimer = {
+    .name = "cpu/kvmtimer",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = kvmtimer_needed,
+    .post_load = cpu_post_load,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(env.kvm_timer_time, RISCVCPU),
+        VMSTATE_UINT64(env.kvm_timer_compare, RISCVCPU),
+        VMSTATE_UINT64(env.kvm_timer_state, RISCVCPU),
+
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 const VMStateDescription vmstate_riscv_cpu = {
     .name = "cpu",
     .version_id = 3,
@@ -218,6 +247,7 @@ const VMStateDescription vmstate_riscv_cpu = {
         &vmstate_hyper,
         &vmstate_vector,
         &vmstate_pointermasking,
+        &vmstate_kvmtimer,
         NULL
     }
 };
--

2.19.1