Add riscv kvm accel support
Subsystems (5)
v3 cover Add riscv kvm accel support
This series adds both riscv32 and riscv64 kvm support, and implements migration based on riscv.
Because of RISC-V KVM has been merged into the Linux master, so this series are changed from RFC to patch.
Several steps to use this:
1. Build emulation
$ ./configure --target-list=riscv64-softmmu $ make -j$(nproc)
2. Build kernel
3. Build QEMU VM
Cross built in riscv toolchain.
$ PKG_CONFIG_LIBDIR=<toolchain pkgconfig path> $ export PKG_CONFIG_SYSROOT_DIR=<toolchain sysroot path> $ ./configure --target-list=riscv64-softmmu --enable-kvm \ --cross-prefix=riscv64-linux-gnu- --disable-libiscsi --disable-glusterfs \ --disable-libusb --disable-usb-redir --audio-drv-list= --disable-opengl \ --disable-libxml2 $ make -j$(nproc)
4. Start emulation
$ ./qemu-system-riscv64 -M virt -m 4096M -cpu rv64,x-h=true -nographic \
-name guest=riscv-hyp,debug-threads=on \
-smp 4 \
-bios ./fw_jump.bin \
-kernel ./Image \
-drive file=./hyp.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0 \
-append "root=/dev/vda rw console=ttyS0 earlycon=sbi"5. Start kvm-acceled QEMU VM in emulation
$ ./qemu-system-riscv64 -M virt,accel=kvm -m 1024M -cpu host -nographic \
-name guest=riscv-guset \
-smp 2 \
-bios none \
-kernel ./Image \
-drive file=./guest.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0 \
-append "root=/dev/vda rw console=ttyS0 earlycon=sbi"Changes since patch v2
- Create a macro for get and put timer csr.
- Remove M-mode PLIC contexts when kvm is enabled.
- Add get timer frequency.
- Move cpu_host_load to vmstate_kvmtimer.
Changes since patch v1
- Rebase on recent commit a216e7cf119c91ffdf5931834a1a030ebea40d70
- Sync-up headers with Linux-5.16-rc4.
- Fixbug in kvm_arch_init_vcpu.
- Create a macro for get and put regs csr.
- Start kernel directly when kvm_enabled.
- Use riscv_cpu_set_irq to inject KVM interrupts.
- Use the Semihosting Console API for RISC-V kvm handle sbi.
- Update vmstate_riscv_cpu version id.
Placing kvm_timer into a subsection.
Changes since RFC v6
- Rebase on recent commit 8627edfb3f1fca24a96a0954148885c3241c10f8
- Sync-up headers with Linux-5.16-rc1
Changes since RFC v5
- Rebase on QEMU v6.1.0-rc1 and kvm-riscv linux v19.
- Move kvm interrupt setting to riscv_cpu_update_mip().
- Replace __u64 with uint64_t.
Changes since RFC v4
- Rebase on QEMU v6.0.0-rc2 and kvm-riscv linux v17.
- Remove time scaling support as software solution is incomplete. Because it will cause unacceptable performance degradation. and We will post a better solution.
- Revise according to Alistair's review comments.
- Remove compile time XLEN checks in kvm_riscv_reg_id
- Surround TYPE_RISCV_CPU_HOST definition by CONFIG_KVM and share it between RV32 and RV64.
- Add kvm-stub.c for reduce unnecessary compilation checks.
- Add riscv_setup_direct_kernel() to direct boot kernel for KVM.
Changes since RFC v3
- Rebase on QEMU v5.2.0-rc2 and kvm-riscv linux v15.
- Add time scaling support(New patches 13, 14 and 15).
- Fix the bug that guest vm can't reboot.
Changes since RFC v2
- Fix checkpatch error at target/riscv/sbi_ecall_interface.h.
- Add riscv migration support.
Changes since RFC v1
- Add separate SBI ecall interface header.
- Add riscv32 kvm accel support.
Yifei Jiang (12):
update-linux-headers: Add asm-riscv/kvm.h
target/riscv: Add target/riscv/kvm.c to place the public kvm interface target/riscv: Implement function kvm_arch_init_vcpu target/riscv: Implement kvm_arch_get_registers target/riscv: Implement kvm_arch_put_registers target/riscv: Support start kernel directly by KVM target/riscv: Support setting external interrupt by KVM target/riscv: Handle KVM_EXIT_RISCV_SBI exit
target/riscv: Add host cpu type
target/riscv: Add kvm_riscv_get/put_regs_timer target/riscv: Implement virtual time adjusting with vm state changing target/riscv: Support virtual time context synchronization
hw/intc/sifive_plic.c | 8 +- hw/riscv/boot.c | 16 +- hw/riscv/virt.c | 87 +++-- include/hw/riscv/boot.h | 1 + linux-headers/asm-riscv/kvm.h | 128 +++++++ meson.build | 2 + target/riscv/cpu.c | 29 +- target/riscv/cpu.h | 11 + target/riscv/kvm-stub.c | 30 ++ target/riscv/kvm.c | 533 +++++++++++++++++++++++++++++ target/riscv/kvm_riscv.h | 25 ++ target/riscv/machine.c | 30 ++ target/riscv/meson.build | 1 + target/riscv/sbi_ecall_interface.h | 72 ++++ 14 files changed, 944 insertions(+), 29 deletions(-) create mode 100644 linux-headers/asm-riscv/kvm.h create mode 100644 target/riscv/kvm-stub.c create mode 100644 target/riscv/kvm.c create mode 100644 target/riscv/kvm_riscv.h create mode 100644 target/riscv/sbi_ecall_interface.h
Signature · 1 line
2.19.1
PATCH v3 1/12 update-linux-headers: Add asm-riscv/kvm.h
Add asm-riscv/kvm.h for RISC-V KVM, and update linux/kvm.h
--- linux-headers/asm-riscv/kvm.h | 128 ++++++++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) create mode 100644 linux-headers/asm-riscv/kvm.h
linux-headers/asm-riscv/kvm.h new file +128 -0
@@ -0,0 +1,128 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* +* Copyright (C) 2019 Western Digital Corporation or its affiliates. +* +* Authors: +* Anup Patel <[email protected]> +*/ + +#ifndef __LINUX_KVM_RISCV_H +#define __LINUX_KVM_RISCV_H + +#ifndef __ASSEMBLY__ + +#include <linux/types.h> +#include <asm/ptrace.h> + +#define __KVM_HAVE_READONLY_MEM + +#define KVM_COALESCED_MMIO_PAGE_OFFSET 1 + +#define KVM_INTERRUPT_SET -1U +#define KVM_INTERRUPT_UNSET -2U + +/* for KVM_GET_REGS and KVM_SET_REGS */ +struct kvm_regs { +}; + +/* for KVM_GET_FPU and KVM_SET_FPU */ +struct kvm_fpu { +}; + +/* KVM Debug exit structure */ +struct kvm_debug_exit_arch { +}; + +/* for KVM_SET_GUEST_DEBUG */ +struct kvm_guest_debug_arch { +}; + +/* definition of registers in kvm_run */ +struct kvm_sync_regs { +}; + +/* for KVM_GET_SREGS and KVM_SET_SREGS */ +struct kvm_sregs { +}; + +/* CONFIG registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_config { +unsigned long isa; +}; + +/* CORE registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_core { +struct user_regs_struct regs; +unsigned long mode; +}; + +/* Possible privilege modes for kvm_riscv_core */ +#define KVM_RISCV_MODE_S 1 +#define KVM_RISCV_MODE_U 0 + +/* CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_csr { +unsigned long sstatus; +unsigned long sie; +unsigned long stvec; +unsigned long sscratch; +unsigned long sepc; +unsigned long scause; +unsigned long stval; +unsigned long sip; +unsigned long satp; +unsigned long scounteren; +}; + +/* TIMER registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_timer { +__u64 frequency; +__u64 time; +__u64 compare; +__u64 state; +}; + +/* Possible states for kvm_riscv_timer */ +#define KVM_RISCV_TIMER_STATE_OFF 0 +#define KVM_RISCV_TIMER_STATE_ON 1 + +#define KVM_REG_SIZE (id) \ + (1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT)) + +/* If you need to interpret the index values, here is the key: */ +#define KVM_REG_RISCV_TYPE_MASK 0x00000000FF000000 +#define KVM_REG_RISCV_TYPE_SHIFT 24 + +/* Config registers are mapped as type 1 */ +#define KVM_REG_RISCV_CONFIG (0x01 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_CONFIG_REG (name) \ + (offsetof(struct kvm_riscv_config, name) / sizeof(unsigned long)) + +/* Core registers are mapped as type 2 */ +#define KVM_REG_RISCV_CORE (0x02 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_CORE_REG (name) \ + (offsetof(struct kvm_riscv_core, name) / sizeof(unsigned long)) + +/* Control and status registers are mapped as type 3 */ +#define KVM_REG_RISCV_CSR (0x03 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_CSR_REG (name) \ + (offsetof(struct kvm_riscv_csr, name) / sizeof(unsigned long)) + +/* Timer registers are mapped as type 4 */ +#define KVM_REG_RISCV_TIMER (0x04 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_TIMER_REG (name) \ + (offsetof(struct kvm_riscv_timer, name) / sizeof(__u64)) + +/* F extension registers are mapped as type 5 */ +#define KVM_REG_RISCV_FP_F (0x05 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_FP_F_REG (name) \ + (offsetof(struct __riscv_f_ext_state, name) / sizeof(__u32)) + +/* D extension registers are mapped as type 6 */ +#define KVM_REG_RISCV_FP_D (0x06 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_FP_D_REG (name) \ + (offsetof(struct __riscv_d_ext_state, name) / sizeof(__u64)) + +#endif + +#endif /* __LINUX_KVM_RISCV_H */
Signature · 1 line
2.19.1
PATCH v3 2/12 target/riscv: Add target/riscv/kvm.c to place the public kvm interface
In reply to Yifei JiangAdd target/riscv/kvm.c to place kvm_arch_* function needed by kvm/kvm-all.c. Meanwhile, add kvm support in meson.build file.
--- meson.build | 2 + target/riscv/kvm.c | 133 +++++++++++++++++++++++++++++++++++++++ target/riscv/meson.build | 1 + 3 files changed, 136 insertions(+) create mode 100644 target/riscv/kvm.c
meson.build +2 -0
@@ -85,6 +85,8 @@ elif cpu in ['ppc', 'ppc64'] kvm_targets = ['ppc-softmmu', 'ppc64-softmmu'] elif cpu in ['mips', 'mips64'] kvm_targets = ['mips-softmmu', 'mipsel-softmmu', 'mips64-softmmu', 'mips64el-softmmu'] +elif cpu in ['riscv'] + kvm_targets = ['riscv32-softmmu', 'riscv64-softmmu'] else kvm_targets = [] endif
target/riscv/kvm.c new file +133 -0
133 lines not on this page · raw message
target/riscv/meson.build +1 -0
@@ -19,6 +19,7 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', )) +riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c')) riscv_softmmu_ss = ss.source_set() riscv_softmmu_ss.add(files(
Signature · 1 line
2.19.1
PATCH v3 3/12 target/riscv: Implement function kvm_arch_init_vcpu
In reply to Yifei JiangGet isa info from kvm while kvm init.
--- target/riscv/kvm.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-)
target/riscv/kvm.c +31 -1
@@ -38,6 +38,23 @@#include "qemu/log.h" #include "hw/loader.h" +static uint64_t kvm_riscv_reg_id (CPURISCVState *env,uint64_t type,uint64_t idx) +{ +uint64_t id =KVM_REG_RISCV | type | idx; + +switch (riscv_cpu_mxl (env)) { +case MXL_RV32 : + id |=KVM_REG_SIZE_U32 ; +break ; +case MXL_RV64 : + id |=KVM_REG_SIZE_U64 ; +break ; +default : +g_assert_not_reached (); + } +return id; +} +const KVMCapabilityInfo kvm_arch_required_capabilities[] = {KVM_CAP_LAST_INFO };
@@ -79,7 +96,20 @@ void kvm_arch_init_irq_routing(KVMState *s)int kvm_arch_init_vcpu (CPUState *cs) { -return 0 ; +int ret =0 ; +target_ulong isa; +RISCVCPU *cpu =RISCV_CPU (cs); +CPURISCVState *env = &cpu->env; +uint64_t id; + + id =kvm_riscv_reg_id (env,KVM_REG_RISCV_CONFIG ,KVM_REG_RISCV_CONFIG_REG (isa)); + ret =kvm_get_one_reg (cs, id, &isa); +if (ret) { +return ret; + } + env->misa_ext = isa; + +return ret; }int kvm_arch_msi_data_to_gsi (uint32_t data)
Signature · 1 line
2.19.1
PATCH v3 4/12 target/riscv: Implement kvm_arch_get_registers
In reply to Yifei JiangGet GPR CSR and FP registers from kvm by KVM_GET_ONE_REG ioctl.
--- target/riscv/kvm.c | 112 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 111 insertions(+), 1 deletion(-)
target/riscv/kvm.c +111 -1
@@ -55,13 +55,123 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idxreturn id; } +#define RISCV_CORE_REG (env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_CORE, \ + KVM_REG_RISCV_CORE_REG(name)) + +#define RISCV_CSR_REG (env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_CSR, \ + KVM_REG_RISCV_CSR_REG(name)) + +#define RISCV_FP_F_REG (env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_F, idx) + +#define RISCV_FP_D_REG (env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_D, idx) + +#define KVM_RISCV_GET_CSR (cs, env, csr, reg) \ + do { \ + int ret = kvm_get_one_reg(cs, RISCV_CSR_REG(env, csr), ®); \ + if (ret) { \ + return ret; \ + } \ + } while(0) + +static int kvm_riscv_get_regs_core (CPUState *cs) +{ +int ret =0 ; +int i; +target_ulong reg; +CPURISCVState *env = &RISCV_CPU (cs)->env; + + ret =kvm_get_one_reg (cs,RISCV_CORE_REG (env, regs.pc), ®); +if (ret) { +return ret; + } + env->pc = reg; + +for (i =1 ; i <32 ; i++) { +uint64_t id =kvm_riscv_reg_id (env,KVM_REG_RISCV_CORE , i); + ret =kvm_get_one_reg (cs, id, ®); +if (ret) { +return ret; + } + env->gpr[i] = reg; + } + +return ret; +} + +static int kvm_riscv_get_regs_csr (CPUState *cs) +{ +int ret =0 ; +CPURISCVState *env = &RISCV_CPU (cs)->env; + +KVM_RISCV_GET_CSR (cs, env, sstatus, env->mstatus); +KVM_RISCV_GET_CSR (cs, env, sie, env->mie); +KVM_RISCV_GET_CSR (cs, env, stvec, env->stvec); +KVM_RISCV_GET_CSR (cs, env, sscratch, env->sscratch); +KVM_RISCV_GET_CSR (cs, env, sepc, env->sepc); +KVM_RISCV_GET_CSR (cs, env, scause, env->scause); +KVM_RISCV_GET_CSR (cs, env, stval, env->stval); +KVM_RISCV_GET_CSR (cs, env, sip, env->mip); +KVM_RISCV_GET_CSR (cs, env, satp, env->satp); +return ret; +} + +static int kvm_riscv_get_regs_fp (CPUState *cs) +{ +int ret =0 ; +int i; +CPURISCVState *env = &RISCV_CPU (cs)->env; + +if (riscv_has_ext (env,RVD )) { +uint64_t reg; +for (i =0 ; i <32 ; i++) { + ret =kvm_get_one_reg (cs,RISCV_FP_D_REG (env, i), ®); +if (ret) { +return ret; + } + env->fpr[i] = reg; + } +return ret; + } + +if (riscv_has_ext (env,RVF )) { +uint32_t reg; +for (i =0 ; i <32 ; i++) { + ret =kvm_get_one_reg (cs,RISCV_FP_F_REG (env, i), ®); +if (ret) { +return ret; + } + env->fpr[i] = reg; + } +return ret; + } + +return ret; +} +const KVMCapabilityInfo kvm_arch_required_capabilities[] = {KVM_CAP_LAST_INFO };int kvm_arch_get_registers (CPUState *cs) { -return 0 ; +int ret =0 ; + + ret =kvm_riscv_get_regs_core (cs); +if (ret) { +return ret; + } + + ret =kvm_riscv_get_regs_csr (cs); +if (ret) { +return ret; + } + + ret =kvm_riscv_get_regs_fp (cs); +if (ret) { +return ret; + } + +return ret; }int kvm_arch_put_registers (CPUState *cs,int level)
Signature · 1 line
2.19.1
PATCH v3 5/12 target/riscv: Implement kvm_arch_put_registers
In reply to Yifei JiangPut GPR CSR and FP registers to kvm by KVM_SET_ONE_REG ioctl
--- target/riscv/kvm.c | 104 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 103 insertions(+), 1 deletion(-)
target/riscv/kvm.c +103 -1
@@ -73,6 +73,14 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx } \ } while(0) +#define KVM_RISCV_SET_CSR (cs, env, csr, reg) \ + do { \ + int ret = kvm_set_one_reg(cs, RISCV_CSR_REG(env, csr), ®); \ + if (ret) { \ + return ret; \ + } \ + } while(0) +static int kvm_riscv_get_regs_core (CPUState *cs) {int ret =0 ;
@@ -98,6 +106,31 @@ static int kvm_riscv_get_regs_core(CPUState *cs)return ret; } +static int kvm_riscv_put_regs_core (CPUState *cs) +{ +int ret =0 ; +int i; +target_ulong reg; +CPURISCVState *env = &RISCV_CPU (cs)->env; + + reg = env->pc; + ret =kvm_set_one_reg (cs,RISCV_CORE_REG (env, regs.pc), ®); +if (ret) { +return ret; + } + +for (i =1 ; i <32 ; i++) { +uint64_t id =kvm_riscv_reg_id (env,KVM_REG_RISCV_CORE , i); + reg = env->gpr[i]; + ret =kvm_set_one_reg (cs, id, ®); +if (ret) { +return ret; + } + } + +return ret; +} +static int kvm_riscv_get_regs_csr (CPUState *cs) {int ret =0 ;
@@ -115,6 +148,24 @@ static int kvm_riscv_get_regs_csr(CPUState *cs)return ret; } +static int kvm_riscv_put_regs_csr (CPUState *cs) +{ +int ret =0 ; +CPURISCVState *env = &RISCV_CPU (cs)->env; + +KVM_RISCV_SET_CSR (cs, env, sstatus, env->mstatus); +KVM_RISCV_SET_CSR (cs, env, sie, env->mie); +KVM_RISCV_SET_CSR (cs, env, stvec, env->stvec); +KVM_RISCV_SET_CSR (cs, env, sscratch, env->sscratch); +KVM_RISCV_SET_CSR (cs, env, sepc, env->sepc); +KVM_RISCV_SET_CSR (cs, env, scause, env->scause); +KVM_RISCV_SET_CSR (cs, env, stval, env->stval); +KVM_RISCV_SET_CSR (cs, env, sip, env->mip); +KVM_RISCV_SET_CSR (cs, env, satp, env->satp); + +return ret; +} +static int kvm_riscv_get_regs_fp (CPUState *cs) {int ret =0 ;
@@ -148,6 +199,40 @@ static int kvm_riscv_get_regs_fp(CPUState *cs)return ret; } +static int kvm_riscv_put_regs_fp (CPUState *cs) +{ +int ret =0 ; +int i; +CPURISCVState *env = &RISCV_CPU (cs)->env; + +if (riscv_has_ext (env,RVD )) { +uint64_t reg; +for (i =0 ; i <32 ; i++) { + reg = env->fpr[i]; + ret =kvm_set_one_reg (cs,RISCV_FP_D_REG (env, i), ®); +if (ret) { +return ret; + } + } +return ret; + } + +if (riscv_has_ext (env,RVF )) { +uint32_t reg; +for (i =0 ; i <32 ; i++) { + reg = env->fpr[i]; + ret =kvm_set_one_reg (cs,RISCV_FP_F_REG (env, i), ®); +if (ret) { +return ret; + } + } +return ret; + } + +return ret; +} + +const KVMCapabilityInfo kvm_arch_required_capabilities[] = {KVM_CAP_LAST_INFO };
@@ -176,7 +261,24 @@ int kvm_arch_get_registers(CPUState *cs)int kvm_arch_put_registers (CPUState *cs,int level) { -return 0 ; +int ret =0 ; + + ret =kvm_riscv_put_regs_core (cs); +if (ret) { +return ret; + } + + ret =kvm_riscv_put_regs_csr (cs); +if (ret) { +return ret; + } + + ret =kvm_riscv_put_regs_fp (cs); +if (ret) { +return ret; + } + +return ret; }int kvm_arch_release_virq_post (int virq)
Signature · 1 line
2.19.1
PATCH v3 6/12 target/riscv: Support start kernel directly by KVM
In reply to Yifei JiangGet kernel and fdt start address in virt.c, and pass them to KVM when cpu reset. Add kvm_riscv.h to place riscv specific interface.
In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.
--- hw/intc/sifive_plic.c | 8 +++- hw/riscv/boot.c | 16 +++++++- hw/riscv/virt.c | 87 ++++++++++++++++++++++++++++------------ include/hw/riscv/boot.h | 1 + target/riscv/cpu.c | 8 ++++ target/riscv/cpu.h | 3 ++ target/riscv/kvm-stub.c | 25 ++++++++++++ target/riscv/kvm.c | 14 +++++++ target/riscv/kvm_riscv.h | 24 +++++++++++ target/riscv/meson.build | 2 +- 10 files changed, 159 insertions(+), 29 deletions(-) create mode 100644 target/riscv/kvm-stub.c create mode 100644 target/riscv/kvm_riscv.h
hw/intc/sifive_plic.c +6 -2
@@ -30,6 +30,7 @@#include "target/riscv/cpu.h" #include "migration/vmstate.h" #include "hw/irq.h" +#include "sysemu/kvm.h" #define RISCV_DEBUG_PLIC 0
@@ -555,8 +556,11 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,qdev_connect_gpio_out (dev, i,qdev_get_gpio_in (DEVICE (cpu),IRQ_S_EXT )); -qdev_connect_gpio_out (dev, num_harts + i, -qdev_get_gpio_in (DEVICE (cpu),IRQ_M_EXT )); + +if (!kvm_enabled ()) { +qdev_connect_gpio_out (dev, num_harts + i, +qdev_get_gpio_in (DEVICE (cpu),IRQ_M_EXT )); + } }return dev;
hw/riscv/boot.c +15 -1
@@ -30,6 +30,7 @@#include "elf.h" #include "sysemu/device_tree.h" #include "sysemu/qtest.h" +#include "sysemu/kvm.h" #include <libfdt.h>
@@ -51,7 +52,9 @@ char *riscv_plic_hart_config_string(int hart_count)CPUState *cs =qemu_get_cpu (i);CPURISCVState *env = &RISCV_CPU (cs)->env; -if (riscv_has_ext (env,RVS )) { +if (kvm_enabled ()) { + vals[i] ="S" ; + }else if (riscv_has_ext (env,RVS )) { vals[i] ="MS" ; }else { vals[i] ="M" ;
@@ -317,3 +320,14 @@ void riscv_setup_rom_reset_vec(MachineState *machine, RISCVHartArrayState *hartsreturn ; } + +void riscv_setup_direct_kernel (hwaddr kernel_addr,hwaddr fdt_addr) +{ +CPUState *cs; + +for (cs = first_cpu; cs; cs =CPU_NEXT (cs)) { +RISCVCPU *riscv_cpu =RISCV_CPU (cs); + riscv_cpu->env.kernel_addr = kernel_addr; + riscv_cpu->env.fdt_addr = fdt_addr; + } +}
hw/riscv/virt.c +62 -25
133 lines not on this page · raw message
include/hw/riscv/boot.h +1 -0
@@ -58,5 +58,6 @@ void riscv_rom_copy_firmware_info(MachineState *machine, hwaddr rom_base,hwaddr rom_size, uint32_t reset_vec_size, uint64_t kernel_entry); +void riscv_setup_direct_kernel (hwaddr kernel_addr,hwaddr fdt_addr);#endif /* RISCV_BOOT_H */
target/riscv/cpu.c +8 -0
@@ -29,6 +29,8 @@#include "hw/qdev-properties.h" #include "migration/vmstate.h" #include "fpu/softfloat-helpers.h" +#include "sysemu/kvm.h" +#include "kvm_riscv.h" /* RISC-V CPU definitions */
@@ -380,6 +382,12 @@ static void riscv_cpu_reset(DeviceState *dev) cs->exception_index =RISCV_EXCP_NONE ; env->load_res =-1 ;set_default_nan_mode (1 , &env->fp_status); + +#ifndef CONFIG_USER_ONLY +if (kvm_enabled ()) { +kvm_riscv_reset_vcpu (cpu); + } +#endif }static void riscv_cpu_disas_set_info (CPUState *s,disassemble_info *info)
target/riscv/cpu.h +3 -0
@@ -255,6 +255,9 @@ struct CPURISCVState {/* Fields from here on are preserved across CPU reset. */ QEMUTimer *timer;/* Internal timer */ + +hwaddr kernel_addr; +hwaddr fdt_addr; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,
target/riscv/kvm-stub.c new file +25 -0
@@ -0,0 +1,25 @@ +/* +* QEMU KVM RISC-V specific function stubs +* +* Copyright (c) 2020 Huawei Technologies Co., Ltd +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ +#include "qemu/osdep.h" +#include "cpu.h" +#include "kvm_riscv.h" + +void kvm_riscv_reset_vcpu (RISCVCPU *cpu) +{ +abort (); +}
target/riscv/kvm.c +14 -0
@@ -37,6 +37,7 @@#include "hw/irq.h" #include "qemu/log.h" #include "hw/loader.h" +#include "kvm_riscv.h" static uint64_t kvm_riscv_reg_id (CPURISCVState *env,uint64_t type,uint64_t idx) {
@@ -369,6 +370,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)return 0 ; } +void kvm_riscv_reset_vcpu (RISCVCPU *cpu) +{ +CPURISCVState *env = &cpu->env; + +if (!kvm_enabled ()) { +return ; + } + env->pc = cpu->env.kernel_addr; + env->gpr[10 ] =kvm_arch_vcpu_id (CPU (cpu));/* a0 */ + env->gpr[11 ] = cpu->env.fdt_addr;/* a1 */ + env->satp =0 ; +} +bool kvm_arch_cpu_check_are_resettable (void ) {return true;
target/riscv/kvm_riscv.h new file +24 -0
@@ -0,0 +1,24 @@ +/* +* QEMU KVM support -- RISC-V specific functions. +* +* Copyright (c) 2020 Huawei Technologies Co., Ltd +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#ifndef QEMU_KVM_RISCV_H +#define QEMU_KVM_RISCV_H + +void kvm_riscv_reset_vcpu (RISCVCPU *cpu); + +#endif
target/riscv/meson.build +1 -1
@@ -19,7 +19,7 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', )) -riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c')) +riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c')) riscv_softmmu_ss = ss.source_set() riscv_softmmu_ss.add(files(
Signature · 1 line
2.19.1
Anup Patel
maintainerRe: [PATCH v3 06/12] target/riscv: Support start kernel directly by KVM
Quoted 38 lines · Yifei Jiang
Get kernel and fdt start address in
virt.c, and pass them to KVM when cpu reset. Addkvm_riscv.hto place riscv specific interface.In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.
Signed-off-by Yifei Jiang <[email protected]>Signed-off-by Mingwang Li <[email protected]>Reviewed-by Alistair Francis <[email protected]>--- hw/intc/sifive_plic.c | 8 +++- hw/riscv/boot.c | 16 +++++++- hw/riscv/virt.c | 87 ++++++++++++++++++++++++++++------------ include/hw/riscv/boot.h | 1 + target/riscv/cpu.c | 8 ++++ target/riscv/cpu.h | 3 ++ target/riscv/kvm-stub.c | 25 ++++++++++++ target/riscv/kvm.c | 14 +++++++ target/riscv/kvm_riscv.h | 24 +++++++++++ target/riscv/meson.build | 2 +- 10 files changed, 159 insertions(+), 29 deletions(-) create mode 100644 target/riscv/kvm-stub.c create mode 100644 target/riscv/kvm_riscv.hhw/intc/sifive_plic.c +1
-0@@ -30,6 +30,7 @@#include "target/riscv/cpu.h" #include "migration/vmstate.h" #include "hw/irq.h" +#include "sysemu/kvm.h" #define RISCV_DEBUG_PLIC 0
hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, · line 563
Quoted 10 lines
hw/intc/sifive_plic.c +5
-2@@ -555,8 +556,11 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,qdev_connect_gpio_out (dev, i,qdev_get_gpio_in (DEVICE (cpu),IRQ_S_EXT )); -qdev_connect_gpio_out (dev, num_harts + i, -qdev_get_gpio_in (DEVICE (cpu),IRQ_M_EXT )); + +if (!kvm_enabled ()) { +qdev_connect_gpio_out (dev, num_harts + i, +qdev_get_gpio_in (DEVICE (cpu),IRQ_M_EXT )); + }
We should have any KVM enabled check in interrupt controller emulators instead I suggest to re-write the for-loop in sifive_plic_create() as follows:
SiFivePLICState *plic;
(....)
plic = SIFIVE_PLIC(dev);
for (i = 0; i < plic->num_addrs; i++) {
CPUState *cpu = qemu_get_cpu(plic->addr_config[i].hartid); if (plic->addr_mode[i] == PLICMode_S) {
qdev_connect_gpio_out(dev, i,
qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT));
}
if (plic->addr_mode[i] == PLICMode_M) {
qdev_connect_gpio_out(dev, num_harts + i,
qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
}
}Quoted 54 lines
hw/intc/sifive_plic.c +0
-0}return dev;hw/riscv/boot.c +15
-1@@ -30,6 +30,7 @@#include "elf.h" #include "sysemu/device_tree.h" #include "sysemu/qtest.h" +#include "sysemu/kvm.h" #include <libfdt.h> @@ -51,7 +52,9 @@ char *riscv_plic_hart_config_string(int hart_count)CPUState *cs =qemu_get_cpu (i);CPURISCVState *env = &RISCV_CPU (cs)->env; -if (riscv_has_ext (env,RVS )) { +if (kvm_enabled ()) { + vals[i] ="S" ; + }else if (riscv_has_ext (env,RVS )) { vals[i] ="MS" ; }else { vals[i] ="M" ;@@ -317,3 +320,14 @@ void riscv_setup_rom_reset_vec(MachineState *machine, RISCVHartArrayState *hartsreturn ; } + +void riscv_setup_direct_kernel (hwaddr kernel_addr,hwaddr fdt_addr) +{ +CPUState *cs; + +for (cs = first_cpu; cs; cs =CPU_NEXT (cs)) { +RISCVCPU *riscv_cpu =RISCV_CPU (cs); + riscv_cpu->env.kernel_addr = kernel_addr; + riscv_cpu->env.fdt_addr = fdt_addr; + } +}hw/riscv/virt.c +1
-0@@ -38,6 +38,7 @@#include "chardev/char.h" #include "sysemu/device_tree.h" #include "sysemu/sysemu.h" +#include "sysemu/kvm.h" #include "hw/pci/pci.h" #include "hw/pci-host/gpex.h" #include "hw/display/ramfb.h"
hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 58
Quoted 8 lines
hw/riscv/virt.c +4
-0@@ -50,7 +51,11 @@ static const MemMapEntry virt_memmap[] = { [VIRT_CLINT] = { 0x2000000, 0x10000 }, [VIRT_ACLINT_SSWI] = { 0x2F00000, 0x4000 }, [VIRT_PCIE_PIO ] = {0x3000000 ,0x10000 }, +#if defined(CONFIG_KVM) + [VIRT_PLIC ] = {0xc000000 ,VIRT_PLIC_SIZE (VIRT_CPUS_MAX *1 ) }, +#else [VIRT_PLIC] = { 0xc000000, VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 2) }, +#endif
Please drop this change because the same QEMU binary should be usable with and without KVM enabled.
Quoted 281 lines
hw/riscv/virt.c +57
-25[VIRT_UART0] = { 0x10000000, 0x100 }, [VIRT_VIRTIO] = { 0x10001000, 0x1000 }, [VIRT_FW_CFG] = { 0x10100000, 0x18 },@@ -372,13 +377,22 @@ static void create_fdt_socket_plic(RISCVVirtState *s,"sifive,plic-1.0.0" ,"riscv,plic0" }; - plic_cells =g_new0 (uint32_t, s->soc[socket].num_harts *4 ); +if (kvm_enabled ()) { + plic_cells =g_new0 (uint32_t, s->soc[socket].num_harts *2 ); + }else { + plic_cells =g_new0 (uint32_t, s->soc[socket].num_harts *4 ); + }for (cpu =0 ; cpu < s->soc[socket].num_harts; cpu++) { - plic_cells[cpu *4 +0 ] =cpu_to_be32 (intc_phandles[cpu]); - plic_cells[cpu *4 +1 ] =cpu_to_be32 (IRQ_M_EXT ); - plic_cells[cpu *4 +2 ] =cpu_to_be32 (intc_phandles[cpu]); - plic_cells[cpu *4 +3 ] =cpu_to_be32 (IRQ_S_EXT ); +if (kvm_enabled ()) { + plic_cells[cpu *2 +0 ] =cpu_to_be32 (intc_phandles[cpu]); + plic_cells[cpu *2 +1 ] =cpu_to_be32 (IRQ_S_EXT ); + }else { + plic_cells[cpu *4 +0 ] =cpu_to_be32 (intc_phandles[cpu]); + plic_cells[cpu *4 +1 ] =cpu_to_be32 (IRQ_M_EXT ); + plic_cells[cpu *4 +2 ] =cpu_to_be32 (intc_phandles[cpu]); + plic_cells[cpu *4 +3 ] =cpu_to_be32 (IRQ_S_EXT ); + } } plic_phandles[socket] = (*phandle)++;@@ -436,10 +450,12 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,create_fdt_socket_memory (s, memmap, socket); -if (s->have_aclint) { -create_fdt_socket_aclint (s, memmap, socket, intc_phandles); - }else { -create_fdt_socket_clint (s, memmap, socket, intc_phandles); +if (!kvm_enabled ()) { +if (s->have_aclint) { +create_fdt_socket_aclint (s, memmap, socket, intc_phandles); + }else { +create_fdt_socket_clint (s, memmap, socket, intc_phandles); + } } create_fdt_socket_plic(s, memmap, socket, phandle,@@ -801,23 +817,25 @@ static void virt_machine_init(MachineState *machine) hart_count, &error_abort);sysbus_realize (SYS_BUS_DEVICE (&s->soc[i]), &error_abort); -/* Per-socket CLINT */ -riscv_aclint_swi_create ( - memmap[VIRT_CLINT ].base + i * memmap[VIRT_CLINT ].size, - base_hartid, hart_count, false); -riscv_aclint_mtimer_create ( - memmap[VIRT_CLINT ].base + i * memmap[VIRT_CLINT ].size + -RISCV_ACLINT_SWI_SIZE , -RISCV_ACLINT_DEFAULT_MTIMER_SIZE , base_hartid, hart_count, -RISCV_ACLINT_DEFAULT_MTIMECMP ,RISCV_ACLINT_DEFAULT_MTIME , -RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ , true); - -/* Per-socket ACLINT SSWI */ -if (s->have_aclint) { +if (!kvm_enabled ()) { +/* Per-socket CLINT */ riscv_aclint_swi_create ( - memmap[VIRT_ACLINT_SSWI ].base + - i * memmap[VIRT_ACLINT_SSWI ].size, - base_hartid, hart_count, true); + memmap[VIRT_CLINT ].base + i * memmap[VIRT_CLINT ].size, + base_hartid, hart_count, false); +riscv_aclint_mtimer_create ( + memmap[VIRT_CLINT ].base + i * memmap[VIRT_CLINT ].size + +RISCV_ACLINT_SWI_SIZE , +RISCV_ACLINT_DEFAULT_MTIMER_SIZE , base_hartid, hart_count, +RISCV_ACLINT_DEFAULT_MTIMECMP ,RISCV_ACLINT_DEFAULT_MTIME , +RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ , true); + +/* Per-socket ACLINT SSWI */ +if (s->have_aclint) { +riscv_aclint_swi_create ( + memmap[VIRT_ACLINT_SSWI ].base + + i * memmap[VIRT_ACLINT_SSWI ].size, + base_hartid, hart_count, true); + } }/* Per-socket PLIC hart topology configuration string */ @@ -884,6 +902,16 @@ static void virt_machine_init(MachineState *machine)memory_region_add_subregion (system_memory, memmap[VIRT_MROM ].base, mask_rom); +/* +* Only direct boot kernel is currently supported for KVM VM, +* so the "-bios" parameter is ignored and treated like "-bios none" +* when KVM is enabled. +*/ +if (kvm_enabled ()) { +g_free (machine->firmware); + machine->firmware =g_strdup ("none" ); + } +if (riscv_is_32bit (&s->soc[0 ])) { firmware_end_addr =riscv_find_and_load_firmware (machine,RISCV32_BIOS_BIN , start_addr,NULL );@@ -941,6 +969,15 @@ static void virt_machine_init(MachineState *machine) virt_memmap[VIRT_MROM ].size, kernel_entry, fdt_load_addr, machine->fdt); +/* +* Only direct boot kernel is currently supported for KVM VM, +* So here setup kernel start address and fdt address. +* TODO:Support firmware loading and integrate to TCG start +*/ +if (kvm_enabled ()) { +riscv_setup_direct_kernel (kernel_entry, fdt_load_addr); + } +/* SiFive Test MMIO device */ sifive_test_create (memmap[VIRT_TEST ].base);include/hw/riscv/boot.h +1
-0@@ -58,5 +58,6 @@ void riscv_rom_copy_firmware_info(MachineState *machine, hwaddr rom_base,hwaddr rom_size, uint32_t reset_vec_size, uint64_t kernel_entry); +void riscv_setup_direct_kernel (hwaddr kernel_addr,hwaddr fdt_addr);#endif /* RISCV_BOOT_H */ target/riscv/cpu.c +8
-0@@ -29,6 +29,8 @@#include "hw/qdev-properties.h" #include "migration/vmstate.h" #include "fpu/softfloat-helpers.h" +#include "sysemu/kvm.h" +#include "kvm_riscv.h" /* RISC-V CPU definitions */ @@ -380,6 +382,12 @@ static void riscv_cpu_reset(DeviceState *dev) cs->exception_index =RISCV_EXCP_NONE ; env->load_res =-1 ;set_default_nan_mode (1 , &env->fp_status); + +#ifndef CONFIG_USER_ONLY +if (kvm_enabled ()) { +kvm_riscv_reset_vcpu (cpu); + } +#endif }static void riscv_cpu_disas_set_info (CPUState *s,disassemble_info *info)target/riscv/cpu.h +3
-0@@ -255,6 +255,9 @@ struct CPURISCVState {/* Fields from here on are preserved across CPU reset. */ QEMUTimer *timer;/* Internal timer */ + +hwaddr kernel_addr; +hwaddr fdt_addr; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,target/riscv/kvm-stub.c new file +25
-0@@ -0,0 +1,25 @@ +/* +* QEMU KVM RISC-V specific function stubs +* +* Copyright (c) 2020 Huawei Technologies Co., Ltd +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ +#include "qemu/osdep.h" +#include "cpu.h" +#include "kvm_riscv.h" + +void kvm_riscv_reset_vcpu (RISCVCPU *cpu) +{ +abort (); +}target/riscv/kvm.c +14
-0@@ -37,6 +37,7 @@#include "hw/irq.h" #include "qemu/log.h" #include "hw/loader.h" +#include "kvm_riscv.h" static uint64_t kvm_riscv_reg_id (CPURISCVState *env,uint64_t type,uint64_t idx) {@@ -369,6 +370,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)return 0 ; } +void kvm_riscv_reset_vcpu (RISCVCPU *cpu) +{ +CPURISCVState *env = &cpu->env; + +if (!kvm_enabled ()) { +return ; + } + env->pc = cpu->env.kernel_addr; + env->gpr[10 ] =kvm_arch_vcpu_id (CPU (cpu));/* a0 */ + env->gpr[11 ] = cpu->env.fdt_addr;/* a1 */ + env->satp =0 ; +} +bool kvm_arch_cpu_check_are_resettable (void ) {return true;target/riscv/kvm_riscv.h new file +24
-0@@ -0,0 +1,24 @@ +/* +* QEMU KVM support -- RISC-V specific functions. +* +* Copyright (c) 2020 Huawei Technologies Co., Ltd +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#ifndef QEMU_KVM_RISCV_H +#define QEMU_KVM_RISCV_H + +void kvm_riscv_reset_vcpu (RISCVCPU *cpu); + +#endif target/riscv/meson.build +1
-2@@ -19,7 +19,7 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', )) -riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c')) +riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c')) riscv_softmmu_ss = ss.source_set() riscv_softmmu_ss.add(files( --
Quoted 2 lines
2.19.1
Regards,
Anup
Anup Patel
maintainerRe: [PATCH v3 06/12] target/riscv: Support start kernel directly by KVM
Quoted 41 lines · Anup Patel
Quoted 39 lines · Yifei Jiang
Get kernel and fdt start address in
virt.c, and pass them to KVM when cpu reset. Addkvm_riscv.hto place riscv specific interface.In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.
Signed-off-by Yifei Jiang <[email protected]>Signed-off-by Mingwang Li <[email protected]>Reviewed-by Alistair Francis <[email protected]>--- hw/intc/sifive_plic.c | 8 +++- hw/riscv/boot.c | 16 +++++++- hw/riscv/virt.c | 87 ++++++++++++++++++++++++++++------------ include/hw/riscv/boot.h | 1 + target/riscv/cpu.c | 8 ++++ target/riscv/cpu.h | 3 ++ target/riscv/kvm-stub.c | 25 ++++++++++++ target/riscv/kvm.c | 14 +++++++ target/riscv/kvm_riscv.h | 24 +++++++++++ target/riscv/meson.build | 2 +- 10 files changed, 159 insertions(+), 29 deletions(-) create mode 100644 target/riscv/kvm-stub.c create mode 100644 target/riscv/kvm_riscv.hhw/intc/sifive_plic.c +1
-0@@ -30,6 +30,7 @@#include "target/riscv/cpu.h" #include "migration/vmstate.h" #include "hw/irq.h" +#include "sysemu/kvm.h" #define RISCV_DEBUG_PLIC 0@@ -555,8 +556,11 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,
Quoted 12 lines
hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, · line 563
Quoted 9 lines
hw/intc/sifive_plic.c +5
-2qdev_connect_gpio_out (dev, i,qdev_get_gpio_in (DEVICE (cpu),IRQ_S_EXT )); -qdev_connect_gpio_out (dev, num_harts + i, -qdev_get_gpio_in (DEVICE (cpu),IRQ_M_EXT )); + +if (!kvm_enabled ()) { +qdev_connect_gpio_out (dev, num_harts + i, +qdev_get_gpio_in (DEVICE (cpu),IRQ_M_EXT )); + }We should have any KVM enabled check in interrupt controller emulators instead I suggest to re-write the for-loop in
sifive_plic_create()as follows:
Typo correction:
*"We should not have any KVM enabled check in ...."
Regards,
Anup
Quoted 371 lines
SiFivePLICState *plic;
(....)
plic =
SIFIVE_PLIC(dev);for (i = 0; i < plic->num_addrs; i++) { CPUState *cpu = qemu_get_cpu(plic->addr_config[i].hartid);if (plic->addr_mode[i] == PLICMode_S) { qdev_connect_gpio_out(dev, i, qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT)); } if (plic->addr_mode[i] == PLICMode_M) { qdev_connect_gpio_out(dev, num_harts + i, qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT)); } }Quoted 54 lines
hw/intc/sifive_plic.c +0
-0}return dev;hw/riscv/boot.c +15
-1@@ -30,6 +30,7 @@#include "elf.h" #include "sysemu/device_tree.h" #include "sysemu/qtest.h" +#include "sysemu/kvm.h" #include <libfdt.h> @@ -51,7 +52,9 @@ char *riscv_plic_hart_config_string(int hart_count)CPUState *cs =qemu_get_cpu (i);CPURISCVState *env = &RISCV_CPU (cs)->env; -if (riscv_has_ext (env,RVS )) { +if (kvm_enabled ()) { + vals[i] ="S" ; + }else if (riscv_has_ext (env,RVS )) { vals[i] ="MS" ; }else { vals[i] ="M" ;@@ -317,3 +320,14 @@ void riscv_setup_rom_reset_vec(MachineState *machine, RISCVHartArrayState *hartsreturn ; } + +void riscv_setup_direct_kernel (hwaddr kernel_addr,hwaddr fdt_addr) +{ +CPUState *cs; + +for (cs = first_cpu; cs; cs =CPU_NEXT (cs)) { +RISCVCPU *riscv_cpu =RISCV_CPU (cs); + riscv_cpu->env.kernel_addr = kernel_addr; + riscv_cpu->env.fdt_addr = fdt_addr; + } +}hw/riscv/virt.c +1
-0@@ -38,6 +38,7 @@#include "chardev/char.h" #include "sysemu/device_tree.h" #include "sysemu/sysemu.h" +#include "sysemu/kvm.h" #include "hw/pci/pci.h" #include "hw/pci-host/gpex.h" #include "hw/display/ramfb.h" hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 58
Quoted 8 lines
hw/riscv/virt.c +4
-0@@ -50,7 +51,11 @@ static const MemMapEntry virt_memmap[] = { [VIRT_CLINT] = { 0x2000000, 0x10000 }, [VIRT_ACLINT_SSWI] = { 0x2F00000, 0x4000 }, [VIRT_PCIE_PIO ] = {0x3000000 ,0x10000 }, +#if defined(CONFIG_KVM) + [VIRT_PLIC ] = {0xc000000 ,VIRT_PLIC_SIZE (VIRT_CPUS_MAX *1 ) }, +#else [VIRT_PLIC] = { 0xc000000, VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 2) }, +#endifPlease drop this change because the same QEMU binary should be usable with and without KVM enabled.
Quoted 281 lines
hw/riscv/virt.c +57
-25[VIRT_UART0] = { 0x10000000, 0x100 }, [VIRT_VIRTIO] = { 0x10001000, 0x1000 }, [VIRT_FW_CFG] = { 0x10100000, 0x18 },@@ -372,13 +377,22 @@ static void create_fdt_socket_plic(RISCVVirtState *s,"sifive,plic-1.0.0" ,"riscv,plic0" }; - plic_cells =g_new0 (uint32_t, s->soc[socket].num_harts *4 ); +if (kvm_enabled ()) { + plic_cells =g_new0 (uint32_t, s->soc[socket].num_harts *2 ); + }else { + plic_cells =g_new0 (uint32_t, s->soc[socket].num_harts *4 ); + }for (cpu =0 ; cpu < s->soc[socket].num_harts; cpu++) { - plic_cells[cpu *4 +0 ] =cpu_to_be32 (intc_phandles[cpu]); - plic_cells[cpu *4 +1 ] =cpu_to_be32 (IRQ_M_EXT ); - plic_cells[cpu *4 +2 ] =cpu_to_be32 (intc_phandles[cpu]); - plic_cells[cpu *4 +3 ] =cpu_to_be32 (IRQ_S_EXT ); +if (kvm_enabled ()) { + plic_cells[cpu *2 +0 ] =cpu_to_be32 (intc_phandles[cpu]); + plic_cells[cpu *2 +1 ] =cpu_to_be32 (IRQ_S_EXT ); + }else { + plic_cells[cpu *4 +0 ] =cpu_to_be32 (intc_phandles[cpu]); + plic_cells[cpu *4 +1 ] =cpu_to_be32 (IRQ_M_EXT ); + plic_cells[cpu *4 +2 ] =cpu_to_be32 (intc_phandles[cpu]); + plic_cells[cpu *4 +3 ] =cpu_to_be32 (IRQ_S_EXT ); + } } plic_phandles[socket] = (*phandle)++;@@ -436,10 +450,12 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,create_fdt_socket_memory (s, memmap, socket); -if (s->have_aclint) { -create_fdt_socket_aclint (s, memmap, socket, intc_phandles); - }else { -create_fdt_socket_clint (s, memmap, socket, intc_phandles); +if (!kvm_enabled ()) { +if (s->have_aclint) { +create_fdt_socket_aclint (s, memmap, socket, intc_phandles); + }else { +create_fdt_socket_clint (s, memmap, socket, intc_phandles); + } } create_fdt_socket_plic(s, memmap, socket, phandle,@@ -801,23 +817,25 @@ static void virt_machine_init(MachineState *machine) hart_count, &error_abort);sysbus_realize (SYS_BUS_DEVICE (&s->soc[i]), &error_abort); -/* Per-socket CLINT */ -riscv_aclint_swi_create ( - memmap[VIRT_CLINT ].base + i * memmap[VIRT_CLINT ].size, - base_hartid, hart_count, false); -riscv_aclint_mtimer_create ( - memmap[VIRT_CLINT ].base + i * memmap[VIRT_CLINT ].size + -RISCV_ACLINT_SWI_SIZE , -RISCV_ACLINT_DEFAULT_MTIMER_SIZE , base_hartid, hart_count, -RISCV_ACLINT_DEFAULT_MTIMECMP ,RISCV_ACLINT_DEFAULT_MTIME , -RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ , true); - -/* Per-socket ACLINT SSWI */ -if (s->have_aclint) { +if (!kvm_enabled ()) { +/* Per-socket CLINT */ riscv_aclint_swi_create ( - memmap[VIRT_ACLINT_SSWI ].base + - i * memmap[VIRT_ACLINT_SSWI ].size, - base_hartid, hart_count, true); + memmap[VIRT_CLINT ].base + i * memmap[VIRT_CLINT ].size, + base_hartid, hart_count, false); +riscv_aclint_mtimer_create ( + memmap[VIRT_CLINT ].base + i * memmap[VIRT_CLINT ].size + +RISCV_ACLINT_SWI_SIZE , +RISCV_ACLINT_DEFAULT_MTIMER_SIZE , base_hartid, hart_count, +RISCV_ACLINT_DEFAULT_MTIMECMP ,RISCV_ACLINT_DEFAULT_MTIME , +RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ , true); + +/* Per-socket ACLINT SSWI */ +if (s->have_aclint) { +riscv_aclint_swi_create ( + memmap[VIRT_ACLINT_SSWI ].base + + i * memmap[VIRT_ACLINT_SSWI ].size, + base_hartid, hart_count, true); + } }/* Per-socket PLIC hart topology configuration string */ @@ -884,6 +902,16 @@ static void virt_machine_init(MachineState *machine)memory_region_add_subregion (system_memory, memmap[VIRT_MROM ].base, mask_rom); +/* +* Only direct boot kernel is currently supported for KVM VM, +* so the "-bios" parameter is ignored and treated like "-bios none" +* when KVM is enabled. +*/ +if (kvm_enabled ()) { +g_free (machine->firmware); + machine->firmware =g_strdup ("none" ); + } +if (riscv_is_32bit (&s->soc[0 ])) { firmware_end_addr =riscv_find_and_load_firmware (machine,RISCV32_BIOS_BIN , start_addr,NULL );@@ -941,6 +969,15 @@ static void virt_machine_init(MachineState *machine) virt_memmap[VIRT_MROM ].size, kernel_entry, fdt_load_addr, machine->fdt); +/* +* Only direct boot kernel is currently supported for KVM VM, +* So here setup kernel start address and fdt address. +* TODO:Support firmware loading and integrate to TCG start +*/ +if (kvm_enabled ()) { +riscv_setup_direct_kernel (kernel_entry, fdt_load_addr); + } +/* SiFive Test MMIO device */ sifive_test_create (memmap[VIRT_TEST ].base);include/hw/riscv/boot.h +1
-0@@ -58,5 +58,6 @@ void riscv_rom_copy_firmware_info(MachineState *machine, hwaddr rom_base,hwaddr rom_size, uint32_t reset_vec_size, uint64_t kernel_entry); +void riscv_setup_direct_kernel (hwaddr kernel_addr,hwaddr fdt_addr);#endif /* RISCV_BOOT_H */ target/riscv/cpu.c +8
-0@@ -29,6 +29,8 @@#include "hw/qdev-properties.h" #include "migration/vmstate.h" #include "fpu/softfloat-helpers.h" +#include "sysemu/kvm.h" +#include "kvm_riscv.h" /* RISC-V CPU definitions */ @@ -380,6 +382,12 @@ static void riscv_cpu_reset(DeviceState *dev) cs->exception_index =RISCV_EXCP_NONE ; env->load_res =-1 ;set_default_nan_mode (1 , &env->fp_status); + +#ifndef CONFIG_USER_ONLY +if (kvm_enabled ()) { +kvm_riscv_reset_vcpu (cpu); + } +#endif }static void riscv_cpu_disas_set_info (CPUState *s,disassemble_info *info)target/riscv/cpu.h +3
-0@@ -255,6 +255,9 @@ struct CPURISCVState {/* Fields from here on are preserved across CPU reset. */ QEMUTimer *timer;/* Internal timer */ + +hwaddr kernel_addr; +hwaddr fdt_addr; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,target/riscv/kvm-stub.c new file +25
-0@@ -0,0 +1,25 @@ +/* +* QEMU KVM RISC-V specific function stubs +* +* Copyright (c) 2020 Huawei Technologies Co., Ltd +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ +#include "qemu/osdep.h" +#include "cpu.h" +#include "kvm_riscv.h" + +void kvm_riscv_reset_vcpu (RISCVCPU *cpu) +{ +abort (); +}target/riscv/kvm.c +14
-0@@ -37,6 +37,7 @@#include "hw/irq.h" #include "qemu/log.h" #include "hw/loader.h" +#include "kvm_riscv.h" static uint64_t kvm_riscv_reg_id (CPURISCVState *env,uint64_t type,uint64_t idx) {@@ -369,6 +370,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)return 0 ; } +void kvm_riscv_reset_vcpu (RISCVCPU *cpu) +{ +CPURISCVState *env = &cpu->env; + +if (!kvm_enabled ()) { +return ; + } + env->pc = cpu->env.kernel_addr; + env->gpr[10 ] =kvm_arch_vcpu_id (CPU (cpu));/* a0 */ + env->gpr[11 ] = cpu->env.fdt_addr;/* a1 */ + env->satp =0 ; +} +bool kvm_arch_cpu_check_are_resettable (void ) {return true;target/riscv/kvm_riscv.h new file +24
-0@@ -0,0 +1,24 @@ +/* +* QEMU KVM support -- RISC-V specific functions. +* +* Copyright (c) 2020 Huawei Technologies Co., Ltd +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#ifndef QEMU_KVM_RISCV_H +#define QEMU_KVM_RISCV_H + +void kvm_riscv_reset_vcpu (RISCVCPU *cpu); + +#endif target/riscv/meson.build +1
-2@@ -19,7 +19,7 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', )) -riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c')) +riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c')) riscv_softmmu_ss = ss.source_set() riscv_softmmu_ss.add(files( --Quoted 2 lines
2.19.1
Regards,
Anup
RE: [PATCH v3 06/12] target/riscv: Support start kernel directly by KVM
Quoted 74 lines
-----Original Message-----
From: Anup Patel [mailto:anup at brainfault.org]
Sent: Thursday, December 23, 2021 2:04 PM
To: Jiangyifei <[email protected]>
Cc: QEMU Developers <[email protected]>; open list:RISC-V <[email protected]>; kvm-riscv at lists.infradead.org; KVM General <[email protected]>; libvir-list at redhat.com; Anup Patel <[email protected]>; Palmer Dabbelt <[email protected]>; Alistair Francis <[email protected]>; Bin Meng <[email protected]>; Fanliang (EulerOS) <[email protected]>; Wubin (H) <[email protected]>; Wanghaibin (D) <[email protected]>; wanbo (G) <[email protected]>; limingwang (A) <[email protected]>
Subject: Re: [PATCH v3 06/12] target/riscv: Support start kernel directly by KVMQuoted 2 lines · Anup Patel
On Mon, Dec 20, 2021 at 6:39 PM Yifei Jiang <[email protected]>
Quoted 7 lines · wrote:
Quoted 7 lines
Get kernel and fdt start address in
virt.c, and pass them to KVM when cpu reset. Addkvm_riscv.hto place riscv specific interface.In addition, PLIC is created without M-mode PLIC contexts when KVM is enabled.
Quoted 7 lines
Quoted 7 lines
Signed-off-by Yifei Jiang <[email protected]>Signed-off-by Mingwang Li <[email protected]>Reviewed-by Alistair Francis <[email protected]>--- hw/intc/sifive_plic.c | 8 +++- hw/riscv/boot.c | 16 +++++++- hw/riscv/virt.c | 87++++++++++++++++++++++++++++------------
Quoted 21 lines
Quoted 21 lines
include/hw/riscv/boot.h | 1 + target/riscv/cpu.c | 8 ++++ target/riscv/cpu.h | 3 ++ target/riscv/kvm-stub.c | 25 ++++++++++++ target/riscv/kvm.c | 14 +++++++ target/riscv/kvm_riscv.h | 24 +++++++++++ target/riscv/meson.build | 2 +- 10 files changed, 159 insertions(+), 29 deletions(-) create mode100644
target/riscv/kvm-stub.ccreate mode 100644target/riscv/kvm_riscv.hhw/intc/sifive_plic.c index renamed from hw/intc/sifive_plic.c +0
-0
877e76877c..1b2b4cc25e100644hw/intc/sifive_plic.c +1
-0@@ -30,6 +30,7 @@#include "target/riscv/cpu.h" #include "migration/vmstate.h" #include "hw/irq.h" +#include "sysemu/kvm.h" Quoted 7 lines
hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, · line 559
Quoted 7 lines
hw/intc/sifive_plic.c +0
-0#define RISCV_DEBUG_PLIC 0@@ -555,8 +556,11 @@ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, qdev_connect_gpio_out(dev, i, qdev_get_gpio_in(DEVICE(cpu),
IRQ_S_EXT));Quoted 2 lines
hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, · line 560
Quoted 2 lines
hw/intc/sifive_plic.c +0
-2- qdev_connect_gpio_out(dev, num_harts + i, - qdev_get_gpio_in(DEVICE(cpu),
IRQ_M_EXT));Quoted 4 lines
hw/intc/sifive_plic.c DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,
Quoted 4 lines
hw/intc/sifive_plic.c +4
-0+ +if (!kvm_enabled ()) { + qdev_connect_gpio_out(dev, num_harts + i, + qdev_get_gpio_in(DEVICE(cpu),
IRQ_M_EXT));Quoted 2 lines
Quoted 1 line
hw/intc/sifive_plic.c +1
-0+ }
Quoted 28 lines
Quoted 2 lines
We should have any KVM enabled check in interrupt controller emulators instead I suggest to re-write the for-loop in
sifive_plic_create()as follows:Typo correction:
*"We should not have any KVM enabled check in ...."Regards,
AnupQuoted 19 lines
SiFivePLICState *plic;
(....)
plic =
SIFIVE_PLIC(dev);for (i = 0; i < plic->num_addrs; i++) { CPUState *cpu = qemu_get_cpu(plic->addr_config[i].hartid);if (plic->addr_mode[i] == PLICMode_S) { qdev_connect_gpio_out(dev, i, qdev_get_gpio_in(DEVICE(cpu), IRQ_S_EXT)); } if (plic->addr_mode[i] == PLICMode_M) { qdev_connect_gpio_out(dev, num_harts + i, qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT)); } }
Yes, this looks better. It will be re-write in the next series as suggested.
Quoted 61 lines
Quoted 52 lines
Quoted 52 lines
hw/intc/sifive_plic.c +0
-0}return dev;hw/riscv/boot.c index renamed from hw/riscv/boot.c +0
-0
519fa455a1..ccff662d89100644hw/riscv/boot.c +15
-1@@ -30,6 +30,7 @@#include "elf.h" #include "sysemu/device_tree.h" #include "sysemu/qtest.h" +#include "sysemu/kvm.h" #include <libfdt.h> @@ -51,7 +52,9 @@ char *riscv_plic_hart_config_string(int hart_count)CPUState *cs =qemu_get_cpu (i);CPURISCVState *env = &RISCV_CPU (cs)->env; -if (riscv_has_ext (env,RVS )) { +if (kvm_enabled ()) { + vals[i] ="S" ; + }else if (riscv_has_ext (env,RVS )) { vals[i] ="MS" ; }else { vals[i] ="M" ;@@ -317,3 +320,14 @@ void riscv_setup_rom_reset_vec(MachineState *machine, RISCVHartArrayState *harts return; } + +void riscv_setup_direct_kernel (hwaddr kernel_addr,hwaddr fdt_addr) +{ +CPUState *cs; + +for (cs = first_cpu; cs; cs =CPU_NEXT (cs)) { +RISCVCPU *riscv_cpu =RISCV_CPU (cs); + riscv_cpu->env.kernel_addr = kernel_addr; + riscv_cpu->env.fdt_addr = fdt_addr; + } +}hw/riscv/virt.c index renamed from hw/riscv/virt.c +0
-0
3af074148e..cc1a03f284100644hw/riscv/virt.c +1
-0@@ -38,6 +38,7 @@#include "chardev/char.h" #include "sysemu/device_tree.h" #include "sysemu/sysemu.h" +#include "sysemu/kvm.h" #include "hw/pci/pci.h" Quoted 8 lines
hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 55
Quoted 8 lines
hw/riscv/virt.c +2
-0#include "hw/pci-host/gpex.h" #include "hw/display/ramfb.h" @@ -50,7 +51,11 @@ static const MemMapEntry virt_memmap[] = { [VIRT_CLINT] = { 0x2000000, 0x10000 }, [VIRT_ACLINT_SSWI] = { 0x2F00000, 0x4000 }, [VIRT_PCIE_PIO ] = {0x3000000 ,0x10000 }, +#if defined(CONFIG_KVM) + [VIRT_PLIC] = { 0xc000000,
VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 1)},
Quoted 8 lines
Quoted 2 lines
hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 57
Quoted 2 lines
hw/riscv/virt.c +1
-0+#else [VIRT_PLIC] = { 0xc000000,
VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 2)},Quoted 5 lines
hw/riscv/virt.c static const MemMapEntry virt_memmap[] = { · line 58
Quoted 1 line
hw/riscv/virt.c +1
-0+#endif Please drop this change because the same QEMU binary should be usable with and without KVM enabled.
It will be dropped in the next series.
Yifei
Quoted 315 lines
Quoted 4 lines
hw/riscv/virt.c static void
Quoted 4 lines
hw/riscv/virt.c +0
-0[VIRT_UART0] = { 0x10000000, 0x100 }, [VIRT_VIRTIO] = { 0x10001000, 0x1000 }, [VIRT_FW_CFG] = { 0x10100000, 0x18 },@@ -372,13 +377,22 @@ static voidcreate_fdt_socket_plic(RISCVVirtState *s,
Quoted 26 lines
Quoted 26 lines
hw/riscv/virt.c +14
-5"sifive,plic-1.0.0" ,"riscv,plic0" }; - plic_cells =g_new0 (uint32_t, s->soc[socket].num_harts *4 ); +if (kvm_enabled ()) { + plic_cells =g_new0 (uint32_t, s->soc[socket].num_harts *2 ); + }else { + plic_cells =g_new0 (uint32_t, s->soc[socket].num_harts *4 ); + }for (cpu =0 ; cpu < s->soc[socket].num_harts; cpu++) { - plic_cells[cpu *4 +0 ] =cpu_to_be32 (intc_phandles[cpu]); - plic_cells[cpu *4 +1 ] =cpu_to_be32 (IRQ_M_EXT ); - plic_cells[cpu *4 +2 ] =cpu_to_be32 (intc_phandles[cpu]); - plic_cells[cpu *4 +3 ] =cpu_to_be32 (IRQ_S_EXT ); +if (kvm_enabled ()) { + plic_cells[cpu *2 +0 ] =cpu_to_be32 (intc_phandles[cpu]); + plic_cells[cpu *2 +1 ] =cpu_to_be32 (IRQ_S_EXT ); + }else { + plic_cells[cpu *4 +0 ] =cpu_to_be32 (intc_phandles[cpu]); + plic_cells[cpu *4 +1 ] =cpu_to_be32 (IRQ_M_EXT ); + plic_cells[cpu *4 +2 ] =cpu_to_be32 (intc_phandles[cpu]); + plic_cells[cpu *4 +3 ] =cpu_to_be32 (IRQ_S_EXT ); + } }Quoted 8 lines
hw/riscv/virt.c
Quoted 8 lines
hw/riscv/virt.c +0
-0plic_phandles[socket] = (*phandle)++; @@ -436,10 +450,12 @@static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,
create_fdt_socket_memory(s, memmap, socket);hw/riscv/virt.c +0
-2-if (s->have_aclint) { - create_fdt_socket_aclint(s, memmap, socket,
intc_phandles);Quoted 2 lines
hw/riscv/virt.c
Quoted 2 lines
hw/riscv/virt.c +0
-2- }else { - create_fdt_socket_clint(s, memmap, socket,
intc_phandles);Quoted 3 lines
hw/riscv/virt.c static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,
Quoted 3 lines
hw/riscv/virt.c +3
-0+if (!kvm_enabled ()) { +if (s->have_aclint) { + create_fdt_socket_aclint(s, memmap, socket,
intc_phandles);Quoted 2 lines
hw/riscv/virt.c
Quoted 2 lines
hw/riscv/virt.c +2
-0+ }else { + create_fdt_socket_clint(s, memmap, socket,
intc_phandles);Quoted 11 lines
hw/riscv/virt.c
Quoted 11 lines
hw/riscv/virt.c +1
-4+ } } create_fdt_socket_plic(s, memmap, socket, phandle, @@ -801,23 +817,25 @@ static void virt_machine_init(MachineState *machine) hart_count, &error_abort);sysbus_realize (SYS_BUS_DEVICE (&s->soc[i]), &error_abort); -/* Per-socket CLINT */ -riscv_aclint_swi_create ( - memmap[VIRT_CLINT].base + i *memmap[VIRT_CLINT].size,
Quoted 3 lines
hw/riscv/virt.c static void virt_machine_init(MachineState *machine)
Quoted 3 lines
hw/riscv/virt.c +0
-3- base_hartid, hart_count, false); -riscv_aclint_mtimer_create ( - memmap[VIRT_CLINT].base + i *memmap[VIRT_CLINT].size +
Quoted 2 lines
hw/riscv/virt.c
Quoted 2 lines
hw/riscv/virt.c +0
-2-RISCV_ACLINT_SWI_SIZE , - RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid,
hart_count,Quoted 1 line
Quoted 1 line
-
RISCV_ACLINT_DEFAULT_MTIMECMP,
RISCV_ACLINT_DEFAULT_MTIME,Quoted 11 lines
hw/riscv/virt.c static void virt_machine_init(MachineState *machine)
Quoted 11 lines
hw/riscv/virt.c +3
-7- RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true); - -/* Per-socket ACLINT SSWI */ -if (s->have_aclint) { + if (!kvm_enabled()) { + /* Per-socket CLINT */ riscv_aclint_swi_create( - memmap[VIRT_ACLINT_SSWI ].base + - i * memmap[VIRT_ACLINT_SSWI ].size, - base_hartid, hart_count, true); + memmap[VIRT_CLINT].base + i *memmap[VIRT_CLINT].size,
Quoted 3 lines
hw/riscv/virt.c static void virt_machine_init(MachineState *machine)
Quoted 3 lines
hw/riscv/virt.c +3
-0+ base_hartid, hart_count, false); +riscv_aclint_mtimer_create ( + memmap[VIRT_CLINT].base + i *memmap[VIRT_CLINT].size +
Quoted 2 lines
hw/riscv/virt.c
Quoted 2 lines
hw/riscv/virt.c +2
-0+RISCV_ACLINT_SWI_SIZE , + RISCV_ACLINT_DEFAULT_MTIMER_SIZE, base_hartid,
hart_count,Quoted 1 line
Quoted 1 line
+
RISCV_ACLINT_DEFAULT_MTIMECMP,
RISCV_ACLINT_DEFAULT_MTIME,Quoted 11 lines
Quoted 11 lines
hw/riscv/virt.c +9
-0+ RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, true); + +/* Per-socket ACLINT SSWI */ +if (s->have_aclint) { +riscv_aclint_swi_create ( + memmap[VIRT_ACLINT_SSWI ].base + + i * memmap[VIRT_ACLINT_SSWI ].size, + base_hartid, hart_count, true); + } }Quoted 3 lines
Quoted 3 lines
/* Per-socket PLIC hart topology configuration string */ @@-884,6 +902,16 @@ static void
virt_machine_init(MachineState *machine)
memory_region_add_subregion(system_memory,memmap[VIRT_MROM].base,
Quoted 15 lines
Quoted 5 lines
mask_rom);hw/riscv/virt.c +3
-0+/* +* Only direct boot kernel is currently supported for KVM VM, +* so the "-bios" parameter is ignored and treated like "-bios none" hw/riscv/virt.c static void virt_machine_init(MachineState *machine)
Quoted 10 lines
hw/riscv/virt.c +7
-0+* when KVM is enabled. +*/ +if (kvm_enabled ()) { +g_free (machine->firmware); + machine->firmware =g_strdup ("none" ); + } +if (riscv_is_32bit (&s->soc[0 ])) { firmware_end_addr = riscv_find_and_load_firmware(machine, RISCV32_BIOS_BIN,
start_addr,Quoted 1 line
Quoted 1 line
NULL); @@ -941,6 +969,15 @@ static void
virt_machine_init(MachineState *machine)Quoted 1 line
Quoted 1 line
virt_memmap[VIRT_MROM].size,
kernel_entry,Quoted 14 lines
Quoted 14 lines
fdt_load_addr, machine->fdt);hw/riscv/virt.c +9
-0+/* +* Only direct boot kernel is currently supported for KVM VM, +* So here setup kernel start address and fdt address. +* TODO:Support firmware loading and integrate to TCG start +*/ +if (kvm_enabled ()) { +riscv_setup_direct_kernel (kernel_entry, fdt_load_addr); + } +/* SiFive Test MMIO device */ sifive_test_create (memmap[VIRT_TEST ].base);Quoted 5 lines
include/hw/riscv/boot.h void riscv_rom_copy_firmware_info(MachineState
Quoted 5 lines
include/hw/riscv/boot.h index renamed from include/hw/riscv/boot.h +0
-0
baff11dd8a..5834c234aa100644include/hw/riscv/boot.h +0
-0@@ -58,5 +58,6 @@ void riscv_rom_copy_firmware_info(MachineState*machine, hwaddr
rom_base,Quoted 168 lines
Quoted 163 lines
include/hw/riscv/boot.h +2
-0hwaddr rom_size, uint32_t reset_vec_size, uint64_t kernel_entry); +void riscv_setup_direct_kernel (hwaddr kernel_addr,hwaddr +fdt_addr);#endif /* RISCV_BOOT_H */ target/riscv/cpu.c index renamed from target/riscv/cpu.c +0
-0
f812998123..1c944872a3100644target/riscv/cpu.c +8
-0@@ -29,6 +29,8 @@#include "hw/qdev-properties.h" #include "migration/vmstate.h" #include "fpu/softfloat-helpers.h" +#include "sysemu/kvm.h" +#include "kvm_riscv.h" /* RISC-V CPU definitions */ @@ -380,6 +382,12 @@ static void riscv_cpu_reset(DeviceState *dev) cs->exception_index =RISCV_EXCP_NONE ; env->load_res =-1 ;set_default_nan_mode (1 , &env->fp_status); + +#ifndef CONFIG_USER_ONLY +if (kvm_enabled ()) { +kvm_riscv_reset_vcpu (cpu); + } +#endif } static void riscv_cpu_disas_set_info(CPUState *s, disassemble_info*info) diff --git
a/target/riscv/cpu.hb/target/riscv/cpu.hindex0760c0af93..2807eb1bcb100644target/riscv/cpu.h +3
-0@@ -255,6 +255,9 @@ struct CPURISCVState {/* Fields from here on are preserved across CPU reset. */ QEMUTimer *timer;/* Internal timer */ + +hwaddr kernel_addr; +hwaddr fdt_addr; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass, diff --git
a/target/riscv/kvm-stub.cb/target/riscv/kvm-stub.cnew file mode 100644 index 0000000000..39b96fe3f4target/riscv/kvm-stub.c new file +30
-0@@ -0,0 +1,25 @@ +/* +* QEMU KVM RISC-V specific function stubs +* +* Copyright (c) 2020 Huawei Technologies Co., Ltd +* +* This program is free software; you can redistribute it and/or +modify it +* under the terms and conditions of the GNU General Public +License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but +WITHOUT +* ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public +License for +* more details. +* +* You should have received a copy of the GNU General Public +License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ +#include "qemu/osdep.h" +#include "cpu.h" +#include "kvm_riscv.h" + +void kvm_riscv_reset_vcpu (RISCVCPU *cpu) { +abort (); +}target/riscv/kvm.c index renamed from target/riscv/kvm.c +0
-0
e695b91dc7..db6d8a5b6e100644target/riscv/kvm.c +1
-0@@ -37,6 +37,7 @@#include "hw/irq.h" #include "qemu/log.h" #include "hw/loader.h" +#include "kvm_riscv.h" static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type,
uint64_tidx) { @@ -369,6 +370,19 @@ intkvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)return 0;}
target/riscv/kvm.c +12
-0+void kvm_riscv_reset_vcpu (RISCVCPU *cpu) { +CPURISCVState *env = &cpu->env; + +if (!kvm_enabled ()) { +return ; + } + env->pc = cpu->env.kernel_addr; + env->gpr[10 ] =kvm_arch_vcpu_id (CPU (cpu));/* a0 */ + env->gpr[11 ] = cpu->env.fdt_addr;/* a1 */ + env->satp =0 ; +} +bool kvm_arch_cpu_check_are_resettable (void ) {return true;target/riscv/kvm_riscv.h new renamed from target/riscv/kvm_riscv.h +0
-0file mode 100644 index 0000000000..f38c82bf59
target/riscv/kvm_riscv.h new file +30
-0@@ -0,0 +1,24 @@ +/* +* QEMU KVM support -- RISC-V specific functions. +* +* Copyright (c) 2020 Huawei Technologies Co., Ltd +* +* This program is free software; you can redistribute it and/or +modify it +* under the terms and conditions of the GNU General Public +License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but +WITHOUT +* ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public +License for +* more details. +* +* You should have received a copy of the GNU General Public +License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#ifndef QEMU_KVM_RISCV_H +#define QEMU_KVM_RISCV_H + +void kvm_riscv_reset_vcpu (RISCVCPU *cpu); + +#endif target/riscv/meson.build +2
-2@@ -19,7 +19,7 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', )) -riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c')) +riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: +files('kvm-stub.c')) riscv_softmmu_ss = ss.source_set() riscv_softmmu_ss.add(files( --Quoted 2 lines
2.19.1
Regards,
Anup
PATCH v3 7/12 target/riscv: Support setting external interrupt by KVM
In reply to Yifei JiangWhen KVM is enabled, set the S-mode external interrupt through kvm_riscv_set_irq function.
--- target/riscv/cpu.c | 6 +++++- target/riscv/kvm-stub.c | 5 +++++ target/riscv/kvm.c | 17 +++++++++++++++++ target/riscv/kvm_riscv.h | 1 + 4 files changed, 28 insertions(+), 1 deletion(-)
target/riscv/cpu.c +5 -1
@@ -603,7 +603,11 @@ static void riscv_cpu_set_irq(void *opaque, int irq, int level)case IRQ_S_EXT :case IRQ_VS_EXT :case IRQ_M_EXT : -riscv_cpu_update_mip (cpu,1 << irq,BOOL_TO_MASK (level)); +if (kvm_enabled ()) { +kvm_riscv_set_irq (cpu, irq, level); + }else { +riscv_cpu_update_mip (cpu,1 << irq,BOOL_TO_MASK (level)); + }break ;default :g_assert_not_reached ();
target/riscv/kvm-stub.c +5 -0
@@ -23,3 +23,8 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu) {abort (); } + +void kvm_riscv_set_irq (RISCVCPU *cpu,int irq,int level) +{ +abort (); +}
target/riscv/kvm.c +17 -0
@@ -383,6 +383,23 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu) env->satp =0 ; } +void kvm_riscv_set_irq (RISCVCPU *cpu,int irq,int level) +{ +int ret; +unsigned virq = level ?KVM_INTERRUPT_SET :KVM_INTERRUPT_UNSET ; + +if (irq !=IRQ_S_EXT ) { +perror ("kvm riscv set irq != IRQ_S_EXT\n" ); +abort (); + } + + ret =kvm_vcpu_ioctl (CPU (cpu),KVM_INTERRUPT , &virq); +if (ret <0 ) { +perror ("Set irq failed" ); +abort (); + } +} +bool kvm_arch_cpu_check_are_resettable (void ) {return true;
target/riscv/kvm_riscv.h +1 -0
@@ -20,5 +20,6 @@#define QEMU_KVM_RISCV_H void kvm_riscv_reset_vcpu (RISCVCPU *cpu); +void kvm_riscv_set_irq (RISCVCPU *cpu,int irq,int level);#endif
Signature · 1 line
2.19.1
Re: [PATCH v3 07/12] target/riscv: Support setting external interrupt by KVM
Quoted 7 lines · Yifei Jiang
When KVM is enabled, set the S-mode external interrupt through
kvm_riscv_set_irqfunction.Signed-off-by Yifei Jiang <[email protected]>Signed-off-by Mingwang Li <[email protected]>Reviewed-by Alistair Francis <[email protected]>
Looks good to me.
Regards,
Anup
Quoted 79 lines
--- target/riscv/cpu.c | 6 +++++- target/riscv/kvm-stub.c | 5 +++++ target/riscv/kvm.c | 17 +++++++++++++++++ target/riscv/kvm_riscv.h | 1 + 4 files changed, 28 insertions(+), 1 deletion(-)target/riscv/cpu.c +5
-1@@ -603,7 +603,11 @@ static void riscv_cpu_set_irq(void *opaque, int irq, int level)case IRQ_S_EXT :case IRQ_VS_EXT :case IRQ_M_EXT : -riscv_cpu_update_mip (cpu,1 << irq,BOOL_TO_MASK (level)); +if (kvm_enabled ()) { +kvm_riscv_set_irq (cpu, irq, level); + }else { +riscv_cpu_update_mip (cpu,1 << irq,BOOL_TO_MASK (level)); + }break ;default :g_assert_not_reached ();target/riscv/kvm-stub.c +5
-0@@ -23,3 +23,8 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu) {abort (); } + +void kvm_riscv_set_irq (RISCVCPU *cpu,int irq,int level) +{ +abort (); +}target/riscv/kvm.c +17
-0@@ -383,6 +383,23 @@ void kvm_riscv_reset_vcpu(RISCVCPU *cpu) env->satp =0 ; } +void kvm_riscv_set_irq (RISCVCPU *cpu,int irq,int level) +{ +int ret; +unsigned virq = level ?KVM_INTERRUPT_SET :KVM_INTERRUPT_UNSET ; + +if (irq !=IRQ_S_EXT ) { +perror ("kvm riscv set irq != IRQ_S_EXT\n" ); +abort (); + } + + ret =kvm_vcpu_ioctl (CPU (cpu),KVM_INTERRUPT , &virq); +if (ret <0 ) { +perror ("Set irq failed" ); +abort (); + } +} +bool kvm_arch_cpu_check_are_resettable (void ) {return true;target/riscv/kvm_riscv.h +1
-1@@ -20,5 +20,6 @@#define QEMU_KVM_RISCV_H void kvm_riscv_reset_vcpu (RISCVCPU *cpu); +void kvm_riscv_set_irq (RISCVCPU *cpu,int irq,int level);#endif --2.19.1
PATCH v3 8/12 target/riscv: Handle KVM_EXIT_RISCV_SBI exit
In reply to Yifei JiangUse char-fe to handle console sbi call, which implement early console io while apply 'earlycon=sbi' into kernel parameters.
--- target/riscv/kvm.c | 43 +++++++++++++++++- target/riscv/sbi_ecall_interface.h | 72 ++++++++++++++++++++++++++++++ 2 files changed, 114 insertions(+), 1 deletion(-) create mode 100644 target/riscv/sbi_ecall_interface.h
target/riscv/kvm.c +42 -1
@@ -38,6 +38,9 @@#include "qemu/log.h" #include "hw/loader.h" #include "kvm_riscv.h" +#include "sbi_ecall_interface.h" +#include "chardev/char-fe.h" +#include "semihosting/console.h" static uint64_t kvm_riscv_reg_id (CPURISCVState *env,uint64_t type,uint64_t idx) {
@@ -365,9 +368,47 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cs)return true; } +static int kvm_riscv_handle_sbi (CPUState *cs,struct kvm_run *run) +{ +int ret =0 ; +unsigned char ch; +switch (run->riscv_sbi.extension_id) { +case SBI_EXT_0_1_CONSOLE_PUTCHAR : + ch = run->riscv_sbi.args[0 ]; +qemu_semihosting_log_out ((const char *)&ch,sizeof (ch)); +break ; +case SBI_EXT_0_1_CONSOLE_GETCHAR : + ret =qemu_chr_fe_read_all (serial_hd (0 )->be, &ch,sizeof (ch)); +if (ret ==sizeof (ch)) { + run->riscv_sbi.args[0 ] = ch; + }else { + run->riscv_sbi.args[0 ] =-1 ; + } +break ; +default : +qemu_log_mask (LOG_UNIMP , +"%s: un-handled SBI EXIT, specific reasons is %lu\n" , + __func__, run->riscv_sbi.extension_id); + ret =-1 ; +break ; + } +return ret; +} +int kvm_arch_handle_exit (CPUState *cs,struct kvm_run *run) { -return 0 ; +int ret =0 ; +switch (run->exit_reason) { +case KVM_EXIT_RISCV_SBI : + ret =kvm_riscv_handle_sbi (cs, run); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s: un-handled exit reason %d\n" , + __func__, run->exit_reason); + ret =-1 ; +break ; + } +return ret; }void kvm_riscv_reset_vcpu (RISCVCPU *cpu)
target/riscv/sbi_ecall_interface.h new file +72 -0
72 lines not on this page · raw message
Signature · 1 line
2.19.1
Re: [PATCH v3 08/12] target/riscv: Handle KVM_EXIT_RISCV_SBI exit
Quoted 29 lines · Yifei Jiang via
Use char-fe to handle console sbi call, which implement early console io while apply 'earlycon=sbi' into kernel parameters.
Signed-off-by Yifei Jiang <[email protected]>Signed-off-by Mingwang Li <[email protected]>Reviewed-by Anup Patel <[email protected]>--- target/riscv/kvm.c | 43 +++++++++++++++++- target/riscv/sbi_ecall_interface.h | 72 ++++++++++++++++++++++++++++++ 2 files changed, 114 insertions(+), 1 deletion(-) create mode 100644 target/riscv/sbi_ecall_interface.htarget/riscv/kvm.c +3
-0@@ -38,6 +38,9 @@#include "qemu/log.h" #include "hw/loader.h" #include "kvm_riscv.h" +#include "sbi_ecall_interface.h" +#include "chardev/char-fe.h" +#include "semihosting/console.h" static uint64_t kvm_riscv_reg_id (CPURISCVState *env,uint64_t type,uint64_t idx) {@@ -365,9 +368,47 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cs)return true;
target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs) · line 378
Quoted 10 lines
target/riscv/kvm.c +8
-0} +static int kvm_riscv_handle_sbi (CPUState *cs,struct kvm_run *run) +{ +int ret =0 ; +unsigned char ch; +switch (run->riscv_sbi.extension_id) { +case SBI_EXT_0_1_CONSOLE_PUTCHAR : + ch = run->riscv_sbi.args[0 ]; +qemu_semihosting_log_out ((const char *)&ch,sizeof (ch));
Hmmm... We print to the semihosting
target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs) · line 381
Quoted 3 lines
target/riscv/kvm.c +3
-0+break ; +case SBI_EXT_0_1_CONSOLE_GETCHAR : + ret =qemu_chr_fe_read_all (serial_hd (0 )->be, &ch,sizeof (ch));
but then read from the first serial device.
That seems a little strange. Would it be better to just print to serial as well?
Alistair
Quoted 116 lines
target/riscv/kvm.c +28
-1+if (ret ==sizeof (ch)) { + run->riscv_sbi.args[0 ] = ch; + }else { + run->riscv_sbi.args[0 ] =-1 ; + } +break ; +default : +qemu_log_mask (LOG_UNIMP , +"%s: un-handled SBI EXIT, specific reasons is %lu\n" , + __func__, run->riscv_sbi.extension_id); + ret =-1 ; +break ; + } +return ret; +} +int kvm_arch_handle_exit (CPUState *cs,struct kvm_run *run) { -return 0 ; +int ret =0 ; +switch (run->exit_reason) { +case KVM_EXIT_RISCV_SBI : + ret =kvm_riscv_handle_sbi (cs, run); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s: un-handled exit reason %d\n" , + __func__, run->exit_reason); + ret =-1 ; +break ; + } +return ret; }void kvm_riscv_reset_vcpu (RISCVCPU *cpu)target/riscv/sbi_ecall_interface.h new file +72
-1@@ -0,0 +1,72 @@ +/* +* SPDX-License-Identifier: BSD-2-Clause +* +* Copyright (c) 2019 Western Digital Corporation or its affiliates. +* +* Authors: +* Anup Patel <[email protected]> +*/ + +#ifndef __SBI_ECALL_INTERFACE_H__ +#define __SBI_ECALL_INTERFACE_H__ + +/* clang-format off */ + +/* SBI Extension IDs */ +#define SBI_EXT_0_1_SET_TIMER 0x0 +#define SBI_EXT_0_1_CONSOLE_PUTCHAR 0x1 +#define SBI_EXT_0_1_CONSOLE_GETCHAR 0x2 +#define SBI_EXT_0_1_CLEAR_IPI 0x3 +#define SBI_EXT_0_1_SEND_IPI 0x4 +#define SBI_EXT_0_1_REMOTE_FENCE_I 0x5 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA 0x6 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID 0x7 +#define SBI_EXT_0_1_SHUTDOWN 0x8 +#define SBI_EXT_BASE 0x10 +#define SBI_EXT_TIME 0x54494D45 +#define SBI_EXT_IPI 0x735049 +#define SBI_EXT_RFENCE 0x52464E43 +#define SBI_EXT_HSM 0x48534D + +/* SBI function IDs for BASE extension*/ +#define SBI_EXT_BASE_GET_SPEC_VERSION 0x0 +#define SBI_EXT_BASE_GET_IMP_ID 0x1 +#define SBI_EXT_BASE_GET_IMP_VERSION 0x2 +#define SBI_EXT_BASE_PROBE_EXT 0x3 +#define SBI_EXT_BASE_GET_MVENDORID 0x4 +#define SBI_EXT_BASE_GET_MARCHID 0x5 +#define SBI_EXT_BASE_GET_MIMPID 0x6 + +/* SBI function IDs for TIME extension*/ +#define SBI_EXT_TIME_SET_TIMER 0x0 + +/* SBI function IDs for IPI extension*/ +#define SBI_EXT_IPI_SEND_IPI 0x0 + +/* SBI function IDs for RFENCE extension*/ +#define SBI_EXT_RFENCE_REMOTE_FENCE_I 0x0 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA 0x1 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID 0x2 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA 0x3 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID 0x4 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA 0x5 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID 0x6 + +/* SBI function IDs for HSM extension */ +#define SBI_EXT_HSM_HART_START 0x0 +#define SBI_EXT_HSM_HART_STOP 0x1 +#define SBI_EXT_HSM_HART_GET_STATUS 0x2 + +#define SBI_HSM_HART_STATUS_STARTED 0x0 +#define SBI_HSM_HART_STATUS_STOPPED 0x1 +#define SBI_HSM_HART_STATUS_START_PENDING 0x2 +#define SBI_HSM_HART_STATUS_STOP_PENDING 0x3 + +#define SBI_SPEC_VERSION_MAJOR_OFFSET 24 +#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f +#define SBI_SPEC_VERSION_MINOR_MASK 0xffffff +#define SBI_EXT_VENDOR_START 0x09000000 +#define SBI_EXT_VENDOR_END 0x09FFFFFF +/* clang-format on */ + +#endif --2.19.1
RE: [PATCH v3 08/12] target/riscv: Handle KVM_EXIT_RISCV_SBI exit
Quoted 65 lines
-----Original Message-----
From: Alistair Francis [mailto:alistair23 at gmail.com]
Sent: Thursday, January 6, 2022 6:04 AM
To: Jiangyifei <[email protected]>
Cc: qemu-devel at nongnu.org Developers <[email protected]>; open list:RISC-V <[email protected]>; kvm-riscv at lists.infradead.org; open list:Overall <[email protected]>; libvir-list at redhat.com; Anup Patel <[email protected]>; Palmer Dabbelt <[email protected]>; Alistair Francis <[email protected]>; Bin Meng <[email protected]>; Fanliang (EulerOS) <[email protected]>; Wubin (H) <[email protected]>; Wanghaibin (D) <[email protected]>; wanbo (G) <[email protected]>; limingwang (A) <[email protected]>
Subject: Re: [PATCH v3 08/12] target/riscv: HandleKVM_EXIT_RISCV_SBIexitQuoted 32 lines · Yifei Jiang via
Use char-fe to handle console sbi call, which implement early console io while apply 'earlycon=sbi' into kernel parameters.
Signed-off-by Yifei Jiang <[email protected]>Signed-off-by Mingwang Li <[email protected]>Reviewed-by Anup Patel <[email protected]>--- target/riscv/kvm.c | 43 +++++++++++++++++- target/riscv/sbi_ecall_interface.h | 72 ++++++++++++++++++++++++++++++ 2 files changed, 114 insertions(+), 1 deletion(-) create mode 100644
target/riscv/sbi_ecall_interface.htarget/riscv/kvm.c index renamed from target/riscv/kvm.c +0
-0
0027f11f45..4d08669c81100644target/riscv/kvm.c +3
-0@@ -38,6 +38,9 @@#include "qemu/log.h" #include "hw/loader.h" #include "kvm_riscv.h" +#include "sbi_ecall_interface.h" +#include "chardev/char-fe.h" +#include "semihosting/console.h" static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type,
uint64_tidx) { @@ -365,9 +368,47 @@ boolkvm_arch_stop_on_emulation_error(CPUState *cs)return true;}
target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs) · line 378
Quoted 7 lines
target/riscv/kvm.c +7
-0+static int kvm_riscv_handle_sbi (CPUState *cs,struct kvm_run *run) { +int ret =0 ; +unsigned char ch; +switch (run->riscv_sbi.extension_id) { +case SBI_EXT_0_1_CONSOLE_PUTCHAR : + ch = run->riscv_sbi.args[0 ]; +qemu_semihosting_log_out ((const char *)&ch,sizeof (ch));Hmmm... We print to the semihosting
target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs)
Quoted 4 lines
target/riscv/kvm.c +4
-0+break ; +case SBI_EXT_0_1_CONSOLE_GETCHAR : + ret =qemu_chr_fe_read_all (serial_hd (0 )->be, &ch, +sizeof (ch));
Quoted 6 lines
but then read from the first serial device.
That seems a little strange. Would it be better to just print to serial as well?
Alistair
Hm...I also think so. It will be modified in the next series.
Yifei
Quoted 117 lines
target/riscv/kvm.c bool kvm_arch_stop_on_emulation_error(CPUState *cs)
Quoted 9 lines
target/riscv/kvm.c +9
-0+if (ret ==sizeof (ch)) { + run->riscv_sbi.args[0 ] = ch; + }else { + run->riscv_sbi.args[0 ] =-1 ; + } +break ; +default : + qemu_log_mask(LOG_UNIMP, + "%s: un-handled SBI EXIT, specific reasonsis %lu\n",
Quoted 107 lines
target/riscv/kvm.c +19
-1+ __func__, run->riscv_sbi.extension_id); + ret =-1 ; +break ; + } +return ret; +} +int kvm_arch_handle_exit (CPUState *cs,struct kvm_run *run) { -return 0 ; +int ret =0 ; +switch (run->exit_reason) { +case KVM_EXIT_RISCV_SBI : + ret =kvm_riscv_handle_sbi (cs, run); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s: un-handled exit reason %d\n" , + __func__, run->exit_reason); + ret =-1 ; +break ; + } +return ret; } void kvm_riscv_reset_vcpu(RISCVCPU *cpu) diff --git
a/target/riscv/sbi_ecall_interface.hb/target/riscv/sbi_ecall_interface.h
new file mode 100644
index 0000000000..fb1a3fa8f2target/riscv/sbi_ecall_interface.h new file +72
-1@@ -0,0 +1,72 @@ +/* +* SPDX-License-Identifier: BSD-2-Clause +* +* Copyright (c) 2019 Western Digital Corporation or its affiliates. +* +* Authors: +* Anup Patel <[email protected]> +*/ + +#ifndef __SBI_ECALL_INTERFACE_H__ +#define __SBI_ECALL_INTERFACE_H__ + +/* clang-format off */ + +/* SBI Extension IDs */ +#define SBI_EXT_0_1_SET_TIMER 0x0 +#define SBI_EXT_0_1_CONSOLE_PUTCHAR 0x1 +#define SBI_EXT_0_1_CONSOLE_GETCHAR 0x2 +#define SBI_EXT_0_1_CLEAR_IPI 0x3 +#define SBI_EXT_0_1_SEND_IPI 0x4 +#define SBI_EXT_0_1_REMOTE_FENCE_I 0x5 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA 0x6 +#define SBI_EXT_0_1_REMOTE_SFENCE_VMA_ASID 0x7 +#define SBI_EXT_0_1_SHUTDOWN 0x8 +#define SBI_EXT_BASE 0x10 +#define SBI_EXT_TIME 0x54494D45 +#define SBI_EXT_IPI 0x735049 +#define SBI_EXT_RFENCE 0x52464E43 +#define SBI_EXT_HSM 0x48534D + +/* SBI function IDs for BASE extension*/ +#define SBI_EXT_BASE_GET_SPEC_VERSION 0x0 +#define SBI_EXT_BASE_GET_IMP_ID 0x1 +#define SBI_EXT_BASE_GET_IMP_VERSION 0x2 +#define SBI_EXT_BASE_PROBE_EXT 0x3 +#define SBI_EXT_BASE_GET_MVENDORID 0x4 +#define SBI_EXT_BASE_GET_MARCHID 0x5 +#define SBI_EXT_BASE_GET_MIMPID 0x6 + +/* SBI function IDs for TIME extension*/ +#define SBI_EXT_TIME_SET_TIMER 0x0 + +/* SBI function IDs for IPI extension*/ +#define SBI_EXT_IPI_SEND_IPI 0x0 + +/* SBI function IDs for RFENCE extension*/ +#define SBI_EXT_RFENCE_REMOTE_FENCE_I 0x0 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA 0x1 +#define SBI_EXT_RFENCE_REMOTE_SFENCE_VMA_ASID 0x2 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA 0x3 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_GVMA_VMID 0x4 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA 0x5 +#define SBI_EXT_RFENCE_REMOTE_HFENCE_VVMA_ASID 0x6 + +/* SBI function IDs for HSM extension */ +#define SBI_EXT_HSM_HART_START 0x0 +#define SBI_EXT_HSM_HART_STOP 0x1 +#define SBI_EXT_HSM_HART_GET_STATUS 0x2 + +#define SBI_HSM_HART_STATUS_STARTED 0x0 +#define SBI_HSM_HART_STATUS_STOPPED 0x1 +#define SBI_HSM_HART_STATUS_START_PENDING 0x2 +#define SBI_HSM_HART_STATUS_STOP_PENDING 0x3 + +#define SBI_SPEC_VERSION_MAJOR_OFFSET 24 +#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f +#define SBI_SPEC_VERSION_MINOR_MASK 0xffffff +#define SBI_EXT_VENDOR_START 0x09000000 +#define SBI_EXT_VENDOR_END 0x09FFFFFF +/* clang-format on */ + +#endif --2.19.1
PATCH v3 9/12 target/riscv: Add host cpu type
In reply to Yifei Jiang'host' type cpu is set isa to RV32 or RV64 simply, more isa info will obtain from KVM in kvm_arch_init_vcpu()
--- target/riscv/cpu.c | 15 +++++++++++++++ target/riscv/cpu.h | 1 + 2 files changed, 16 insertions(+)
target/riscv/cpu.c +15 -0
@@ -213,6 +213,18 @@ static void rv32_imafcu_nommu_cpu_init(Object *obj) }#endif +#if defined(CONFIG_KVM ) +static void riscv_host_cpu_init (Object *obj) +{ +CPURISCVState *env = &RISCV_CPU (obj)->env; +#if defined(TARGET_RISCV32 ) +set_misa (env,MXL_RV32 ,0 ); +#elif defined(TARGET_RISCV64 ) +set_misa (env,MXL_RV64 ,0 ); +#endif +} +#endif +static ObjectClass *riscv_cpu_class_by_name (const char *cpu_model) {ObjectClass *oc;
@@ -814,6 +826,9 @@ static const TypeInfo riscv_cpu_type_infos[] = { .class_init = riscv_cpu_class_init, }, DEFINE_CPU(TYPE_RISCV_CPU_ANY, riscv_any_cpu_init), +#if defined(CONFIG_KVM ) + DEFINE_CPU(TYPE_RISCV_CPU_HOST, riscv_host_cpu_init), +#endif #if defined(TARGET_RISCV32 )DEFINE_CPU (TYPE_RISCV_CPU_BASE32 , rv32_base_cpu_init), DEFINE_CPU(TYPE_RISCV_CPU_IBEX, rv32_ibex_cpu_init),
target/riscv/cpu.h +1 -0
@@ -45,6 +45,7 @@#define TYPE_RISCV_CPU_SIFIVE_E51 RISCV_CPU_TYPE_NAME("sifive-e51")#define TYPE_RISCV_CPU_SIFIVE_U34 RISCV_CPU_TYPE_NAME("sifive-u34")#define TYPE_RISCV_CPU_SIFIVE_U54 RISCV_CPU_TYPE_NAME("sifive-u54") +#define TYPE_RISCV_CPU_HOST RISCV_CPU_TYPE_NAME("host")#if defined(TARGET_RISCV32 )# define TYPE_RISCV_CPU_BASE TYPE_RISCV_CPU_BASE32
Signature · 1 line
2.19.1
PATCH v3 10/12 target/riscv: Add kvm_riscv_get/put_regs_timer
In reply to Yifei JiangAdd kvm_riscv_get/put_regs_timer to synchronize virtual time context from KVM.
To set register of RISCV_TIMER_REG(state) will occur a error from KVM on kvm_timer_state == 0. It's better to adapt in KVM, but it doesn't matter that adaping in QEMU.
--- target/riscv/cpu.h | 7 +++++ target/riscv/kvm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+)
target/riscv/cpu.h +7 -0
@@ -259,6 +259,13 @@ struct CPURISCVState {hwaddr kernel_addr;hwaddr fdt_addr; + +/* kvm timer */ +bool kvm_timer_dirty; +uint64_t kvm_timer_time; +uint64_t kvm_timer_compare; +uint64_t kvm_timer_state; +uint64_t kvm_timer_frequency; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,
target/riscv/kvm.c +72 -0
96 lines not on this page · raw message
Signature · 1 line
2.19.1
Re: [PATCH v3 10/12] target/riscv: Add kvm_riscv_get/put_regs_timer
Quoted 10 lines · Yifei Jiang
Add kvm_riscv_get/put_regs_timer to synchronize virtual time context from KVM.
To set register of
RISCV_TIMER_REG(state)will occur a error from KVM onkvm_timer_state== 0. It's better to adapt in KVM, but it doesn't matter that adaping in QEMU.Signed-off-by Yifei Jiang <[email protected]>Signed-off-by Mingwang Li <[email protected]>
Looks good to me.
Regards,
Anup
Quoted 130 lines
--- target/riscv/cpu.h | 7 +++++ target/riscv/kvm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+)target/riscv/cpu.h +7
-0@@ -259,6 +259,13 @@ struct CPURISCVState {hwaddr kernel_addr;hwaddr fdt_addr; + +/* kvm timer */ +bool kvm_timer_dirty; +uint64_t kvm_timer_time; +uint64_t kvm_timer_compare; +uint64_t kvm_timer_state; +uint64_t kvm_timer_frequency; }; OBJECT_DECLARE_TYPE(RISCVCPU, RISCVCPUClass,target/riscv/kvm.c +72
-1@@ -41,6 +41,7 @@#include "sbi_ecall_interface.h" #include "chardev/char-fe.h" #include "semihosting/console.h" +#include "migration/migration.h" static uint64_t kvm_riscv_reg_id (CPURISCVState *env,uint64_t type,uint64_t idx) {@@ -65,6 +66,9 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx#define RISCV_CSR_REG (env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_CSR, \ KVM_REG_RISCV_CSR_REG(name)) +#define RISCV_TIMER_REG (env, name) kvm_riscv_reg_id(env, KVM_REG_RISCV_TIMER, \ + KVM_REG_RISCV_TIMER_REG(name)) +#define RISCV_FP_F_REG (env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_F, idx)#define RISCV_FP_D_REG (env, idx) kvm_riscv_reg_id(env, KVM_REG_RISCV_FP_D, idx)@@ -85,6 +89,22 @@ static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx } \ } while(0) +#define KVM_RISCV_GET_TIMER (cs, env, name, reg) \ + do { \ + int ret = kvm_get_one_reg(cs, RISCV_TIMER_REG(env, name), ®); \ + if (ret) { \ + abort(); \ + } \ + } while(0) + +#define KVM_RISCV_SET_TIMER (cs, env, name, reg) \ + do { \ + int ret = kvm_set_one_reg(cs, RISCV_TIMER_REG(env, time), ®); \ + if (ret) { \ + abort(); \ + } \ + } while (0) +static int kvm_riscv_get_regs_core (CPUState *cs) {int ret =0 ;@@ -236,6 +256,58 @@ static int kvm_riscv_put_regs_fp(CPUState *cs)return ret; } +static void kvm_riscv_get_regs_timer (CPUState *cs) +{ +CPURISCVState *env = &RISCV_CPU (cs)->env; + +if (env->kvm_timer_dirty) { +return ; + } + +KVM_RISCV_GET_TIMER (cs, env, time, env->kvm_timer_time); +KVM_RISCV_GET_TIMER (cs, env, compare, env->kvm_timer_compare); +KVM_RISCV_GET_TIMER (cs, env, state, env->kvm_timer_state); +KVM_RISCV_GET_TIMER (cs, env, frequency, env->kvm_timer_frequency); + + env->kvm_timer_dirty = true; +} + +static void kvm_riscv_put_regs_timer (CPUState *cs) +{ +uint64_t reg; +CPURISCVState *env = &RISCV_CPU (cs)->env; + +if (!env->kvm_timer_dirty) { +return ; + } + +KVM_RISCV_SET_TIMER (cs, env, time, env->kvm_timer_time); +KVM_RISCV_SET_TIMER (cs, env, compare, env->kvm_timer_compare); + +/* +* To set register of RISCV_TIMER_REG(state) will occur a error from KVM +* on env->kvm_timer_state == 0, It's better to adapt in KVM, but it +* doesn't matter that adaping in QEMU now. +* TODO If KVM changes, adapt here. +*/ +if (env->kvm_timer_state) { +KVM_RISCV_SET_TIMER (cs, env, state, env->kvm_timer_state); + } + +/* +* For now, migration will not work between Hosts with different timer +* frequency. Therefore, we should check whether they are the same here +* during the migration. +*/ +if (migration_is_running (migrate_get_current ()->state)) { +KVM_RISCV_GET_TIMER (cs, env, frequency, reg); +if (reg != env->kvm_timer_frequency) { +error_report ("Dst Hosts timer frequency != Src Hosts" ); + } + } + + env->kvm_timer_dirty = false; +}const KVMCapabilityInfo kvm_arch_required_capabilities[] = {KVM_CAP_LAST_INFO --2.19.1
PATCH v3 11/12 target/riscv: Implement virtual time adjusting with vm state changing
In reply to Yifei JiangWe hope that virtual time adjusts with vm state changing. When a vm is stopped, guest virtual time should stop counting and kvm_timer should be stopped. When the vm is resumed, guest virtual time should continue to count and kvm_timer should be restored.
--- target/riscv/kvm.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
target/riscv/kvm.c +14 -0
@@ -42,6 +42,7 @@#include "chardev/char-fe.h" #include "semihosting/console.h" #include "migration/migration.h" +#include "sysemu/runstate.h" static uint64_t kvm_riscv_reg_id (CPURISCVState *env,uint64_t type,uint64_t idx) {
@@ -378,6 +379,17 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)return cpu->cpu_index; } +static void kvm_riscv_vm_state_change (void *opaque,bool running,RunState state) +{ +CPUState *cs = opaque; + +if (running) { +kvm_riscv_put_regs_timer (cs); + }else { +kvm_riscv_get_regs_timer (cs); + } +} +void kvm_arch_init_irq_routing (KVMState *s) { }
@@ -390,6 +402,8 @@ int kvm_arch_init_vcpu(CPUState *cs)CPURISCVState *env = &cpu->env;uint64_t id; +qemu_add_vm_change_state_handler (kvm_riscv_vm_state_change, cs); + id =kvm_riscv_reg_id (env,KVM_REG_RISCV_CONFIG ,KVM_REG_RISCV_CONFIG_REG (isa)); ret =kvm_get_one_reg (cs, id, &isa);if (ret) {
Signature · 1 line
2.19.1
Re: [PATCH v3 11/12] target/riscv: Implement virtual time adjusting with vm state changing
Quoted 9 lines · Yifei Jiang via
We hope that virtual time adjusts with vm state changing. When a vm is stopped, guest virtual time should stop counting and
kvm_timershould be stopped. When the vm is resumed, guest virtual time should continue to count andkvm_timershould be restored.Signed-off-by Yifei Jiang <[email protected]>Signed-off-by Mingwang Li <[email protected]>Reviewed-by Anup Patel <[email protected]>
Alistair
Quoted 47 lines
--- target/riscv/kvm.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)target/riscv/kvm.c +14
-1@@ -42,6 +42,7 @@#include "chardev/char-fe.h" #include "semihosting/console.h" #include "migration/migration.h" +#include "sysemu/runstate.h" static uint64_t kvm_riscv_reg_id (CPURISCVState *env,uint64_t type,uint64_t idx) {@@ -378,6 +379,17 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)return cpu->cpu_index; } +static void kvm_riscv_vm_state_change (void *opaque,bool running,RunState state) +{ +CPUState *cs = opaque; + +if (running) { +kvm_riscv_put_regs_timer (cs); + }else { +kvm_riscv_get_regs_timer (cs); + } +} +void kvm_arch_init_irq_routing (KVMState *s) { }@@ -390,6 +402,8 @@ int kvm_arch_init_vcpu(CPUState *cs)CPURISCVState *env = &cpu->env;uint64_t id; +qemu_add_vm_change_state_handler (kvm_riscv_vm_state_change, cs); + id =kvm_riscv_reg_id (env,KVM_REG_RISCV_CONFIG ,KVM_REG_RISCV_CONFIG_REG (isa)); ret =kvm_get_one_reg (cs, id, &isa);if (ret) { --2.19.1
PATCH v3 12/12 target/riscv: Support virtual time context synchronization
In reply to Yifei JiangAdd virtual time context description to vmstate_kvmtimer. After cpu being loaded, virtual time context is updated to KVM.
--- target/riscv/machine.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
target/riscv/machine.c +30 -0
@@ -164,6 +164,35 @@ static const VMStateDescription vmstate_pointermasking = { } }; +static bool kvmtimer_needed (void *opaque) +{ +return kvm_enabled (); +} + +static int cpu_post_load (void *opaque,int version_id) +{ +RISCVCPU *cpu = opaque; +CPURISCVState *env = &cpu->env; + + env->kvm_timer_dirty = true; +return 0 ; +} + +static const VMStateDescription vmstate_kvmtimer = { + .name ="cpu/kvmtimer" , + .version_id =1 , + .minimum_version_id =1 , + .needed = kvmtimer_needed, + .post_load = cpu_post_load, + .fields = (VMStateField []) { +VMSTATE_UINT64 (env.kvm_timer_time,RISCVCPU ), +VMSTATE_UINT64 (env.kvm_timer_compare,RISCVCPU ), +VMSTATE_UINT64 (env.kvm_timer_state,RISCVCPU ), + +VMSTATE_END_OF_LIST () + } +}; +const VMStateDescription vmstate_riscv_cpu = { .name ="cpu" , .version_id =3 ,
@@ -218,6 +247,7 @@ const VMStateDescription vmstate_riscv_cpu = { &vmstate_hyper, &vmstate_vector, &vmstate_pointermasking, + &vmstate_kvmtimer, NULL } };
Signature · 1 line
2.19.1
Re: [PATCH v3 12/12] target/riscv: Support virtual time context synchronization
Quoted 7 lines · Yifei Jiang via
Add virtual time context description to
vmstate_kvmtimer. After cpu being loaded, virtual time context is updated to KVM.Signed-off-by Yifei Jiang <[email protected]>Signed-off-by Mingwang Li <[email protected]>Reviewed-by Anup Patel <[email protected]>
Alistair
Quoted 56 lines
--- target/riscv/machine.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)target/riscv/machine.c +30
-1@@ -164,6 +164,35 @@ static const VMStateDescription vmstate_pointermasking = { } }; +static bool kvmtimer_needed (void *opaque) +{ +return kvm_enabled (); +} + +static int cpu_post_load (void *opaque,int version_id) +{ +RISCVCPU *cpu = opaque; +CPURISCVState *env = &cpu->env; + + env->kvm_timer_dirty = true; +return 0 ; +} + +static const VMStateDescription vmstate_kvmtimer = { + .name ="cpu/kvmtimer" , + .version_id =1 , + .minimum_version_id =1 , + .needed = kvmtimer_needed, + .post_load = cpu_post_load, + .fields = (VMStateField []) { +VMSTATE_UINT64 (env.kvm_timer_time,RISCVCPU ), +VMSTATE_UINT64 (env.kvm_timer_compare,RISCVCPU ), +VMSTATE_UINT64 (env.kvm_timer_state,RISCVCPU ), + +VMSTATE_END_OF_LIST () + } +}; +const VMStateDescription vmstate_riscv_cpu = { .name ="cpu" , .version_id =3 ,@@ -218,6 +247,7 @@ const VMStateDescription vmstate_riscv_cpu = { &vmstate_hyper, &vmstate_vector, &vmstate_pointermasking, + &vmstate_kvmtimer, NULL } }; --2.19.1