Add loongarch kvm accel support
Subsystems (2)
v4 cover Add loongarch kvm accel support
This series add loongarch kvm support, mainly implement some interfaces used by kvm such as kvm_arch_get/set_regs, kvm_arch_handle_exit, kvm_loongarch_set_interrupt, etc.
Currently, we are able to boot LoongArch KVM Linux Guests. In loongarch VM, mmio devices and iocsr devices are emulated in user space such as APIC, IPI, pci devices, etc, other hardwares such as MMU, timer and csr are emulated in kernel.
It is based on temporarily unaccepted linux kvm:
https://github.com/loongson/linux-loongarch-kvm
And We will remove the RFC flag until the linux kvm patches are merged.
The running environment of LoongArch virt machine:
1. Get the linux source by the above mentioned link.
git checkout kvm-loongarch
make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu- loongson3_defconfig make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu-
2. Get the qemu source: https://github.com/loongson/qemu
git checkout kvm-loongarch
./configure --target-list="loongarch64-softmmu" --enable-kvm
make
3. Get uefi bios of LoongArch virt machine:
4. Also you can access the binary files we have already build:
https://github.com/yangxiaojuan-loongson/qemu-binary
The command to boot loongarch virt machine:
$ qemu-system-loongarch64 -machine virt -m 4G -cpu la464 \ -smp 1 -bios QEMU_EFI.fd -kernel vmlinuz.efi -initrd ramdisk \ -serial stdio -monitor telnet:localhost:4495,server,nowait \ -append "root=/dev/ram rdinit=/sbin/init console=ttyS0,115200" \ --nographic
Changes for RFC v4:
1. Added function interfaces kvm_loongarch_get_cpucfg and kvm_loongarch_put_cpucfg for passing the value of vcpu cfg to kvm. Move the macro definition KVM_IOC_CSRID from kvm.c to kvm.h. 2.Delete the duplicate CSR_CPUID field in CPUArchState. 3.Add kvm_arch_get_default_type function in kvm.c.
4.Disable LSX,LASX in cpucfg2 in KVM. And disable LBT in cpucfg2 in KVM.
Changes for RFC v3:
1. Move the init mp_state to KVM_MP_STATE_RUNNABLE function into kvm.c.
2. Fix some unstandard code problems in kvm_get/set_regs_ioctl, such as sort loongarch to keep alphabetic ordering in meson.build, gpr[0] should be always 0, remove unnecessary inline statement, etc.
3. Rename the counter_value variable to kvm_state_counter in cpu_env, and add comments for it to explain the meaning.
Changes for RFC v2:
1. Mark the "Add KVM headers for loongarch" patch as a placeholder, as we will use the update-linux-headers.sh to generate the kvm headers when the linux loongarch KVM patch series are accepted.
2. Remove the DPRINTF macro in kvm.c and use trace events to replace it, we add some trace functions such as trace_kvm_handle_exit, trace_kvm_set_intr, trace_kvm_failed_get_csr, etc.
3. Remove the unused functions in kvm_stub.c and move stub function into the suitable patch.
Tianrui Zhao (9):
linux-headers: Add KVM headers for loongarch target/loongarch: Define some kvm_arch interfaces target/loongarch: Supplement vcpu env initial when vcpu reset target/loongarch: Implement kvm get/set registers target/loongarch: Implement kvm_arch_init function target/loongarch: Implement kvm_arch_init_vcpu target/loongarch: Implement kvm_arch_handle_exit target/loongarch: Implement set vcpu intr for kvm target/loongarch: Add loongarch kvm into meson build
linux-headers/asm-loongarch/kvm.h | 100 +++++ linux-headers/linux/kvm.h | 9 + meson.build | 3 + target/loongarch/cpu.c | 23 +- target/loongarch/cpu.h | 6 +- target/loongarch/kvm-stub.c | 11 + target/loongarch/kvm.c | 594 ++++++++++++++++++++++++++++++ target/loongarch/kvm_loongarch.h | 13 + target/loongarch/meson.build | 1 + target/loongarch/trace-events | 17 + target/loongarch/trace.h | 1 + 11 files changed, 772 insertions(+), 6 deletions(-) create mode 100644 linux-headers/asm-loongarch/kvm.h create mode 100644 target/loongarch/kvm-stub.c create mode 100644 target/loongarch/kvm.c create mode 100644 target/loongarch/kvm_loongarch.h create mode 100644 target/loongarch/trace-events create mode 100644 target/loongarch/trace.h
Signature · 1 line
2.39.1
RFC PATCH v4 1/9 linux-headers: Add KVM headers for loongarch
From: Tianrui Zhao <[email protected]>
This patch is only a placeholder now, which is used to show some kvm structures and macros for reviewers. And it will be replaced by using update-linux-headers.sh when the linux loongarch kvm patches are accepted.
--- linux-headers/asm-loongarch/kvm.h | 100 ++++++++++++++++++++++++++++++ linux-headers/linux/kvm.h | 9 +++ 2 files changed, 109 insertions(+) create mode 100644 linux-headers/asm-loongarch/kvm.h
linux-headers/asm-loongarch/kvm.h new file +100 -0
@@ -0,0 +1,100 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* +* Copyright (C) 2023 Loongson Technology Corporation Limited +*/ + +#ifndef __UAPI_ASM_LOONGARCH_KVM_H +#define __UAPI_ASM_LOONGARCH_KVM_H + +#include <linux/types.h> + +/* +* KVM Loongarch specific structures and definitions. +*/ + +#define __KVM_HAVE_READONLY_MEM + +#define KVM_COALESCED_MMIO_PAGE_OFFSET 1 + +/* +* for KVM_GET_REGS and KVM_SET_REGS +*/ +struct kvm_regs { +/* out (KVM_GET_REGS) / in (KVM_SET_REGS) */ +__u64 gpr[32 ]; +__u64 pc; +}; + +/* +* for KVM_GET_FPU and KVM_SET_FPU +*/ +struct kvm_fpu { +__u32 fcsr; +__u32 none; +__u64 fcc;/* 8x8 */ +struct kvm_fpureg { +__u64 val64[4 ]; + } fpr[32 ]; +}; + +/* +* For LoongArch, we use KVM_SET_ONE_REG and KVM_GET_ONE_REG to access various +* registers. The id field is broken down as follows: +* +* bits[63..52] - As per linux/kvm.h +* bits[51..32] - Must be zero. +* bits[31..16] - Register set. +* +* Register set = 0: GP registers from kvm_regs (see definitions below). +* +* Register set = 1: CSR registers. +* +* Register set = 2: KVM specific registers (see definitions below). +* +* Register set = 3: FPU / SIMD registers (see definitions below). +* +* Other sets registers may be added in the future. Each set would +* have its own identifier in bits[31..16]. +*/ + +#define KVM_REG_LOONGARCH_GP (KVM_REG_LOONGARCH | 0x00000ULL) +#define KVM_REG_LOONGARCH_CSR (KVM_REG_LOONGARCH | 0x10000ULL) +#define KVM_REG_LOONGARCH_KVM (KVM_REG_LOONGARCH | 0x20000ULL) +#define KVM_REG_LOONGARCH_FPU (KVM_REG_LOONGARCH | 0x30000ULL) +#define KVM_REG_LOONGARCH_CPUCFG (KVM_REG_LOONGARCH | 0x40000ULL) +#define KVM_REG_LOONGARCH_MASK (KVM_REG_LOONGARCH | 0x70000ULL) +#define KVM_CSR_IDX_MASK 0x7fff +#define KVM_CPUCFG_IDX_MASK 0x7fff + +/* +* KVM_REG_LOONGARCH_KVM - KVM specific control registers. +*/ + +#define KVM_REG_LOONGARCH_COUNTER (KVM_REG_LOONGARCH_KVM | KVM_REG_SIZE_U64 | 3) +#define KVM_REG_LOONGARCH_VCPU_RESET (KVM_REG_LOONGARCH_KVM | KVM_REG_SIZE_U64 | 4) + +#define LOONGARCH_REG_SHIFT 3 +#define LOONGARCH_REG_64 (TYPE ,REG ) (TYPE | KVM_REG_SIZE_U64 | (REG << LOONGARCH_REG_SHIFT)) +#define KVM_IOC_CSRID (REG ) LOONGARCH_REG_64(KVM_REG_LOONGARCH_CSR, REG) +#define KVM_IOC_CPUCFG (REG ) LOONGARCH_REG_64(KVM_REG_LOONGARCH_CPUCFG, REG) + +struct kvm_debug_exit_arch { +}; + +/* for KVM_SET_GUEST_DEBUG */ +struct kvm_guest_debug_arch { +}; + +/* definition of registers in kvm_run */ +struct kvm_sync_regs { +}; + +/* dummy definition */ +struct kvm_sregs { +}; + +#define KVM_NR_IRQCHIPS 1 +#define KVM_IRQCHIP_NUM_PINS 64 +#define KVM_MAX_CORES 256 + +#endif /* __UAPI_ASM_LOONGARCH_KVM_H */
linux-headers/linux/kvm.h +9 -0
@@ -264,6 +264,7 @@ struct kvm_xen_exit {#define KVM_EXIT_RISCV_SBI 35#define KVM_EXIT_RISCV_CSR 36#define KVM_EXIT_NOTIFY 37 +#define KVM_EXIT_LOONGARCH_IOCSR 38/* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */
@@ -336,6 +337,13 @@ struct kvm_run {__u32 len;__u8 is_write; } mmio; +/* KVM_EXIT_LOONGARCH_IOCSR */ +struct { +__u64 phys_addr; +__u8 data[8 ]; +__u32 len; +__u8 is_write; + } iocsr_io;/* KVM_EXIT_HYPERCALL */ struct {__u64 nr;
@@ -1358,6 +1366,7 @@ struct kvm_dirty_tlb {#define KVM_REG_ARM64 0x6000000000000000ULL#define KVM_REG_MIPS 0x7000000000000000ULL#define KVM_REG_RISCV 0x8000000000000000ULL +#define KVM_REG_LOONGARCH 0x9000000000000000ULL#define KVM_REG_SIZE_SHIFT 52#define KVM_REG_SIZE_MASK 0x00f0000000000000ULL
Signature · 1 line
2.39.1
RFC PATCH v4 2/9 target/loongarch: Define some kvm_arch interfaces
In reply to xianglai liFrom: Tianrui Zhao <[email protected]>
Define some functions in target/loongarch/kvm.c, such as kvm_arch_put_registers, kvm_arch_get_registers and kvm_arch_handle_exit, etc. which are needed by kvm/kvm-all.c. Now the most functions has no content and they will be implemented in the next patches.
--- target/loongarch/kvm.c | 131 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 target/loongarch/kvm.c
target/loongarch/kvm.c new file +131 -0
@@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* QEMU LoongArch KVM +* +* Copyright (c) 2023 Loongson Technology Corporation Limited +*/ + +#include "qemu/osdep.h" +#include <sys/ioctl.h> +#include <linux/kvm.h> + +#include "qemu/timer.h" +#include "qemu/error-report.h" +#include "qemu/main-loop.h" +#include "sysemu/sysemu.h" +#include "sysemu/kvm.h" +#include "sysemu/kvm_int.h" +#include "hw/pci/pci.h" +#include "exec/memattrs.h" +#include "exec/address-spaces.h" +#include "hw/boards.h" +#include "hw/irq.h" +#include "qemu/log.h" +#include "hw/loader.h" +#include "migration/migration.h" +#include "sysemu/runstate.h" +#include "cpu-csr.h" +#include "kvm_loongarch.h" + +static bool cap_has_mp_state; +const KVMCapabilityInfo kvm_arch_required_capabilities[] = { +KVM_CAP_LAST_INFO +}; + +int kvm_arch_get_registers (CPUState *cs) +{ +return 0 ; +} +int kvm_arch_put_registers (CPUState *cs,int level) +{ +return 0 ; +} + +int kvm_arch_init_vcpu (CPUState *cs) +{ +return 0 ; +} + +int kvm_arch_destroy_vcpu (CPUState *cs) +{ +return 0 ; +} + +unsigned long kvm_arch_vcpu_id (CPUState *cs) +{ +return cs->cpu_index; +} + +int kvm_arch_release_virq_post (int virq) +{ +return 0 ; +} + +int kvm_arch_msi_data_to_gsi (uint32_t data) +{ +abort (); +} + +int kvm_arch_fixup_msi_route (struct kvm_irq_routing_entry *route, +uint64_t address,uint32_t data,PCIDevice *dev) +{ +return 0 ; +} + +int kvm_arch_add_msi_route_post (struct kvm_irq_routing_entry *route, +int vector,PCIDevice *dev) +{ +return 0 ; +} + +void kvm_arch_init_irq_routing (KVMState *s) +{ +} + +int kvm_arch_get_default_type (MachineState *ms) +{ +return 0 ; +} + +int kvm_arch_init (MachineState *ms,KVMState *s) +{ +return 0 ; +} + +int kvm_arch_irqchip_create (KVMState *s) +{ +return 0 ; +} + +void kvm_arch_pre_run (CPUState *cs,struct kvm_run *run) +{ +} + +MemTxAttrs kvm_arch_post_run (CPUState *cs,struct kvm_run *run) +{ +return MEMTXATTRS_UNSPECIFIED ; +} + +int kvm_arch_process_async_events (CPUState *cs) +{ +return cs->halted; +} + +bool kvm_arch_stop_on_emulation_error (CPUState *cs) +{ +return true; +} + +bool kvm_arch_cpu_check_are_resettable (void ) +{ +return true; +} + +int kvm_arch_handle_exit (CPUState *cs,struct kvm_run *run) +{ +return 0 ; +} + +void kvm_arch_accel_class_init (ObjectClass *oc) +{ +}
Signature · 1 line
2.39.1
RFC PATCH v4 3/9 target/loongarch: Supplement vcpu env initial when vcpu reset
In reply to xianglai liFrom: Tianrui Zhao <[email protected]>
Supplement vcpu env initial when vcpu reset, including init vcpu CSR_CPUID,CSR_TID to cpu->cpu_index. The two regs will be used in kvm_get/set_csr_ioctl.
--- target/loongarch/cpu.c | 2 ++ target/loongarch/cpu.h | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-)
target/loongarch/cpu.c +2 -0
@@ -524,10 +524,12 @@ static void loongarch_cpu_reset_hold(Object *obj) env->CSR_ESTAT = env->CSR_ESTAT & (~MAKE_64BIT_MASK (0 ,2 )); env->CSR_RVACFG =FIELD_DP64 (env->CSR_RVACFG,CSR_RVACFG ,RBITS ,0 ); + env->CSR_CPUID = cs->cpu_index; env->CSR_TCFG =FIELD_DP64 (env->CSR_TCFG,CSR_TCFG ,EN ,0 ); env->CSR_LLBCTL =FIELD_DP64 (env->CSR_LLBCTL,CSR_LLBCTL ,KLO ,0 ); env->CSR_TLBRERA =FIELD_DP64 (env->CSR_TLBRERA,CSR_TLBRERA ,ISTLBR ,0 ); env->CSR_MERRCTL =FIELD_DP64 (env->CSR_MERRCTL,CSR_MERRCTL ,ISMERR ,0 ); + env->CSR_TID = cs->cpu_index; env->CSR_PRCFG3 =FIELD_DP64 (env->CSR_PRCFG3,CSR_PRCFG3 ,TLB_TYPE ,2 ); env->CSR_PRCFG3 =FIELD_DP64 (env->CSR_PRCFG3,CSR_PRCFG3 ,MTLB_ENTRY ,63 );
target/loongarch/cpu.h +1 -1
@@ -318,6 +318,7 @@ typedef struct CPUArchState {uint64_t CSR_PWCH ;uint64_t CSR_STLBPS ;uint64_t CSR_RVACFG ; +uint64_t CSR_CPUID ;uint64_t CSR_PRCFG1 ;uint64_t CSR_PRCFG2 ;uint64_t CSR_PRCFG3 ;
@@ -349,7 +350,6 @@ typedef struct CPUArchState {uint64_t CSR_DBG ;uint64_t CSR_DERA ;uint64_t CSR_DSAVE ; -uint64_t CSR_CPUID ;#ifndef CONFIG_USER_ONLY LoongArchTLB tlb[LOONGARCH_TLB_MAX ];
Signature · 1 line
2.39.1
RFC PATCH v4 4/9 target/loongarch: Implement kvm get/set registers
In reply to xianglai liFrom: Tianrui Zhao <[email protected]>
Implement kvm_arch_get/set_registers interfaces, many regs can be get/set in the function, such as core regs, csr regs, fpu regs, mp state, etc.
--- meson.build | 1 + target/loongarch/cpu.c | 3 + target/loongarch/cpu.h | 2 + target/loongarch/kvm.c | 406 +++++++++++++++++++++++++++++++++- target/loongarch/trace-events | 13 ++ target/loongarch/trace.h | 1 + 6 files changed, 424 insertions(+), 2 deletions(-) create mode 100644 target/loongarch/trace-events create mode 100644 target/loongarch/trace.h
meson.build +1 -0
@@ -3305,6 +3305,7 @@ if have_system or have_user 'target/hppa', 'target/i386', 'target/i386/kvm', + 'target/loongarch', 'target/mips/tcg', 'target/nios2', 'target/ppc',
target/loongarch/cpu.c +3 -0
@@ -546,6 +546,9 @@ static void loongarch_cpu_reset_hold(Object *obj)#ifndef CONFIG_USER_ONLY env->pc =0x1c000000 ;memset (env->tlb,0 ,sizeof (env->tlb)); +if (kvm_enabled ()) { +kvm_arch_reset_vcpu (env); + }#endif restore_fp_status (env);
target/loongarch/cpu.h +2 -0
@@ -359,6 +359,7 @@ typedef struct CPUArchState {MemoryRegion iocsr_mem;bool load_elf;uint64_t elf_address; +uint32_t mp_state;/* Store ipistate to access from this struct */ DeviceState *ipistate;#endif
@@ -477,6 +478,7 @@ static inline void cpu_get_tb_cpu_state(CPULoongArchState *env, vaddr *pc, }void loongarch_cpu_list (void ); +void kvm_arch_reset_vcpu (CPULoongArchState *env);#define cpu_list loongarch_cpu_list
target/loongarch/kvm.c +404 -2
423 lines not on this page · raw message
target/loongarch/trace-events new file +13 -0
@@ -0,0 +1,13 @@ +# See docs/devel/tracing.rst for syntax documentation. + +#kvm.c +kvm_failed_get_regs_core(const char *msg) "Failed to get core regs from KVM: %s" +kvm_failed_put_regs_core(const char *msg) "Failed to put core regs into KVM: %s" +kvm_failed_get_csr(int csr, const char *msg) "Failed to get csr 0x%x from KVM: %s" +kvm_failed_put_csr(int csr, const char *msg) "Failed to put csr 0x%x into KVM: %s" +kvm_failed_get_fpu(const char *msg) "Failed to get fpu from KVM: %s" +kvm_failed_put_fpu(const char *msg) "Failed to put fpu into KVM: %s" +kvm_failed_get_mpstate(const char *msg) "Failed to get mp_state from KVM: %s" +kvm_failed_put_mpstate(const char *msg) "Failed to put mp_state into KVM: %s" +kvm_failed_get_cpucfg(const char *msg) "Failed to get cpucfg from KVM: %s" +kvm_failed_put_cpucfg(const char *msg) "Failed to put cpucfg into KVM: %s"
target/loongarch/trace.h new file +1 -0
@@ -0,0 +1 @@ +#include "trace/trace-target_loongarch.h"
Signature · 1 line
2.39.1
Re: [PATCH RFC v4 4/9] target/loongarch: Implement kvm get/set registers
Hi Li and Zhao,
Quoted 34 lines · xianglai li
From: Tianrui Zhao <[email protected]>
Implement kvm_arch_get/set_registers interfaces, many regs can be get/set in the function, such as core regs, csr regs, fpu regs, mp state, etc.
Cc "Michael S. Tsirkin" <[email protected]>Cc Cornelia Huck <[email protected]>Cc Paolo Bonzini <[email protected]>Cc "Marc-André Lureau" <[email protected]>Cc "Daniel P. Berrangé" <[email protected]>Cc Thomas Huth <[email protected]>Cc "Philippe Mathieu-Daudé" <[email protected]>Cc Richard Henderson <[email protected]>Cc Peter Maydell <[email protected]>Cc Bibo Mao <[email protected]>Cc Song Gao <[email protected]>Cc Xiaojuan Yang <[email protected]>Cc Tianrui Zhao <[email protected]>Signed-off-by Tianrui Zhao <[email protected]>Signed-off-by xianglai li <[email protected]>--- meson.build | 1 + target/loongarch/cpu.c | 3 + target/loongarch/cpu.h | 2 + target/loongarch/kvm.c | 406 +++++++++++++++++++++++++++++++++- target/loongarch/trace-events | 13 ++ target/loongarch/trace.h | 1 + 6 files changed, 424 insertions(+), 2 deletions(-)create mode 100644 target/loongarch/trace-events create mode 100644
target/loongarch/trace.h
target/loongarch/kvm.c line 91
Quoted 10 lines
target/loongarch/kvm.c +10
-0+static int kvm_larch_getq (CPUState *cs,uint64_t reg_id, +uint64_t *addr) +{ +struct kvm_one_reg csrreg = { + .id = reg_id, + .addr = (uintptr_t )addr + }; + +return kvm_vcpu_ioctl (cs,KVM_GET_ONE_REG , &csrreg); +}
This is kvm_get_one_reg().
target/loongarch/kvm.c line 102
Quoted 10 lines
target/loongarch/kvm.c +10
-0+static int kvm_larch_putq (CPUState *cs,uint64_t reg_id, +uint64_t *addr) +{ +struct kvm_one_reg csrreg = { + .id = reg_id, + .addr = (uintptr_t )addr + }; + +return kvm_vcpu_ioctl (cs,KVM_SET_ONE_REG , &csrreg); +}
This is kvm_set_one_reg().
Quoted 18 lines
target/loongarch/kvm.c +18
-0+ +#define KVM_GET_ONE_UREG64 (cs, ret, regidx, addr) \ + ({ \ + err = kvm_larch_getq(cs, KVM_IOC_CSRID(regidx), addr); \ + if (err < 0) { \ + ret = err; \ + trace_kvm_failed_get_csr(regidx, strerror(errno)); \ + } \ + }) + +#define KVM_PUT_ONE_UREG64 (cs, ret, regidx, addr) \ + ({ \ + err = kvm_larch_putq(cs, KVM_IOC_CSRID(regidx), addr); \ + if (err < 0) { \ + ret = err; \ + trace_kvm_failed_put_csr(regidx, strerror(errno)); \ + } \ + })
Re: [PATCH RFC v4 4/9] target/loongarch: Implement kvm get/set registers
Hi Philippe Mathieu-Daudé:
Quoted 37 lines
Hi Li and Zhao,
Quoted 32 lines · xianglai li
From: Tianrui Zhao <[email protected]>
Implement kvm_arch_get/set_registers interfaces, many regs can be get/set in the function, such as core regs, csr regs, fpu regs, mp state, etc.
Cc "Michael S. Tsirkin" <[email protected]>Cc Cornelia Huck <[email protected]>Cc Paolo Bonzini <[email protected]>Cc "Marc-André Lureau" <[email protected]>Cc "Daniel P. Berrangé" <[email protected]>Cc Thomas Huth <[email protected]>Cc "Philippe Mathieu-Daudé" <[email protected]>Cc Richard Henderson <[email protected]>Cc Peter Maydell <[email protected]>Cc Bibo Mao <[email protected]>Cc Song Gao <[email protected]>Cc Xiaojuan Yang <[email protected]>Cc Tianrui Zhao <[email protected]>Signed-off-by Tianrui Zhao <[email protected]>Signed-off-by xianglai li <[email protected]>---
meson.build | 1 +target/loongarch/cpu.c | 3 + target/loongarch/cpu.h | 2 + target/loongarch/kvm.c | 406 +++++++++++++++++++++++++++++++++- target/loongarch/trace-events | 13 ++ target/loongarch/trace.h | 1 + 6 files changed, 424 insertions(+), 2 deletions(-)create mode 100644 target/loongarch/trace-events create mode 100644
target/loongarch/trace.h
Quoted 12 lines
target/loongarch/kvm.c line 91
Quoted 10 lines
target/loongarch/kvm.c +10
-0+static int kvm_larch_getq (CPUState *cs,uint64_t reg_id, + uint64_t *addr) +{ + struct kvm_one_reg csrreg = { + .id = reg_id, + .addr = (uintptr_t)addr + }; + + return kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &csrreg); +}This is
kvm_get_one_reg().
I'll replace kvm_larch_getq() with kvm_get_one_reg().
Quoted 15 lines
target/loongarch/kvm.c line 102
Quoted 12 lines
target/loongarch/kvm.c +10
-0+static int kvm_larch_putq (CPUState *cs,uint64_t reg_id, + uint64_t *addr) +{ + struct kvm_one_reg csrreg = { + .id = reg_id, + .addr = (uintptr_t)addr + }; + + return kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &csrreg); +}This is
kvm_set_one_reg().
I'll replace kvm_larch_putq() with kvm_set_one_reg().
Thanks,
Xianglai.
Quoted 19 lines
Quoted 18 lines
target/loongarch/kvm.c +18
-0+ +#define KVM_GET_ONE_UREG64 (cs, ret, regidx, addr) \ + ({ \ + err = kvm_larch_getq(cs, KVM_IOC_CSRID(regidx), addr); \ + if (err < 0) { \ + ret = err; \ + trace_kvm_failed_get_csr(regidx, strerror(errno)); \ + } \ + }) + +#define KVM_PUT_ONE_UREG64 (cs, ret, regidx, addr) \ + ({ \ + err = kvm_larch_putq(cs, KVM_IOC_CSRID(regidx), addr); \ + if (err < 0) { \ + ret = err; \ + trace_kvm_failed_put_csr(regidx, strerror(errno)); \ + } \ + })
RFC PATCH v4 5/9 target/loongarch: Implement kvm_arch_init function
In reply to xianglai liFrom: Tianrui Zhao <[email protected]>
Implement the kvm_arch_init of loongarch, in the function, the KVM_CAP_MP_STATE cap is checked by kvm ioctl.
--- target/loongarch/kvm.c | 1 + 1 file changed, 1 insertion(+)
target/loongarch/kvm.c +1 -0
@@ -491,6 +491,7 @@ int kvm_arch_get_default_type(MachineState *ms)int kvm_arch_init (MachineState *ms,KVMState *s) { + cap_has_mp_state =kvm_check_extension (s,KVM_CAP_MP_STATE );return 0 ; }
Signature · 1 line
2.39.1
RFC PATCH v4 6/9 target/loongarch: Implement kvm_arch_init_vcpu
In reply to xianglai liFrom: Tianrui Zhao <[email protected]>
Implement kvm_arch_init_vcpu interface for loongarch, in this function, we register VM change state handler. And when VM state changes to running, the counter value should be put into kvm to keep consistent with kvm, and when state change to stop, counter value should be refreshed from kvm.
--- target/loongarch/cpu.h | 2 ++ target/loongarch/kvm.c | 23 +++++++++++++++++++++++ target/loongarch/trace-events | 2 ++ 3 files changed, 27 insertions(+)
target/loongarch/cpu.h +2 -0
@@ -382,6 +382,8 @@ struct ArchCPU {/* 'compatible' string for this CPU for Linux device trees */ const char *dtb_compatible; +/* used by KVM_REG_LOONGARCH_COUNTER ioctl to access guest time counters */ +uint64_t kvm_state_counter; };#define TYPE_LOONGARCH_CPU "loongarch-cpu"
target/loongarch/kvm.c +23 -0
@@ -443,8 +443,31 @@ int kvm_arch_put_registers(CPUState *cs, int level)return ret; } +static void kvm_loongarch_vm_stage_change (void *opaque,bool running, +RunState state) +{ +int ret; +CPUState *cs = opaque; +LoongArchCPU *cpu =LOONGARCH_CPU (cs); + +if (running) { + ret =kvm_larch_putq (cs,KVM_REG_LOONGARCH_COUNTER , + &cpu->kvm_state_counter); +if (ret <0 ) { +trace_kvm_failed_put_counter (strerror (errno)); + } + }else { + ret =kvm_larch_getq (cs,KVM_REG_LOONGARCH_COUNTER , + &cpu->kvm_state_counter); +if (ret <0 ) { +trace_kvm_failed_get_counter (strerror (errno)); + } + } +} +int kvm_arch_init_vcpu (CPUState *cs) { +qemu_add_vm_change_state_handler (kvm_loongarch_vm_stage_change, cs);return 0 ; }
target/loongarch/trace-events +2 -0
@@ -9,5 +9,7 @@ kvm_failed_get_fpu(const char *msg) "Failed to get fpu from KVM: %s" kvm_failed_put_fpu(const char *msg) "Failed to put fpu into KVM: %s" kvm_failed_get_mpstate(const char *msg) "Failed to get mp_state from KVM: %s" kvm_failed_put_mpstate(const char *msg) "Failed to put mp_state into KVM: %s" +kvm_failed_get_counter(const char *msg) "Failed to get counter from KVM: %s" +kvm_failed_put_counter(const char *msg) "Failed to put counter into KVM: %s" kvm_failed_get_cpucfg(const char *msg) "Failed to get cpucfg from KVM: %s" kvm_failed_put_cpucfg(const char *msg) "Failed to put cpucfg into KVM: %s"
Signature · 1 line
2.39.1
RFC PATCH v4 7/9 target/loongarch: Implement kvm_arch_handle_exit
In reply to xianglai liFrom: Tianrui Zhao <[email protected]>
Implement kvm_arch_handle_exit for loongarch. In this function, the KVM_EXIT_LOONGARCH_IOCSR is handled, we read or write the iocsr address space by the addr, length and is_write argument in kvm_run.
--- target/loongarch/kvm.c | 24 +++++++++++++++++++++++- target/loongarch/trace-events | 1 + 2 files changed, 24 insertions(+), 1 deletion(-)
target/loongarch/kvm.c +23 -1
@@ -549,7 +549,29 @@ bool kvm_arch_cpu_check_are_resettable(void)int kvm_arch_handle_exit (CPUState *cs,struct kvm_run *run) { -return 0 ; +int ret =0 ; +LoongArchCPU *cpu =LOONGARCH_CPU (cs); +CPULoongArchState *env = &cpu->env; +MemTxAttrs attrs = {}; + + attrs.requester_id =env_cpu (env)->cpu_index; + +trace_kvm_arch_handle_exit (run->exit_reason); +switch (run->exit_reason) { +case KVM_EXIT_LOONGARCH_IOCSR : +address_space_rw (&env->address_space_iocsr, + run->iocsr_io.phys_addr, + attrs, + run->iocsr_io.data, + run->iocsr_io.len, + run->iocsr_io.is_write); +break ; +default : + ret =-1 ; +warn_report ("KVM: unknown exit reason %d" , run->exit_reason); +break ; + } +return ret; }void kvm_arch_accel_class_init (ObjectClass *oc)
target/loongarch/trace-events +1 -0
@@ -13,3 +13,4 @@ kvm_failed_get_counter(const char *msg) "Failed to get counter from KVM: %s" kvm_failed_put_counter(const char *msg) "Failed to put counter into KVM: %s" kvm_failed_get_cpucfg(const char *msg) "Failed to get cpucfg from KVM: %s" kvm_failed_put_cpucfg(const char *msg) "Failed to put cpucfg into KVM: %s" +kvm_arch_handle_exit(int num) "kvm arch handle exit, the reason number: %d"
Signature · 1 line
2.39.1
RFC PATCH v4 8/9 target/loongarch: Implement set vcpu intr for kvm
In reply to xianglai liFrom: Tianrui Zhao <[email protected]>
Implement loongarch kvm set vcpu interrupt interface, when a irq is set in vcpu, we use the KVM_INTERRUPT ioctl to set intr into kvm.
--- target/loongarch/cpu.c | 18 +++++++++++++----- target/loongarch/kvm-stub.c | 11 +++++++++++ target/loongarch/kvm.c | 15 +++++++++++++++ target/loongarch/kvm_loongarch.h | 13 +++++++++++++ target/loongarch/trace-events | 1 + 5 files changed, 53 insertions(+), 5 deletions(-) create mode 100644 target/loongarch/kvm-stub.c create mode 100644 target/loongarch/kvm_loongarch.h
target/loongarch/cpu.c +13 -5
@@ -20,6 +20,11 @@#include "sysemu/reset.h" #include "tcg/tcg.h" #include "vec.h" +#include "sysemu/kvm.h" +#include "kvm_loongarch.h" +#ifdef CONFIG_KVM +#include <linux/kvm.h> +#endif const char *const regnames[32 ] = {"r0" ,"r1" ,"r2" ,"r3" ,"r4" ,"r5" ,"r6" ,"r7" ,
@@ -108,12 +113,15 @@ void loongarch_cpu_set_irq(void *opaque, int irq, int level)return ; } - env->CSR_ESTAT =deposit64 (env->CSR_ESTAT, irq,1 , level !=0 ); - -if (FIELD_EX64 (env->CSR_ESTAT,CSR_ESTAT ,IS )) { -cpu_interrupt (cs,CPU_INTERRUPT_HARD ); +if (kvm_enabled ()) { +kvm_loongarch_set_interrupt (cpu, irq, level); }else { -cpu_reset_interrupt (cs,CPU_INTERRUPT_HARD ); + env->CSR_ESTAT =deposit64 (env->CSR_ESTAT, irq,1 , level !=0 ); +if (FIELD_EX64 (env->CSR_ESTAT,CSR_ESTAT ,IS )) { +cpu_interrupt (cs,CPU_INTERRUPT_HARD ); + }else { +cpu_reset_interrupt (cs,CPU_INTERRUPT_HARD ); + } } }
target/loongarch/kvm-stub.c new file +11 -0
@@ -0,0 +1,11 @@ +/* +* QEMU KVM LoongArch specific function stubs +* +* Copyright (c) 2023 Loongson Technology Corporation Limited +*/ +#include "cpu.h" + +void kvm_loongarch_set_interrupt (LoongArchCPU *cpu,int irq,int level) +{ +g_assert_not_reached (); +}
target/loongarch/kvm.c +15 -0
@@ -574,6 +574,21 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)return ret; } +int kvm_loongarch_set_interrupt (LoongArchCPU *cpu,int irq,int level) +{ +struct kvm_interrupt intr; +CPUState *cs =CPU (cpu); + +if (level) { + intr.irq = irq; + }else { + intr.irq = -irq; + } + +trace_kvm_set_intr (irq, level); +return kvm_vcpu_ioctl (cs,KVM_INTERRUPT , &intr); +} +void kvm_arch_accel_class_init (ObjectClass *oc) { }
target/loongarch/kvm_loongarch.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* QEMU LoongArch kvm interface +* +* Copyright (c) 2023 Loongson Technology Corporation Limited +*/ + +#ifndef QEMU_KVM_LOONGARCH_H +#define QEMU_KVM_LOONGARCH_H + +int kvm_loongarch_set_interrupt (LoongArchCPU *cpu,int irq,int level); + +#endif
target/loongarch/trace-events +1 -0
@@ -14,3 +14,4 @@ kvm_failed_put_counter(const char *msg) "Failed to put counter into KVM: %s" kvm_failed_get_cpucfg(const char *msg) "Failed to get cpucfg from KVM: %s" kvm_failed_put_cpucfg(const char *msg) "Failed to put cpucfg into KVM: %s" kvm_arch_handle_exit(int num) "kvm arch handle exit, the reason number: %d" +kvm_set_intr(int irq, int level) "kvm set interrupt, irq num: %d, level: %d"
Signature · 1 line
2.39.1
RFC PATCH v4 9/9 target/loongarch: Add loongarch kvm into meson build
In reply to xianglai liFrom: Tianrui Zhao <[email protected]>
Add kvm.c and kvm-stub.c into meson.build to compile it when kvm is configed. Meanwhile in meson.build, we set the kvm_targets to loongarch64-softmmu when the cpu is loongarch.
--- meson.build | 2 ++ target/loongarch/meson.build | 1 + 2 files changed, 3 insertions(+)
meson.build +2 -0
@@ -114,6 +114,8 @@ elif cpu in ['riscv32'] kvm_targets = ['riscv32-softmmu'] elif cpu in ['riscv64'] kvm_targets = ['riscv64-softmmu'] +elif cpu in ['loongarch64'] + kvm_targets = ['loongarch64-softmmu'] else kvm_targets = [] endif
target/loongarch/meson.build +1 -0
@@ -27,6 +27,7 @@ loongarch_system_ss.add(files( common_ss.add(when: 'CONFIG_LOONGARCH_DIS', if_true: [files('disas.c'), gen]) +loongarch_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c')) loongarch_ss.add_all(when: 'CONFIG_TCG', if_true: [loongarch_tcg_ss]) target_arch += {'loongarch': loongarch_ss}
Signature · 1 line
2.39.1
Hi,
Quoted 28 lines · xianglai li
This series add loongarch kvm support, mainly implement some interfaces used by kvm such as kvm_arch_get/set_regs,
kvm_arch_handle_exit,kvm_loongarch_set_interrupt, etc.Currently, we are able to boot LoongArch KVM Linux Guests. In loongarch VM, mmio devices and iocsr devices are emulated in user space such as APIC, IPI, pci devices, etc, other hardwares such as MMU, timer and csr are emulated in kernel.
It is based on temporarily unaccepted linux kvm:
https://github.com/loongson/linux-loongarch-kvm
And We will remove the RFC flag until the linux kvm patches are merged.The running environment of LoongArch virt machine:
1. Get the linux source by the above mentioned link.
git checkout kvm-loongarch
make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu-loongson3_defconfigmake ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu-
2. Get the qemu source: https://github.com/loongson/qemu
git checkout kvm-loongarch
./configure --target-list="loongarch64-softmmu" --enable-kvm
make
3. Get uefi bios of LoongArch virt machine:4. Also you can access the binary files we have already build:
https://github.com/yangxiaojuan-loongson/qemu-binary
Quoted 6 lines
The command to boot loongarch virt machine:
$ qemu-system-loongarch64 -machine virt -m 4G -cpu la464 \ -smp 1 -bios QEMU_EFI.fd -kernel vmlinuz.efi -initrd ramdisk \ -serial stdio -monitor telnet:localhost:4495,server,nowait \ -append "root=/dev/ram rdinit=/sbin/init console=ttyS0,115200" \ --nographic
2 years ago Song helped with an access to a LoongArch 3a5000 machine but it stopped working (IP was x.242.206.180).
Would it be possible to add a Loongarch64 runner to our CI (ideally with KVM support, but that can come later)? See:
https://www.qemu.org/docs/master/devel/ci.html#jobs-on-custom-runners
Regards,
Phil.
Hi Philippe Mathieu-Daudé :
Quoted 38 lines
Hi,
Quoted 28 lines · xianglai li
This series add loongarch kvm support, mainly implement some interfaces used by kvm such as kvm_arch_get/set_regs,
kvm_arch_handle_exit,kvm_loongarch_set_interrupt, etc.Currently, we are able to boot LoongArch KVM Linux Guests. In loongarch VM, mmio devices and iocsr devices are emulated in user space such as APIC, IPI, pci devices, etc, other hardwares such as MMU, timer and csr are emulated in kernel.
It is based on temporarily unaccepted linux kvm:
https://github.com/loongson/linux-loongarch-kvm
And We will remove the RFC flag until the linux kvm patches are merged.The running environment of LoongArch virt machine:
1. Get the linux source by the above mentioned link.
git checkout kvm-loongarch
make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu-loongson3_defconfigmake ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu-
2. Get the qemu source: https://github.com/loongson/qemu
git checkout kvm-loongarch
./configure --target-list="loongarch64-softmmu" --enable-kvm
make
3. Get uefi bios of LoongArch virt machine:4. Also you can access the binary files we have already build:
https://github.com/yangxiaojuan-loongson/qemu-binaryQuoted 6 lines
The command to boot loongarch virt machine:
$ qemu-system-loongarch64 -machine virt -m 4G -cpu la464 \ -smp 1 -bios QEMU_EFI.fd -kernel vmlinuz.efi -initrd ramdisk \ -serial stdio -monitor telnet:localhost:4495,server,nowait \ -append "root=/dev/ram rdinit=/sbin/init console=ttyS0,115200" \ --nographic
Quoted 7 lines
2 years ago Song helped with an access to a LoongArch 3a5000 machine but it stopped working (IP was x.242.206.180).
Would it be possible to add a Loongarch64 runner to our CI (ideally with KVM support, but that can come later)? See:
https://www.qemu.org/docs/master/devel/ci.html#jobs-on-custom-runners
Ok, Song Gao will rebuild the Loongarch64 runner environment in the next few days,
and we will publish it to the community as soon as it is completed.
Thanks,
Xianglai.
Quoted 3 lines
Regards,
Phil.