Add RISC-V Hypervisor Extension
Subsystems (2)
cover Add RISC-V Hypervisor Extension
This patch series adds the RISC-V Hypervisor extension 0.3. This is the latest draft spec of the Hypervisor extension.
The Hypervisor extension is disabled by default, so this series should result in no changes to anyone using QEMU unless they enable the extension. The extention can be enabled with the -cpu property (see below).
At the moment the spec does not include information about the mstatush register. As it is not in the spec I haven't added it to QEMU. This means the extension won't work correctly for 32-bit guests. This should be a small fix to add the CSR once the spec is updated.
All testing of this implementation has been done by using the baremetal Xvisor Hypervisor. We are able to run two Linux guests (that's all I have tried) as guests.
At the moment this spec is in a draft state and is subject to change. As QEMU is extreamly useful in early bring up I think it makes sense for QEMU to support non-frozen extensions. I would like to decide with this series how QEMU will handle all future non-frozen extensions. That is a standard way that QEMU users can test future RISC-V extensions while still understanding things will change. One idea is just to disable it by default, another option is to maybe use the Kconfig to make it a compile time option which developers can use. Should we also display a warning when running non-frozen extensions?
Thanks to Anup for doing the initial port of Xvisor. The port is avaliable here:
https://github.com/avpatel/xvisor-next and will run on QEMU.
Also thanks to Atish for implementing the SBI call support in Xvisor and for lots of help debugging.
To run this yourself:
1. Apply this patch series to QEMU. The latest branch can be found here:
https://github.com/alistair23/qemu/tree/mainline/alistair/riscv-hyp-work.next
2. Get the version of OpenSBI that supports the H extension. This can be found here:
https://github.com/riscv/opensbi/tree/hyp_ext_changes_v1
3. Build the next release of Xvisor. It is available here:
https://github.com/avpatel/xvisor-next
4. Make sure you build the Xvisor tests, see here for details:
https://github.com/avpatel/xvisor-next/tree/master/tests/riscv/virt64/linux
5. Run QEMU:
./riscv64-softmmu/qemu-system-riscv64 -nographic \
-machine virt -cpu rv64,h=true\
-serial mon:stdio -serial null -m 4G \
-device loader,file=vmm.bin,addr=0x80200000 \
-kernel fw_jump.elf \
-initrd vmm-disk-linux.img \
-append "vmm.console=uart@10000000 vmm.bootcmd=\"vfs mount initrd /;vfs run /boot.xscript;vfs cat /system/banner.txt\"" Once you get to the prompt you can start the geust by running:
guest kick guest0
You can then bind to the serial port using:
vserial bind guest0/uart0
Then you can start Linux using:
autoexec
This was all tested with the mainline 5.1 kernel. I don't know if it will work on older kernels.
So far all of the QEMU work has been tested on Xvisor.
Known Issues/TODO:
- Add mstatush to support 32-bit Hypervisors
- Fix the random hang that sometimes appears when running a Hypervisor guest (~5%)
There is also on going work from Anup to port KVM.
We have code complete implementation of RISC-V KVM kernel module and RISC-V KVMTOOL. Currently, we are debugging KVM on QEMU and we will send-out RFC PATCHES for KVM in June/July 2019.
The KVM RISC-V kernel module is available in riscv_kvm_v1 branch at: https://github.com/avpatel/linux.git
The KVMTOOL RISC-V port is available in riscv_v1 branch of
https://github.com/avpatel/kvmtool.git
There is very early work on a Xen port as well which is avaliable here:
https://github.com/alistair23/xen/tree/alistair/riscv-port
Changes from RFC:
- Rebase on latest master
- Add floating point changes from Hypervisor extension
Alistair Francis (27):
target/riscv: Don't set write permissions on dirty PTEs target/riscv: Add the Hypervisor extension target/riscv: Add the virtulisation mode target/riscv: Add the force HS exception mode target/riscv: Add the Hypervisor CSRs to CPUState target/riscv: Dump Hypervisor registers if enabled target/riscv: Remove strict perm checking for CSR R/W target/riscv: Create function to test if FP is enabled target/riscv: Add support for background interrupt setting target/riscv: Add Hypervisor CSR access functions target/riscv: Add background CSRs accesses target/riscv: Add background register swapping function target/ricsv: Flush the TLB on virtulisation mode changes target/riscv: Generate illegal instruction on WFI when V=1
riscv: plic: Remove unused interrupt functions
riscv: plic: Always set sip.SEIP bit for HS target/riscv: Add hypvervisor trap support target/riscv: Add Hypervisor trap return support target/riscv: Add hfence instructions
target/riscv: Disable guest FP support based on backgrond status target/riscv: Mark both sstatus and bsstatus as dirty target/riscv: Respect MPRV and SPRV for floating point ops target/riscv: Allow specifying MMU stage target/riscv: Allow specifying number of MMU stages target/riscv: Implement second stage MMU target/riscv: Call the second stage MMU in virtualisation mode target/riscv: Allow enabling the Hypervisor extension
hw/riscv/sifive_plic.c | 24 +- include/hw/riscv/sifive_plic.h | 3 - target/riscv/cpu.c | 31 ++ target/riscv/cpu.h | 32 +- target/riscv/cpu_bits.h | 32 +- target/riscv/cpu_helper.c | 443 ++++++++++++++++-- target/riscv/csr.c | 216 ++++++++- target/riscv/insn32.decode | 23 +- .../riscv/insn_trans/trans_privileged.inc.c | 40 ++ target/riscv/op_helper.c | 71 ++- target/riscv/translate.c | 26 + 11 files changed, 839 insertions(+), 102 deletions(-)
Signature · 1 line
2.21.0
PATCH 1/27 target/riscv: Don't set write permissions on dirty PTEs
Setting write permission on dirty PTEs results in userspace inside a Hypervisor guest (VU) becoming corrupted. This appears to be becuase it ends up with write permission in the second stage translation in cases where we aren't doing a store.
--- target/riscv/cpu_helper.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
target/riscv/cpu_helper.c +2 -4
@@ -326,10 +326,8 @@ restart:if ((pte &PTE_X )) { *prot |=PAGE_EXEC ; } -/* add write permission on stores or if the page is already dirty, -so that we TLB miss on later writes to update the dirty bit */ -if ((pte &PTE_W ) && - (access_type ==MMU_DATA_STORE || (pte &PTE_D ))) { +/* add write permission on stores */ +if ((pte &PTE_W ) && (access_type ==MMU_DATA_STORE )) { *prot |=PAGE_WRITE ; }return TRANSLATE_SUCCESS ;
Signature · 1 line
2.21.0
PATCH 2/27 target/riscv: Add the Hypervisor extension
In reply to Alistair Francis--- target/riscv/cpu.h | 1 + 1 file changed, 1 insertion(+)
target/riscv/cpu.h +1 -0
@@ -81,6 +81,7 @@#define RVC RV('C')#define RVS RV('S')#define RVU RV('U') +#define RVH RV('H')/* S extension denotes that Supervisor mode exists, however it is possible to have a core that support S mode but does not have an MMU and there
Signature · 1 line
2.21.0
PATCH 3/27 target/riscv: Add the virtulisation mode
In reply to Alistair Francis--- target/riscv/cpu.h | 4 ++++ target/riscv/cpu_bits.h | 6 ++++++ target/riscv/cpu_helper.c | 23 +++++++++++++++++++++++ 3 files changed, 33 insertions(+)
target/riscv/cpu.h +4 -0
@@ -132,6 +132,8 @@ struct CPURISCVState {#ifndef CONFIG_USER_ONLY target_ulong priv; +/* This contains QEMU specific information about the virt state. */ +target_ulong virt;target_ulong resetvec;target_ulong mhartid;
@@ -278,6 +280,8 @@ void riscv_cpu_do_interrupt(CPUState *cpu);int riscv_cpu_gdb_read_register (CPUState *cpu,uint8_t *buf,int reg);int riscv_cpu_gdb_write_register (CPUState *cpu,uint8_t *buf,int reg);bool riscv_cpu_exec_interrupt (CPUState *cs,int interrupt_request); +bool riscv_cpu_virt_enabled (CPURISCVState *env); +void riscv_cpu_set_virt_enabled (CPURISCVState *env,bool enable);int riscv_cpu_mmu_index (CPURISCVState *env,bool ifetch);hwaddr riscv_cpu_get_phys_page_debug (CPUState *cpu,vaddr addr); void riscv_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
target/riscv/cpu_bits.h +6 -0
@@ -417,6 +417,12 @@#define PRV_H 2/* Reserved */ #define PRV_M 3 +/* Virtulisation modes */ +#define VIRT_OFF 0 +#define VIRT_ON 1 +#define VIRT_MODE_SHIFT 0 +#define VIRT_MODE_MASK (1 << VIRT_MODE_SHIFT) +/* RV32 satp CSR field masks */ #define SATP32_MODE 0x80000000#define SATP32_ASID 0x7fc00000
target/riscv/cpu_helper.c +23 -0
@@ -71,6 +71,29 @@ bool riscv_cpu_exec_interrupt(CPUState *cs, int interrupt_request)#if !defined(CONFIG_USER_ONLY ) +bool riscv_cpu_virt_enabled (CPURISCVState *env) +{ +bool tmp; + +if (!riscv_has_ext (env,RVH )) { +return false; + } + + tmp = (env->virt &VIRT_MODE_MASK ) >>VIRT_MODE_SHIFT ; + +return tmp ==VIRT_ON ; +} + +void riscv_cpu_set_virt_enabled (CPURISCVState *env,bool enable) +{ +if (!riscv_has_ext (env,RVH )) { +return ; + } + + env->virt &= ~VIRT_MODE_MASK ; + env->virt |= enable <<VIRT_MODE_SHIFT ; +} +int riscv_cpu_claim_interrupts (RISCVCPU *cpu,uint32_t interrupts) {CPURISCVState *env = &cpu->env;
Signature · 1 line
2.21.0
PATCH 4/27 target/riscv: Add the force HS exception mode
In reply to Alistair Francis--- target/riscv/cpu.h | 2 ++ target/riscv/cpu_bits.h | 6 ++++++ target/riscv/cpu_helper.c | 23 +++++++++++++++++++++++ 3 files changed, 31 insertions(+)
target/riscv/cpu.h +2 -0
@@ -282,6 +282,8 @@ int riscv_cpu_gdb_write_register(CPUState *cpu, uint8_t *buf, int reg);bool riscv_cpu_exec_interrupt (CPUState *cs,int interrupt_request);bool riscv_cpu_virt_enabled (CPURISCVState *env);void riscv_cpu_set_virt_enabled (CPURISCVState *env,bool enable); +bool riscv_cpu_force_hs_excep_enabled (CPURISCVState *env); +void riscv_cpu_set_force_hs_excep (CPURISCVState *env,bool enable);int riscv_cpu_mmu_index (CPURISCVState *env,bool ifetch);hwaddr riscv_cpu_get_phys_page_debug (CPUState *cpu,vaddr addr); void riscv_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
target/riscv/cpu_bits.h +6 -0
@@ -423,6 +423,12 @@#define VIRT_MODE_SHIFT 0#define VIRT_MODE_MASK (1 << VIRT_MODE_SHIFT) +/* HS-level exceptions modes */ +#define CLEAR_HS_EXCEP 0 +#define FORCE_HS_EXCEP 1 +#define FORCE_HS_EXCEP_SHIFT 1 +#define FORCE_HS_EXCEP_MASK (1 << FORCE_HS_EXCEP_SHIFT) +/* RV32 satp CSR field masks */ #define SATP32_MODE 0x80000000#define SATP32_ASID 0x7fc00000
target/riscv/cpu_helper.c +23 -0
@@ -94,6 +94,29 @@ void riscv_cpu_set_virt_enabled(CPURISCVState *env, bool enable) env->virt |= enable <<VIRT_MODE_SHIFT ; } +bool riscv_cpu_force_hs_excep_enabled (CPURISCVState *env) +{ +bool tmp; + +if (!riscv_has_ext (env,RVH )) { +return false; + } + + tmp = (env->virt &FORCE_HS_EXCEP_MASK ) >>FORCE_HS_EXCEP_SHIFT ; + +return tmp ==FORCE_HS_EXCEP ; +} + +void riscv_cpu_set_force_hs_excep (CPURISCVState *env,bool enable) +{ +if (!riscv_has_ext (env,RVH )) { +return ; + } + + env->virt &= ~FORCE_HS_EXCEP_MASK ; + env->virt |= enable <<FORCE_HS_EXCEP_SHIFT ; +} +int riscv_cpu_claim_interrupts (RISCVCPU *cpu,uint32_t interrupts) {CPURISCVState *env = &cpu->env;
Signature · 1 line
2.21.0
PATCH 5/27 target/riscv: Add the Hypervisor CSRs to CPUState
In reply to Alistair Francis--- target/riscv/cpu.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
target/riscv/cpu.h +17 -0
@@ -169,12 +169,29 @@ struct CPURISCVState {target_ulong mcause;target_ulong mtval;/* since: priv-1.10.0 */ +/* Hypervisor CSRs */ +target_ulong hstatus; +target_ulong hedeleg; +target_ulong hideleg; +target_ulong hgatp; +target_ulong scounteren;target_ulong mcounteren;target_ulong sscratch;target_ulong mscratch; +/* Background CSRs */ +target_ulong bsstatus; +target_ulong bsip; +target_ulong bsie; +target_ulong bstvec; +target_ulong bsscratch; +target_ulong bsepc; +target_ulong bscause; +target_ulong bstval; +target_ulong bsatp; +/* temporary htif regs */ uint64_t mfromhost;uint64_t mtohost;
Signature · 1 line
2.21.0
PATCH 6/27 target/riscv: Dump Hypervisor registers if enabled
In reply to Alistair Francis--- target/riscv/cpu.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
target/riscv/cpu.c +27 -0
@@ -220,14 +220,41 @@ static void riscv_cpu_dump_state(CPUState *cs, FILE *f, int flags)#ifndef CONFIG_USER_ONLY qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"mhartid " , env->mhartid);qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"mstatus " , env->mstatus); +if (riscv_has_ext (env,RVH )) { +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"hstatus " , env->hstatus); +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"bstatus " , env->bsstatus); + }qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"mip " , (target_ulong )atomic_read (&env->mip)); +if (riscv_has_ext (env,RVH )) { +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"bsip " , + (target_ulong )atomic_read (&env->bsip)); + }qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"mie " , env->mie); +if (riscv_has_ext (env,RVH )) { +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"bsie " , env->bsie); + }qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"mideleg " , env->mideleg); +if (riscv_has_ext (env,RVH )) { +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"hideleg " , env->hideleg); + }qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"medeleg " , env->medeleg); +if (riscv_has_ext (env,RVH )) { +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"hedeleg " , env->hedeleg); + }qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"mtvec " , env->mtvec); +if (riscv_has_ext (env,RVH )) { +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"bstvec " , env->bstvec); + }qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"mepc " , env->mepc); +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"sepc " , env->sepc); +if (riscv_has_ext (env,RVH )) { +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"bsepc " , env->bsepc); + }qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"mcause " , env->mcause); +if (riscv_has_ext (env,RVH )) { +qemu_fprintf (f," %s " TARGET_FMT_lx"\n" ,"bscause " , env->bscause); + }#endif for (i =0 ; i <32 ; i++) {
Signature · 1 line
2.21.0
PATCH 7/27 target/riscv: Remove strict perm checking for CSR R/W
In reply to Alistair FrancisThe privledge check based on the CSR address mask 0x300 doesn't work when using Hypervisor extensions so remove the check
--- target/riscv/csr.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
target/riscv/csr.c +1 -2
@@ -776,9 +776,8 @@ int riscv_csrrw(CPURISCVState *env, int csrno, target_ulong *ret_value,/* check privileges and return -1 if check fails */ #if !defined(CONFIG_USER_ONLY ) -int csr_priv =get_field (csrno,0x300 );int read_only =get_field (csrno,0xC00 ) ==3 ; -if ((write_mask && read_only) || (env->priv < csr_priv)) { +if (write_mask && read_only) {return -1 ; }#endif
Signature · 1 line
2.21.0
PATCH 8/27 target/riscv: Create function to test if FP is enabled
In reply to Alistair FrancisLet's creaate a function that tests if floating point support is enabled. We can then protect all floating point operations based on if they are enabled.
This patch so far doesn't change anything, it's just preparing for the Hypervisor support for floating point operations.
--- target/riscv/cpu.h | 6 +++++- target/riscv/cpu_helper.c | 10 ++++++++++ target/riscv/csr.c | 22 +++++++++++++--------- 3 files changed, 28 insertions(+), 10 deletions(-)
target/riscv/cpu.h +5 -1
@@ -301,6 +301,7 @@ bool riscv_cpu_virt_enabled(CPURISCVState *env);void riscv_cpu_set_virt_enabled (CPURISCVState *env,bool enable);bool riscv_cpu_force_hs_excep_enabled (CPURISCVState *env);void riscv_cpu_set_force_hs_excep (CPURISCVState *env,bool enable); +bool riscv_cpu_fp_enabled (CPURISCVState *env);int riscv_cpu_mmu_index (CPURISCVState *env,bool ifetch);hwaddr riscv_cpu_get_phys_page_debug (CPUState *cpu,vaddr addr); void riscv_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
@@ -344,7 +345,10 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,#ifdef CONFIG_USER_ONLY *flags =TB_FLAGS_MSTATUS_FS ;#else - *flags =cpu_mmu_index (env,0 ) | (env->mstatus &MSTATUS_FS ); + *flags =cpu_mmu_index (env,0 ); +if (riscv_cpu_fp_enabled (env)) { + *flags |= env->mstatus &MSTATUS_FS ; + }#endif }
target/riscv/cpu_helper.c +10 -0
@@ -117,6 +117,16 @@ void riscv_cpu_set_force_hs_excep(CPURISCVState *env, bool enable) env->virt |= enable <<FORCE_HS_EXCEP_SHIFT ; } +/* Return true is floating point support is currently enabled */ +bool riscv_cpu_fp_enabled (CPURISCVState *env) +{ +if (env->mstatus &MSTATUS_FS ) { +return true; + } + +return false; +} +int riscv_cpu_claim_interrupts (RISCVCPU *cpu,uint32_t interrupts) {CPURISCVState *env = &cpu->env;
target/riscv/csr.c +13 -9
@@ -46,7 +46,7 @@ void riscv_set_csr_ops(int csrno, riscv_csr_operations *ops)static int fs (CPURISCVState *env,int csrno) {#if !defined(CONFIG_USER_ONLY ) -if (!env->debugger && !(env->mstatus &MSTATUS_FS )) { +if (!env->debugger && !riscv_cpu_fp_enabled (env)) {return -1 ; }#endif
@@ -92,7 +92,7 @@ static int pmp(CPURISCVState *env, int csrno)static int read_fflags (CPURISCVState *env,int csrno,target_ulong *val) {#if !defined(CONFIG_USER_ONLY ) -if (!env->debugger && !(env->mstatus &MSTATUS_FS )) { +if (!env->debugger && !riscv_cpu_fp_enabled (env)) {return -1 ; }#endif
@@ -103,7 +103,7 @@ static int read_fflags(CPURISCVState *env, int csrno, target_ulong *val)static int write_fflags (CPURISCVState *env,int csrno,target_ulong val) {#if !defined(CONFIG_USER_ONLY ) -if (!env->debugger && !(env->mstatus &MSTATUS_FS )) { +if (!env->debugger && !riscv_cpu_fp_enabled (env)) {return -1 ; } env->mstatus |=MSTATUS_FS ;
@@ -115,7 +115,7 @@ static int write_fflags(CPURISCVState *env, int csrno, target_ulong val)static int read_frm (CPURISCVState *env,int csrno,target_ulong *val) {#if !defined(CONFIG_USER_ONLY ) -if (!env->debugger && !(env->mstatus &MSTATUS_FS )) { +if (!env->debugger && !riscv_cpu_fp_enabled (env)) {return -1 ; }#endif
@@ -126,7 +126,7 @@ static int read_frm(CPURISCVState *env, int csrno, target_ulong *val)static int write_frm (CPURISCVState *env,int csrno,target_ulong val) {#if !defined(CONFIG_USER_ONLY ) -if (!env->debugger && !(env->mstatus &MSTATUS_FS )) { +if (!env->debugger && !riscv_cpu_fp_enabled (env)) {return -1 ; } env->mstatus |=MSTATUS_FS ;
@@ -138,7 +138,7 @@ static int write_frm(CPURISCVState *env, int csrno, target_ulong val)static int read_fcsr (CPURISCVState *env,int csrno,target_ulong *val) {#if !defined(CONFIG_USER_ONLY ) -if (!env->debugger && !(env->mstatus &MSTATUS_FS )) { +if (!env->debugger && !riscv_cpu_fp_enabled (env)) {return -1 ; }#endif
@@ -150,7 +150,7 @@ static int read_fcsr(CPURISCVState *env, int csrno, target_ulong *val)static int write_fcsr (CPURISCVState *env,int csrno,target_ulong val) {#if !defined(CONFIG_USER_ONLY ) -if (!env->debugger && !(env->mstatus &MSTATUS_FS )) { +if (!env->debugger && !riscv_cpu_fp_enabled (env)) {return -1 ; } env->mstatus |=MSTATUS_FS ;
@@ -324,8 +324,12 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val) mstatus = (mstatus & ~mask) | (val & mask); -int dirty = ((mstatus &MSTATUS_FS ) ==MSTATUS_FS ) | - ((mstatus &MSTATUS_XS ) ==MSTATUS_XS ); +int dirty = (mstatus &MSTATUS_XS ) ==MSTATUS_XS ; + +if (riscv_cpu_fp_enabled (env)) { + dirty |= (mstatus &MSTATUS_FS ) ==MSTATUS_FS ; + } + mstatus =set_field (mstatus,MSTATUS_SD , dirty); env->mstatus = mstatus;
Signature · 1 line
2.21.0
PATCH 9/27 target/riscv: Add support for background interrupt setting
In reply to Alistair Francis--- target/riscv/cpu_helper.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)
target/riscv/cpu_helper.c +17 -2
@@ -38,12 +38,27 @@ static int riscv_cpu_local_irq_pending(CPURISCVState *env) {target_ulong mstatus_mie =get_field (env->mstatus,MSTATUS_MIE );target_ulong mstatus_sie =get_field (env->mstatus,MSTATUS_SIE ); +target_ulong bsstatus_sie =get_field (env->bsstatus,MSTATUS_SIE ); +target_ulong pending =atomic_read (&env->mip) & env->mie; -target_ulong mie = env->priv <PRV_M || (env->priv ==PRV_M && mstatus_mie); -target_ulong sie = env->priv <PRV_S || (env->priv ==PRV_S && mstatus_sie); +target_ulong hspending =atomic_read (&env->bsip) & env->bsie; + +target_ulong mie = env->priv <PRV_M || (env->priv ==PRV_M && mstatus_mie); +target_ulong sie = env->priv <PRV_S || (env->priv ==PRV_S && mstatus_sie); +target_ulong bsie = env->priv <PRV_S || (env->priv ==PRV_S && bsstatus_sie); +target_ulong irqs = (pending & ~env->mideleg & -mie) | (pending & env->mideleg & -sie); +if (riscv_cpu_virt_enabled (env)) { +target_ulong pending_hs_irq = hspending & -bsie; + +if (pending_hs_irq) { +riscv_cpu_set_force_hs_excep (env,FORCE_HS_EXCEP ); +return ctz64 (pending_hs_irq); + } + } +if (irqs) {return ctz64 (irqs);/* since non-zero */ }else {
Signature · 1 line
2.21.0
PATCH 10/27 target/riscv: Add Hypervisor CSR access functions
In reply to Alistair Francis--- target/riscv/csr.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+)
target/riscv/csr.c +68 -0
@@ -82,6 +82,20 @@ static int smode(CPURISCVState *env, int csrno)return -!riscv_has_ext (env,RVS ); } +static int hmode (CPURISCVState *env,int csrno) +{ +if (riscv_has_ext (env,RVS ) && +riscv_has_ext (env,RVH )) { +/* Hypervisor extension is supported */ +if ((env->priv ==PRV_S && !riscv_cpu_virt_enabled (env)) || + env->priv ==PRV_M ) { +return 0 ; + } + } + +return -1 ; +} +static int pmp (CPURISCVState *env,int csrno) {return -!riscv_feature (env,RISCV_FEATURE_PMP );
@@ -736,6 +750,55 @@ static int write_satp(CPURISCVState *env, int csrno, target_ulong val)return 0 ; } +/* Hypervisor Extensions */ +static int read_hstatus (CPURISCVState *env,int csrno,target_ulong *val) +{ + *val = env->hstatus; +return 0 ; +} + +static int write_hstatus (CPURISCVState *env,int csrno,target_ulong val) +{ + env->hstatus = val; +return 0 ; +} + +static int read_hedeleg (CPURISCVState *env,int csrno,target_ulong *val) +{ + *val = env->hedeleg; +return 0 ; +} + +static int write_hedeleg (CPURISCVState *env,int csrno,target_ulong val) +{ + env->hedeleg = val; +return 0 ; +} + +static int read_hideleg (CPURISCVState *env,int csrno,target_ulong *val) +{ + *val = env->hideleg; +return 0 ; +} + +static int write_hideleg (CPURISCVState *env,int csrno,target_ulong val) +{ + env->hideleg = val; +return 0 ; +} + +static int read_hgatp (CPURISCVState *env,int csrno,target_ulong *val) +{ + *val = env->hgatp; +return 0 ; +} + +static int write_hgatp (CPURISCVState *env,int csrno,target_ulong val) +{ + env->hgatp = val; +return 0 ; +} +/* Physical Memory Protection */ static int read_pmpcfg (CPURISCVState *env,int csrno,target_ulong *val) {
@@ -919,6 +982,11 @@ static riscv_csr_operations csr_ops[CSR_TABLE_SIZE] = { /* Supervisor Protection and Translation */ [CSR_SATP] = { smode, read_satp, write_satp }, + [CSR_HSTATUS] = { hmode, read_hstatus, write_hstatus }, + [CSR_HEDELEG] = { hmode, read_hedeleg, write_hedeleg }, + [CSR_HIDELEG] = { hmode, read_hideleg, write_hideleg }, + [CSR_HGATP] = { hmode, read_hgatp, write_hgatp }, +/* Physical Memory Protection */ [CSR_PMPCFG0 ... CSR_PMPADDR9] = { pmp, read_pmpcfg, write_pmpcfg }, [CSR_PMPADDR0 ... CSR_PMPADDR15] = { pmp, read_pmpaddr, write_pmpaddr },
Signature · 1 line
2.21.0
PATCH 11/27 target/riscv: Add background CSRs accesses
In reply to Alistair Francis--- target/riscv/cpu_bits.h | 11 ++++ target/riscv/csr.c | 119 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 130 insertions(+)
target/riscv/cpu_bits.h +11 -0
@@ -169,6 +169,17 @@#define CSR_SPTBR 0x180#define CSR_SATP 0x180 +/* Background CSRs */ +#define CSR_BSSTATUS 0x200 +#define CSR_BSIE 0x204 +#define CSR_BSTVEC 0x205 +#define CSR_BSSCRATCH 0x240 +#define CSR_BSEPC 0x241 +#define CSR_BSCAUSE 0x242 +#define CSR_BSTVAL 0x243 +#define CSR_BSIP 0x244 +#define CSR_BSATP 0x280 +/* Physical Memory Protection */ #define CSR_PMPCFG0 0x3a0#define CSR_PMPCFG1 0x3a1
target/riscv/csr.c +119 -0
131 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 12/27 target/riscv: Add background register swapping function
In reply to Alistair Francis--- target/riscv/cpu.h | 1 + target/riscv/cpu_bits.h | 5 ++++ target/riscv/cpu_helper.c | 52 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+)
target/riscv/cpu.h +1 -0
@@ -320,6 +320,7 @@ void riscv_cpu_list(void);#define cpu_mmu_index riscv_cpu_mmu_index#ifndef CONFIG_USER_ONLY +void riscv_cpu_swap_background_regs (CPURISCVState *env);int riscv_cpu_claim_interrupts (RISCVCPU *cpu,uint32_t interrupts);uint32_t riscv_cpu_update_mip (RISCVCPU *cpu,uint32_t mask,uint32_t value);#define BOOL_TO_MASK (x) (-!!(x))/* helper for riscv_cpu_update_mip value */
target/riscv/cpu_bits.h +5 -0
@@ -550,3 +550,8 @@#define SIP_SSIP MIP_SSIP#define SIP_STIP MIP_STIP#define SIP_SEIP MIP_SEIP + +/* MIE masks */ +#define MIE_SEIE (1 << IRQ_S_EXT) +#define MIE_STIE (1 << IRQ_S_TIMER) +#define MIE_SSIE (1 << IRQ_S_SOFT)
target/riscv/cpu_helper.c +52 -0
58 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 13/27 target/ricsv: Flush the TLB on virtulisation mode changes
In reply to Alistair FrancisTo ensure our TLB isn't out-of-date we flush it on all virt mode changes. Unlike priv mode this isn't saved in the mmu_idx as all guests share V=1. The easiest option is just to flush on all changes.
--- target/riscv/cpu_helper.c | 5 +++++ 1 file changed, 5 insertions(+)
target/riscv/cpu_helper.c +5 -0
@@ -157,6 +157,11 @@ void riscv_cpu_set_virt_enabled(CPURISCVState *env, bool enable)return ; } +/* Flush the TLB on all virt mode changes. */ +if (((env->virt &VIRT_MODE_MASK ) >>VIRT_MODE_SHIFT ) != enable) { +tlb_flush (CPU (riscv_env_get_cpu (env))); + } + env->virt &= ~VIRT_MODE_MASK ; env->virt |= enable <<VIRT_MODE_SHIFT ; }
Signature · 1 line
2.21.0
PATCH 14/27 target/riscv: Generate illegal instruction on WFI when V=1
In reply to Alistair Francis--- target/riscv/op_helper.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
target/riscv/op_helper.c +3 -2
@@ -130,9 +130,10 @@ void helper_wfi(CPURISCVState *env) {CPUState *cs =CPU (riscv_env_get_cpu (env)); -if (env->priv ==PRV_S && +if ((env->priv ==PRV_S && env->priv_ver >=PRIV_VERSION_1_10_0 && -get_field (env->mstatus,MSTATUS_TW )) { +get_field (env->mstatus,MSTATUS_TW )) || +riscv_cpu_virt_enabled (env)) {riscv_raise_exception (env,RISCV_EXCP_ILLEGAL_INST ,GETPC ()); }else { cs->halted =1 ;
Signature · 1 line
2.21.0
PATCH 15/27 riscv: plic: Remove unused interrupt functions
In reply to Alistair Francis--- hw/riscv/sifive_plic.c | 12 ------------ include/hw/riscv/sifive_plic.h | 3 --- 2 files changed, 15 deletions(-)
hw/riscv/sifive_plic.c +0 -12
@@ -159,18 +159,6 @@ static void sifive_plic_update(SiFivePLICState *plic) } } -void sifive_plic_raise_irq (SiFivePLICState *plic,uint32_t irq) -{ -sifive_plic_set_pending (plic, irq, true); -sifive_plic_update (plic); -} - -void sifive_plic_lower_irq (SiFivePLICState *plic,uint32_t irq) -{ -sifive_plic_set_pending (plic, irq, false); -sifive_plic_update (plic); -} -static uint32_t sifive_plic_claim (SiFivePLICState *plic,uint32_t addrid) {int i, j;
include/hw/riscv/sifive_plic.h +0 -3
@@ -69,9 +69,6 @@ typedef struct SiFivePLICState {uint32_t aperture_size; } SiFivePLICState; -void sifive_plic_raise_irq (SiFivePLICState *plic,uint32_t irq); -void sifive_plic_lower_irq (SiFivePLICState *plic,uint32_t irq); - DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, uint32_t num_sources, uint32_t num_priorities, uint32_t priority_base, uint32_t pending_base,
Signature · 1 line
2.21.0
PATCH 16/27 riscv: plic: Always set sip.SEIP bit for HS
In reply to Alistair FrancisWhen the PLIC generates an interrupt ensure we always set it for the SIP CSR that corresponds to the HS (V=0) register.
--- hw/riscv/sifive_plic.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
hw/riscv/sifive_plic.c +11 -1
@@ -147,7 +147,17 @@ static void sifive_plic_update(SiFivePLICState *plic)riscv_cpu_update_mip (RISCV_CPU (cpu),MIP_MEIP ,BOOL_TO_MASK (level));break ;case PLICMode_S: -riscv_cpu_update_mip (RISCV_CPU (cpu),MIP_SEIP ,BOOL_TO_MASK (level)); +if (riscv_cpu_virt_enabled (env)) { +if (level) { +atomic_or (&env->bsip,MIP_SEIP ); +g_assert (riscv_cpu_virt_enabled (env)); + }else { +atomic_and (&env->bsip, ~MIP_SEIP ); +g_assert (riscv_cpu_virt_enabled (env)); + } + }else { +riscv_cpu_update_mip (RISCV_CPU (cpu),MIP_SEIP ,BOOL_TO_MASK (level)); + }break ;default :break ;
Signature · 1 line
2.21.0
PATCH 17/27 target/riscv: Add hypvervisor trap support
In reply to Alistair Francis--- target/riscv/cpu_bits.h | 4 +-- target/riscv/cpu_helper.c | 71 +++++++++++++++++++++++++++++++++------ target/riscv/csr.c | 4 +-- 3 files changed, 65 insertions(+), 14 deletions(-)
target/riscv/cpu_bits.h +2 -2
@@ -508,8 +508,8 @@#define RISCV_EXCP_STORE_AMO_ADDR_MIS 0x6#define RISCV_EXCP_STORE_AMO_ACCESS_FAULT 0x7#define RISCV_EXCP_U_ECALL 0x8 -#define RISCV_EXCP_S_ECALL 0x9 -#define RISCV_EXCP_H_ECALL 0xa +#define RISCV_EXCP_HS_ECALL 0x9 +#define RISCV_EXCP_VS_ECALL 0xa#define RISCV_EXCP_M_ECALL 0xb#define RISCV_EXCP_INST_PAGE_FAULT 0xc/* since: priv-1.10.0 */ #define RISCV_EXCP_LOAD_PAGE_FAULT 0xd/* since: priv-1.10.0 */
target/riscv/cpu_helper.c +61 -10
101 lines not on this page · raw message
target/riscv/csr.c +2 -2
@@ -239,8 +239,8 @@ static const target_ulong delegable_excps = (1ULL << (RISCV_EXCP_STORE_AMO_ADDR_MIS )) | (1ULL << (RISCV_EXCP_STORE_AMO_ACCESS_FAULT )) | (1ULL << (RISCV_EXCP_U_ECALL )) | - (1ULL << (RISCV_EXCP_S_ECALL )) | - (1ULL << (RISCV_EXCP_H_ECALL )) | + (1ULL << (RISCV_EXCP_VS_ECALL )) | + (1ULL << (RISCV_EXCP_HS_ECALL )) | (1ULL << (RISCV_EXCP_M_ECALL )) | (1ULL << (RISCV_EXCP_INST_PAGE_FAULT )) | (1ULL << (RISCV_EXCP_LOAD_PAGE_FAULT)) |
Signature · 1 line
2.21.0
PATCH 18/27 target/riscv: Add Hypervisor trap return support
In reply to Alistair Francis--- target/riscv/op_helper.c | 66 ++++++++++++++++++++++++++++++++-------- 1 file changed, 54 insertions(+), 12 deletions(-)
target/riscv/op_helper.c +54 -12
@@ -73,6 +73,8 @@ target_ulong helper_csrrc(CPURISCVState *env, target_ulong src,target_ulong helper_sret (CPURISCVState *env,target_ulong cpu_pc_deb) { +target_ulong prev_priv, prev_virt, mstatus; +if (!(env->priv >=PRV_S )) {riscv_raise_exception (env,RISCV_EXCP_ILLEGAL_INST ,GETPC ()); }
@@ -87,16 +89,46 @@ target_ulong helper_sret(CPURISCVState *env, target_ulong cpu_pc_deb)riscv_raise_exception (env,RISCV_EXCP_ILLEGAL_INST ,GETPC ()); } -target_ulong mstatus = env->mstatus; -target_ulong prev_priv =get_field (mstatus,MSTATUS_SPP ); - mstatus =set_field (mstatus, - env->priv_ver >=PRIV_VERSION_1_10_0 ? -MSTATUS_SIE :MSTATUS_UIE << prev_priv, -get_field (mstatus,MSTATUS_SPIE )); - mstatus =set_field (mstatus,MSTATUS_SPIE ,0 ); - mstatus =set_field (mstatus,MSTATUS_SPP ,PRV_U ); + mstatus = env->mstatus; + +if (riscv_has_ext (env,RVH ) && !riscv_cpu_virt_enabled (env)) { +/* We support Hypervisor extensions and virtulisation is disabled */ +target_ulong hstatus = env->hstatus; + + prev_priv =get_field (mstatus,MSTATUS_SPP ); + prev_virt =get_field (hstatus,HSTATUS_SPV ); + + hstatus =set_field (hstatus,HSTATUS_SPV , +get_field (hstatus,HSTATUS_SP2V )); + mstatus =set_field (mstatus,MSTATUS_SPP , +get_field (hstatus,HSTATUS_SP2P )); + hstatus =set_field (hstatus,HSTATUS_SP2V ,0 ); + hstatus =set_field (hstatus,HSTATUS_SP2P ,0 ); + mstatus =set_field (mstatus,SSTATUS_SIE , +get_field (mstatus,SSTATUS_SPIE )); + mstatus =set_field (mstatus,SSTATUS_SPIE ,1 ); + + env->mstatus = mstatus; + env->hstatus = hstatus; + +if (prev_virt ==VIRT_ON ) { +riscv_cpu_swap_background_regs (env); + } + +riscv_cpu_set_virt_enabled (env, prev_virt); + }else { + prev_priv =get_field (mstatus,MSTATUS_SPP ); + + mstatus =set_field (mstatus, + env->priv_ver >=PRIV_VERSION_1_10_0 ? +MSTATUS_SIE :MSTATUS_UIE << prev_priv, +get_field (mstatus,MSTATUS_SPIE )); + mstatus =set_field (mstatus,MSTATUS_SPIE ,0 ); + mstatus =set_field (mstatus,MSTATUS_SPP ,PRV_U ); + env->mstatus = mstatus; + } +riscv_cpu_set_mode (env, prev_priv); - env->mstatus = mstatus;return retpc; }
@@ -114,14 +146,24 @@ target_ulong helper_mret(CPURISCVState *env, target_ulong cpu_pc_deb)target_ulong mstatus = env->mstatus;target_ulong prev_priv =get_field (mstatus,MSTATUS_MPP ); +target_ulong prev_virt =get_field (mstatus,MSTATUS_MPV ); mstatus =set_field (mstatus, env->priv_ver >=PRIV_VERSION_1_10_0 ?MSTATUS_MIE :MSTATUS_UIE << prev_priv,get_field (mstatus,MSTATUS_MPIE )); - mstatus =set_field (mstatus,MSTATUS_MPIE ,0 ); - mstatus =set_field (mstatus,MSTATUS_MPP ,PRV_U ); -riscv_cpu_set_mode (env, prev_priv); + mstatus =set_field (mstatus,MSTATUS_MPIE ,1 ); + mstatus =set_field (mstatus,MSTATUS_MPP ,0 ); + mstatus =set_field (mstatus,MSTATUS_MPV ,0 ); env->mstatus = mstatus; +riscv_cpu_set_mode (env, prev_priv); + +if (riscv_has_ext (env,RVH )) { +if (prev_virt ==VIRT_ON ) { +riscv_cpu_swap_background_regs (env); + } + +riscv_cpu_set_virt_enabled (env, prev_virt); + }return retpc; }
Signature · 1 line
2.21.0
PATCH 19/27 target/riscv: Add hfence instructions
In reply to Alistair Francis--- target/riscv/insn32.decode | 23 ++++++----- .../riscv/insn_trans/trans_privileged.inc.c | 40 +++++++++++++++++++ 2 files changed, 54 insertions(+), 9 deletions(-)
target/riscv/insn32.decode +14 -9
@@ -63,20 +63,25 @@ @r2_rm ....... ..... ..... ... ..... ....... %rs1 %rm %rd @r2 ....... ..... ..... ... ..... ....... %rs1 %rd +@hfence_gvma ....... ..... ..... ... ..... ....... %rs2 %rs1 +@hfence_bvma ....... ..... ..... ... ..... ....... %rs2 %rs1 + @sfence_vma ....... ..... ..... ... ..... ....... %rs2 %rs1 @sfence_vm ....... ..... ..... ... ..... ....... %rs1 # *** Privileged Instructions *** -ecall 000000000000 00000 000 00000 1110011 -ebreak 000000000001 00000 000 00000 1110011 -uret 0000000 00010 00000 000 00000 1110011 -sret 0001000 00010 00000 000 00000 1110011 -hret 0010000 00010 00000 000 00000 1110011 -mret 0011000 00010 00000 000 00000 1110011 -wfi 0001000 00101 00000 000 00000 1110011 -sfence_vma 0001001 ..... ..... 000 00000 1110011 @sfence_vma -sfence_vm 0001000 00100 ..... 000 00000 1110011 @sfence_vm +ecall 000000000000 00000 000 00000 1110011 +ebreak 000000000001 00000 000 00000 1110011 +uret 0000000 00010 00000 000 00000 1110011 +sret 0001000 00010 00000 000 00000 1110011 +hret 0010000 00010 00000 000 00000 1110011 +mret 0011000 00010 00000 000 00000 1110011 +wfi 0001000 00101 00000 000 00000 1110011 +hfence_gvma 1010001 ..... ..... 000 00000 1110011 @hfence_gvma +hfence_bvma 0010001 ..... ..... 000 00000 1110011 @hfence_bvma +sfence_vma 0001001 ..... ..... 000 00000 1110011 @sfence_vma +sfence_vm 0001000 00100 ..... 000 00000 1110011 @sfence_vm # *** RV32I Base Instruction Set *** lui .................... ..... 0110111 @u
target/riscv/insn_trans/trans_privileged.inc.c +40 -0
@@ -108,3 +108,43 @@ static bool trans_sfence_vm(DisasContext *ctx, arg_sfence_vm *a)#endif return false; } + +static bool trans_hfence_gvma (DisasContext *ctx,arg_sfence_vma *a) +{ +#ifndef CONFIG_USER_ONLY +if (ctx->priv_ver ==PRIV_VERSION_1_10_0 && +has_ext (ctx,RVH )) { +/* Hpervisor extensions exist */ +/* +* if (env->priv == PRV_M || +* (env->priv == PRV_S && +* !riscv_cpu_virt_enabled(env) && +* get_field(ctx->mstatus_fs, MSTATUS_TVM))) { +*/ +gen_helper_tlb_flush (cpu_env); +return true; +/* } */ + } +#endif +return false; +} + +static bool trans_hfence_bvma (DisasContext *ctx,arg_sfence_vma *a) +{ +#ifndef CONFIG_USER_ONLY +if (ctx->priv_ver ==PRIV_VERSION_1_10_0 && +has_ext (ctx,RVH )) { +/* Hpervisor extensions exist */ +/* +* if (env->priv == PRV_M || +* (env->priv == PRV_S && +* !riscv_cpu_virt_enabled(env) && +* get_field(ctx->mstatus_fs, MSTATUS_TVM))) { +*/ +gen_helper_tlb_flush (cpu_env); +return true; +/* } */ + } +#endif +return false; +}
Signature · 1 line
2.21.0
PATCH 20/27 target/riscv: Disable guest FP support based on backgrond status
In reply to Alistair FrancisWhen the Hypervisor extension is in use we only enable floating point support when both status and bsstatus have enabled floating point support.
--- target/riscv/cpu_helper.c | 3 +++ 1 file changed, 3 insertions(+)
target/riscv/cpu_helper.c +3 -0
@@ -193,6 +193,9 @@ void riscv_cpu_set_force_hs_excep(CPURISCVState *env, bool enable)bool riscv_cpu_fp_enabled (CPURISCVState *env) {if (env->mstatus &MSTATUS_FS ) { +if (riscv_cpu_virt_enabled (env) && !(env->bsstatus &MSTATUS_FS )) { +return false; + }return true; }
Signature · 1 line
2.21.0
PATCH 21/27 target/riscv: Mark both sstatus and bsstatus as dirty
In reply to Alistair FrancisMark both sstatus and bsstatus as dirty (3).
--- target/riscv/translate.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
target/riscv/translate.c +12 -0
@@ -44,6 +44,7 @@ typedef struct DisasContext {/* pc_succ_insn points to the instruction following base.pc_next */ target_ulong pc_succ_insn;target_ulong priv_ver; +bool virt_enabled;uint32_t opcode;uint32_t mstatus_fs;uint32_t misa;
@@ -397,6 +398,12 @@ static void mark_fs_dirty(DisasContext *ctx)tcg_gen_ld_tl (tmp, cpu_env, offsetof(CPURISCVState , mstatus));tcg_gen_ori_tl (tmp, tmp,MSTATUS_FS );tcg_gen_st_tl (tmp, cpu_env, offsetof(CPURISCVState , mstatus)); + +if (ctx->virt_enabled) { +tcg_gen_ld_tl (tmp, cpu_env, offsetof(CPURISCVState , bsstatus)); +tcg_gen_ori_tl (tmp, tmp,MSTATUS_FS ); +tcg_gen_st_tl (tmp, cpu_env, offsetof(CPURISCVState , bsstatus)); + } tcg_temp_free(tmp); }#else
@@ -757,6 +764,11 @@ static void riscv_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs) ctx->mem_idx = ctx->base.tb->flags &TB_FLAGS_MMU_MASK ; ctx->mstatus_fs = ctx->base.tb->flags &TB_FLAGS_MSTATUS_FS ; ctx->priv_ver = env->priv_ver; +#if !defined(CONFIG_USER_ONLY ) + ctx->virt_enabled =riscv_cpu_virt_enabled (env); +#else + ctx->virt_enabled = false; +#endif ctx->misa = env->misa; ctx->frm =-1 ;/* unknown rounding mode */ }
Signature · 1 line
2.21.0
PATCH 22/27 target/riscv: Respect MPRV and SPRV for floating point ops
In reply to Alistair FrancisRespect the contents of MSTATUS.MPRV and HSTATUS.SPRV when performing floating point operations when V=0.
--- target/riscv/translate.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
target/riscv/translate.c +15 -1
@@ -765,7 +765,21 @@ static void riscv_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs) ctx->mstatus_fs = ctx->base.tb->flags &TB_FLAGS_MSTATUS_FS ; ctx->priv_ver = env->priv_ver;#if !defined(CONFIG_USER_ONLY ) - ctx->virt_enabled =riscv_cpu_virt_enabled (env); +if (riscv_has_ext (env,RVH )) { + ctx->virt_enabled =riscv_cpu_virt_enabled (env); +if (env->priv_ver ==PRV_M && +get_field (env->mstatus,MSTATUS_MPRV ) && +get_field (env->mstatus,MSTATUS_MPV )) { + ctx->virt_enabled = true; + }else if (env->priv ==PRV_S && + !riscv_cpu_virt_enabled (env) && +get_field (env->hstatus,HSTATUS_SPRV ) && +get_field (env->hstatus,HSTATUS_SPV )) { + ctx->virt_enabled = true; + } + }else { + ctx->virt_enabled = false; + }#else ctx->virt_enabled = false;#endif
Signature · 1 line
2.21.0
PATCH 23/27 target/riscv: Allow specifying MMU stage
In reply to Alistair Francis--- target/riscv/cpu_helper.c | 35 +++++++++++++++++++++++++++-------- 1 file changed, 27 insertions(+), 8 deletions(-)
target/riscv/cpu_helper.c +27 -8
@@ -274,10 +274,19 @@ void riscv_cpu_set_mode(CPURISCVState *env, target_ulong newpriv)* * Adapted from Spike's mmu_t::translate and mmu_t::walk * +* @env: CPURISCVState +* @physical: This will be set to the calculated physical address +* @prot: The returned protection attributes +* @addr: The virtual address to be translated +* @access_type: The type of MMU access +* @mmu_idx: Indicates current privilege level +* @first_stage: Are we in first stage translation? +* Second stage is used for hypervisor guest translation */ static int get_physical_address (CPURISCVState *env,hwaddr *physical,int *prot,target_ulong addr, -int access_type,int mmu_idx) +int access_type,int mmu_idx, +bool first_stage) {/* NOTE: the env->pc value visible here will not be * correct, but the value visible to the exception handler
@@ -468,12 +477,20 @@ restart: }static void raise_mmu_exception (CPURISCVState *env,target_ulong address, -MMUAccessType access_type) +MMUAccessType access_type,bool first_stage) {CPUState *cs =CPU (riscv_env_get_cpu (env)); -int page_fault_exceptions = - (env->priv_ver >=PRIV_VERSION_1_10_0 ) && -get_field (env->satp,SATP_MODE ) !=VM_1_10_MBARE ; +int page_fault_exceptions; +if (first_stage) { + page_fault_exceptions = + (env->priv_ver >=PRIV_VERSION_1_10_0 ) && +get_field (env->satp,SATP_MODE ) !=VM_1_10_MBARE ; +riscv_cpu_set_force_hs_excep (env,CLEAR_HS_EXCEP ); + }else { + page_fault_exceptions = +get_field (env->hgatp,HGATP_MODE ) !=VM_1_10_MBARE ; +riscv_cpu_set_force_hs_excep (env,FORCE_HS_EXCEP ); + }switch (access_type) {case MMU_INST_FETCH : cs->exception_index = page_fault_exceptions ?
@@ -500,7 +517,8 @@ hwaddr riscv_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)int prot;int mmu_idx =cpu_mmu_index (&cpu->env, false); -if (get_physical_address (&cpu->env, &phys_addr, &prot, addr,0 , mmu_idx)) { +if (get_physical_address (&cpu->env, &phys_addr, &prot, addr,0 , mmu_idx, + true)) {return -1 ; }return phys_addr;
@@ -560,7 +578,8 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,qemu_log_mask (CPU_LOG_MMU ,"%s ad %" VADDR_PRIx" rw %d mmu_idx %d\n" , __func__, address, access_type, mmu_idx); - ret =get_physical_address (env, &pa, &prot, address, access_type, mmu_idx); + ret =get_physical_address (env, &pa, &prot, address, access_type, mmu_idx, + true);qemu_log_mask (CPU_LOG_MMU , "%s address=%" VADDR_PRIx " ret %d physical " TARGET_FMT_plx
@@ -577,7 +596,7 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size, } else if (probe) {return false; }else { -raise_mmu_exception (env, address, access_type); +raise_mmu_exception (env, address, access_type, true);riscv_raise_exception (env, cs->exception_index, retaddr); }#else
Signature · 1 line
2.21.0
PATCH 24/27 target/riscv: Allow specifying number of MMU stages
In reply to Alistair Francis--- target/riscv/cpu_helper.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
target/riscv/cpu_helper.c +4 -3
@@ -286,7 +286,7 @@ void riscv_cpu_set_mode(CPURISCVState *env, target_ulong newpriv)static int get_physical_address (CPURISCVState *env,hwaddr *physical,int *prot,target_ulong addr,int access_type,int mmu_idx, -bool first_stage) +bool first_stage,bool two_stage) {/* NOTE: the env->pc value visible here will not be * correct, but the value visible to the exception handler
@@ -518,9 +518,10 @@ hwaddr riscv_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)int mmu_idx =cpu_mmu_index (&cpu->env, false);if (get_physical_address (&cpu->env, &phys_addr, &prot, addr,0 , mmu_idx, - true)) { + true, false)) {return -1 ; } +return phys_addr; }
@@ -579,7 +580,7 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size, __func__, address, access_type, mmu_idx); ret =get_physical_address (env, &pa, &prot, address, access_type, mmu_idx, - true); + true, false);qemu_log_mask (CPU_LOG_MMU , "%s address=%" VADDR_PRIx " ret %d physical " TARGET_FMT_plx
Signature · 1 line
2.21.0
PATCH 25/27 target/riscv: Implement second stage MMU
In reply to Alistair Francis--- target/riscv/cpu_helper.c | 87 +++++++++++++++++++++++++++++++++++---- 1 file changed, 78 insertions(+), 9 deletions(-)
target/riscv/cpu_helper.c +78 -9
@@ -293,13 +293,40 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical,* (riscv_cpu_do_interrupt) is correct */ int mode = mmu_idx; - +bool use_background = false; + +/* +* Check if we should use the background registers for the two +* stage translation. We don't need to check if we actually need +* two stage translation as that happened before this function +* was called. Background registers will be used if the guest has +* forced a two stage translation to be on (in HS or M mode). +*/ if (mode ==PRV_M && access_type !=MMU_INST_FETCH ) {if (get_field (env->mstatus,MSTATUS_MPRV )) { mode =get_field (env->mstatus,MSTATUS_MPP ); + +if (riscv_has_ext (env,RVH ) && +get_field (env->mstatus,MSTATUS_MPV )) { + use_background = true; + } + } + } + +if (mode ==PRV_S && access_type !=MMU_INST_FETCH && +riscv_has_ext (env,RVH ) && !riscv_cpu_virt_enabled (env)) { +if (get_field (env->hstatus,HSTATUS_SPRV )) { + mode =get_field (env->mstatus,SSTATUS_SPP ); + use_background = true; } } +if (first_stage == false) { +/* We are in stage 2 translation, this is similar to stage 1. */ +/* Stage 2 is always taken as U-mode */ + mode =PRV_U ; + } +if (mode ==PRV_M || !riscv_feature (env,RISCV_FEATURE_MMU )) { *physical = addr; *prot =PAGE_READ |PAGE_WRITE |PAGE_EXEC ;
@@ -309,13 +336,30 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical, *prot =0 ;target_ulong base; -int levels, ptidxbits, ptesize, vm, sum; -int mxr =get_field (env->mstatus,MSTATUS_MXR ); +int levels, ptidxbits, ptesize, vm, sum, mxr, widened; + +if (first_stage == true) { + mxr =get_field (env->mstatus,MSTATUS_MXR ); + }else { + mxr =get_field (env->bsstatus,MSTATUS_MXR ); + }if (env->priv_ver >=PRIV_VERSION_1_10_0 ) { - base =get_field (env->satp,SATP_PPN ) <<PGSHIFT ; +if (first_stage == true) { +if (use_background) { + base =get_field (env->bsatp,SATP_PPN ) <<PGSHIFT ; + vm =get_field (env->bsatp,SATP_MODE ); + }else { + base =get_field (env->satp,SATP_PPN ) <<PGSHIFT ; + vm =get_field (env->satp,SATP_MODE ); + } + widened =0 ; + }else { + base =get_field (env->hgatp,HGATP_PPN ) <<PGSHIFT ; + vm =get_field (env->hgatp,HGATP_MODE ); + widened =2 ; + } sum =get_field (env->mstatus,MSTATUS_SUM ); - vm =get_field (env->satp,SATP_MODE );switch (vm) {case VM_1_10_SV32 : levels =2 ; ptidxbits =10 ; ptesize =4 ;break ;
@@ -333,6 +377,7 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical,g_assert_not_reached (); } }else { + widened =0 ; base = env->sptbr <<PGSHIFT ; sum = !get_field (env->mstatus,MSTATUS_PUM ); vm =get_field (env->mstatus,MSTATUS_VM );
@@ -353,7 +398,7 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical, }CPUState *cs =CPU (riscv_env_get_cpu (env)); -int va_bits =PGSHIFT + levels * ptidxbits; +int va_bits =PGSHIFT + levels * ptidxbits + widened;target_ulong mask = (1L << (TARGET_LONG_BITS - (va_bits -1 ))) -1 ;target_ulong masked_msbs = (addr >> (va_bits -1 )) & mask;if (masked_msbs !=0 && masked_msbs != mask) {
@@ -367,11 +412,30 @@ static int get_physical_address(CPURISCVState *env, hwaddr *physical, restart:#endif for (i =0 ; i < levels; i++, ptshift -= ptidxbits) { -target_ulong idx = (addr >> (PGSHIFT + ptshift)) & +target_ulong idx; +if (i ==0 ) { + idx = (addr >> (PGSHIFT + ptshift)) & + ((1 << (ptidxbits + widened)) -1 ); + }else { + idx = (addr >> (PGSHIFT + ptshift)) & ((1 << ptidxbits) -1 ); + }/* check that physical address of PTE is legal */ -target_ulong pte_addr = base + idx * ptesize; +target_ulong pte_addr; + +if (two_stage && first_stage) { +hwaddr vbase; + +/* Do the second stage translation on the base PTE address. */ +get_physical_address (env, &vbase, prot, base, access_type, + mmu_idx, false, true); + + pte_addr = vbase + idx * ptesize; + }else { + pte_addr = base + idx * ptesize; + } +#if defined(TARGET_RISCV32 )target_ulong pte =ldl_phys (cs->as, pte_addr);#elif defined(TARGET_RISCV64 )
@@ -457,7 +521,12 @@ restart:/* for superpage mappings, make a fake leaf PTE for the TLB's benefit. */ target_ulong vpn = addr >>PGSHIFT ; - *physical = (ppn | (vpn & ((1L << ptshift) -1 ))) <<PGSHIFT ; +if (i ==0 ) { + *physical = (ppn | (vpn & ((1L << (ptshift + widened)) -1 ))) << +PGSHIFT ; + }else { + *physical = (ppn | (vpn & ((1L << ptshift) -1 ))) <<PGSHIFT ; + }/* set permissions on the TLB entry */ if ((pte &PTE_R ) || ((pte &PTE_X ) && mxr)) {
Signature · 1 line
2.21.0
PATCH 26/27 target/riscv: Call the second stage MMU in virtualisation mode
In reply to Alistair FrancisThe qemu_log_mask(CPU_LOG_MMU,... calls trigger false positive checkpatch errors which are being ignored.
--- target/riscv/cpu_helper.c | 118 ++++++++++++++++++++++++++++++++------ 1 file changed, 99 insertions(+), 19 deletions(-)
target/riscv/cpu_helper.c +99 -19
@@ -582,15 +582,23 @@ static void raise_mmu_exception(CPURISCVState *env, target_ulong address,hwaddr riscv_cpu_get_phys_page_debug (CPUState *cs,vaddr addr) {RISCVCPU *cpu =RISCV_CPU (cs); +CPURISCVState *env = &cpu->env;hwaddr phys_addr;int prot;int mmu_idx =cpu_mmu_index (&cpu->env, false); -if (get_physical_address (&cpu->env, &phys_addr, &prot, addr,0 , mmu_idx, - true, false)) { +if (get_physical_address (env, &phys_addr, &prot, addr,0 , mmu_idx, + true,riscv_cpu_virt_enabled (env))) {return -1 ; } +if (riscv_cpu_virt_enabled (env)) { +if (get_physical_address (env, &phys_addr, &prot, phys_addr, +0 , mmu_idx, false, true)) { +return -1 ; + } + } +return phys_addr; }
@@ -641,34 +649,106 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,#ifndef CONFIG_USER_ONLY RISCVCPU *cpu =RISCV_CPU (cs);CPURISCVState *env = &cpu->env; +vaddr im_address;hwaddr pa =0 ;int prot; +bool m_mode_two_stage = false; +bool hs_mode_two_stage = false;int ret =TRANSLATE_FAIL ;qemu_log_mask (CPU_LOG_MMU ,"%s ad %" VADDR_PRIx" rw %d mmu_idx %d\n" , __func__, address, access_type, mmu_idx); - ret =get_physical_address (env, &pa, &prot, address, access_type, mmu_idx, - true, false); +/* +* Determine if we are in M mode and MPRV is set or in HS mode and SPRV is +* set and we want to access a virtulisation address. +*/ +if (riscv_has_ext (env,RVH )) { + m_mode_two_stage = env->priv ==PRV_M && + access_type !=MMU_INST_FETCH && +get_field (env->mstatus,MSTATUS_MPRV ) && +get_field (env->mstatus,MSTATUS_MPV ); + + hs_mode_two_stage = env->priv ==PRV_S && + !riscv_cpu_virt_enabled (env) && + access_type !=MMU_INST_FETCH && +get_field (env->hstatus,HSTATUS_SPRV ) && +get_field (env->hstatus,HSTATUS_SPV ); + } + +if (riscv_cpu_virt_enabled (env) || m_mode_two_stage || hs_mode_two_stage) { +/* Two stage lookup */ + ret =get_physical_address (env, &pa, &prot, address, access_type, + mmu_idx, true, true); -qemu_log_mask (CPU_LOG_MMU , -"%s address=%" VADDR_PRIx" ret %d physical " TARGET_FMT_plx -" prot %d\n" , __func__, address, ret, pa, prot); +qemu_log_mask (CPU_LOG_MMU , +"%s 1st-stage address=%" VADDR_PRIx" ret %d physical " + TARGET_FMT_plx" prot %d\n" , + __func__, address, ret, pa, prot); -if (riscv_feature (env,RISCV_FEATURE_PMP ) && - !pmp_hart_has_privs (env, pa,TARGET_PAGE_SIZE ,1 << access_type)) { - ret =TRANSLATE_FAIL ; - } -if (ret ==TRANSLATE_SUCCESS ) { -tlb_set_page (cs, address &TARGET_PAGE_MASK , pa &TARGET_PAGE_MASK , - prot, mmu_idx,TARGET_PAGE_SIZE ); -return true; - }else if (probe) { -return false; +if (ret ==TRANSLATE_FAIL ) { +if (!probe) { +raise_mmu_exception (env, address, access_type, true); +riscv_raise_exception (env, cs->exception_index, retaddr); + } +return ret; + } + +/* Second stage lookup */ + im_address = pa; + + ret =get_physical_address (env, &pa, &prot, im_address, access_type, mmu_idx, + false, true); + +qemu_log_mask (CPU_LOG_MMU , +"%s 2nd-stage address=%" VADDR_PRIx" ret %d physical " + TARGET_FMT_plx" prot %d\n" , + __func__, im_address, ret, pa, prot); + +if (riscv_feature (env,RISCV_FEATURE_PMP ) && + !pmp_hart_has_privs (env, pa,TARGET_PAGE_SIZE ,1 << access_type)) { + ret =TRANSLATE_FAIL ; + } + +if (ret ==TRANSLATE_FAIL ) { +/* +* Guest physical address translation failed, this is a HS +* level exception +*/ +if (!probe) { +raise_mmu_exception (env, im_address | (address & (TARGET_PAGE_SIZE -1 )), access_type, false); +riscv_raise_exception (env, cs->exception_index, retaddr); + } +return ret; + } }else { -raise_mmu_exception (env, address, access_type, true); -riscv_raise_exception (env, cs->exception_index, retaddr); +/* Single stage lookup */ + ret =get_physical_address (env, &pa, &prot, address, access_type, + mmu_idx, true, false); + +qemu_log_mask (CPU_LOG_MMU , +"%s address=%" VADDR_PRIx" ret %d physical " + TARGET_FMT_plx" prot %d\n" , + __func__, address, ret, pa, prot); + +if (riscv_feature (env,RISCV_FEATURE_PMP ) && + !pmp_hart_has_privs (env, pa,TARGET_PAGE_SIZE ,1 << access_type)) { + ret =TRANSLATE_FAIL ; + } + +if (ret ==TRANSLATE_FAIL ) { +if (!probe) { +raise_mmu_exception (env, address, access_type, true); +riscv_raise_exception (env, cs->exception_index, retaddr); + } +return ret; + } } + +tlb_set_page (cs, address &TARGET_PAGE_MASK , pa &TARGET_PAGE_MASK , + prot, mmu_idx,TARGET_PAGE_SIZE ); +return true; +#else switch (access_type) {case MMU_INST_FETCH :
Signature · 1 line
2.21.0
PATCH 27/27 target/riscv: Allow enabling the Hypervisor extension
In reply to Alistair Francis--- target/riscv/cpu.c | 4 ++++ target/riscv/cpu.h | 1 + 2 files changed, 5 insertions(+)
target/riscv/cpu.c +4 -0
@@ -436,6 +436,9 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)if (cpu->cfg.ext_u) { target_misa |=RVU ; } +if (cpu->cfg.ext_h) { + target_misa |=RVH ; + }set_misa (env,RVXLEN | target_misa); }
@@ -472,6 +475,7 @@ static Property riscv_cpu_properties[] = {DEFINE_PROP_BOOL ("c" ,RISCVCPU , cfg.ext_c, true),DEFINE_PROP_BOOL ("s" ,RISCVCPU , cfg.ext_s, true),DEFINE_PROP_BOOL ("u" ,RISCVCPU , cfg.ext_u, true), +DEFINE_PROP_BOOL ("h" ,RISCVCPU , cfg.ext_h, false),DEFINE_PROP_STRING ("priv_spec" ,RISCVCPU , cfg.priv_spec),DEFINE_PROP_STRING ("user_spec" ,RISCVCPU , cfg.user_spec), DEFINE_PROP_BOOL("mmu", RISCVCPU, cfg.mmu, true),
target/riscv/cpu.h +1 -0
@@ -259,6 +259,7 @@ typedef struct RISCVCPU {bool ext_c;bool ext_s;bool ext_u; +bool ext_h;char *priv_spec;char *user_spec;
Signature · 1 line
2.21.0
Quoted 131 lines · Alistair Francis
This patch series adds the RISC-V Hypervisor extension 0.3. This is the latest draft spec of the Hypervisor extension.
The Hypervisor extension is disabled by default, so this series should result in no changes to anyone using QEMU unless they enable the extension. The extention can be enabled with the -cpu property (see below).
At the moment the spec does not include information about the mstatush register. As it is not in the spec I haven't added it to QEMU. This means the extension won't work correctly for 32-bit guests. This should be a small fix to add the CSR once the spec is updated.
All testing of this implementation has been done by using the baremetal Xvisor Hypervisor. We are able to run two Linux guests (that's all I have tried) as guests.
At the moment this spec is in a draft state and is subject to change. As QEMU is extreamly useful in early bring up I think it makes sense for QEMU to support non-frozen extensions. I would like to decide with this series how QEMU will handle all future non-frozen extensions. That is a standard way that QEMU users can test future RISC-V extensions while still understanding things will change. One idea is just to disable it by default, another option is to maybe use the
Kconfigto make it a compile time option which developers can use. Should we also display a warning when running non-frozen extensions?Thanks to Anup for doing the initial port of Xvisor. The port is avaliable here:
https://github.com/avpatel/xvisor-next and will run on QEMU.Also thanks to Atish for implementing the SBI call support in Xvisor and for lots of help debugging.
To run this yourself:
1. Apply this patch series to QEMU. The latest branch can be found here:https://github.com/alistair23/qemu/tree/mainline/alistair/riscv-hyp-work.next
2. Get the version of OpenSBI that supports the H extension. This can be found here:
https://github.com/riscv/opensbi/tree/hyp_ext_changes_v1
3. Build the next release of Xvisor. It is available here:
https://github.com/avpatel/xvisor-next
4. Make sure you build the Xvisor tests, see here for details:https://github.com/avpatel/xvisor-next/tree/master/tests/riscv/virt64/linux
5. Run QEMU:./riscv64-softmmu/qemu-system-riscv64 -nographic \ -machine virt -cpu rv64,h=true\ -serial mon:stdio -serial null -m 4G \ -device loader,file=vmm.bin,addr=0x80200000 \ -kernel fw_jump.elf \ -initrd vmm-disk-linux.img \ -append "vmm.console=uart@10000000 vmm.bootcmd=\"vfs mount initrd/;vfs run /boot.xscript;vfs cat /system/banner.txt\""
Once you get to the prompt you can start the geust by running:
guest kick guest0
You can then bind to the serial port using:
vserial bind guest0/uart0
Then you can start Linux using:
autoexecThis was all tested with the mainline 5.1 kernel. I don't know if it will work on older kernels.
So far all of the QEMU work has been tested on Xvisor.
Known Issues/TODO:
- Add mstatush to support 32-bit Hypervisors
- Fix the random hang that sometimes appears when running a Hypervisor
guest (~5%)There is also on going work from Anup to port KVM.
We have code complete implementation of RISC-V KVM kernel module and RISC-V KVMTOOL. Currently, we are debugging KVM on QEMU and we will send-out RFC PATCHES for KVM in June/July 2019.
The KVM RISC-V kernel module is available inriscv_kvm_v1branch at: https://github.com/avpatel/linux.git
The KVMTOOL RISC-V port is available inriscv_v1branch of
https://github.com/avpatel/kvmtool.gitThere is very early work on a Xen port as well which is avaliable here:
https://github.com/alistair23/xen/tree/alistair/riscv-portChanges from RFC:
- Rebase on latest master
- Add floating point changes from Hypervisor extensionAlistair Francis (27):
target/riscv: Don't set write permissions on dirty PTEs target/riscv: Add the Hypervisor extension target/riscv: Add the virtulisation mode target/riscv: Add the force HS exception mode target/riscv: Add the Hypervisor CSRs to CPUState target/riscv: Dump Hypervisor registers if enabled target/riscv: Remove strict perm checking for CSR R/W target/riscv: Create function to test if FP is enabled target/riscv: Add support for background interrupt setting target/riscv: Add Hypervisor CSR access functions target/riscv: Add background CSRs accesses target/riscv: Add background register swapping function target/ricsv: Flush the TLB on virtulisation mode changes target/riscv: Generate illegal instruction on WFI when V=1
riscv: plic: Remove unused interrupt functions
riscv: plic: Always set sip.SEIP bit for HS target/riscv: Add hypvervisor trap support target/riscv: Add Hypervisor trap return support target/riscv: Add hfence instructions
target/riscv: Disable guest FP support based on backgrond status target/riscv: Mark both sstatus and bsstatus as dirty target/riscv: Respect MPRV and SPRV for floating point ops target/riscv: Allow specifying MMU stage target/riscv: Allow specifying number of MMU stages target/riscv: Implement second stage MMU target/riscv: Call the second stage MMU in virtualisation mode target/riscv: Allow enabling the Hypervisor extensionhw/riscv/sifive_plic.c | 24 +- include/hw/riscv/sifive_plic.h | 3 - target/riscv/cpu.c | 31 ++ target/riscv/cpu.h | 32 +- target/riscv/cpu_bits.h | 32 +- target/riscv/cpu_helper.c | 443 ++++++++++++++++-- target/riscv/csr.c | 216 ++++++++- target/riscv/insn32.decode | 23 +- .../riscv/insn_trans/trans_privileged.inc.c | 40 ++ target/riscv/op_helper.c | 71 ++- target/riscv/translate.c | 26 + 11 files changed, 839 insertions(+), 102 deletions(-)
Quoted 4 lines
--
2.21.0
tested with Linux kernel v5.2 and related opensbi/xvisor branches
described above.
Quoted 131 lines · Chih-Min Chao
Quoted 127 lines · Alistair Francis
This patch series adds the RISC-V Hypervisor extension 0.3. This is the latest draft spec of the Hypervisor extension.
The Hypervisor extension is disabled by default, so this series should result in no changes to anyone using QEMU unless they enable the extension. The extention can be enabled with the -cpu property (see below).
At the moment the spec does not include information about the mstatush register. As it is not in the spec I haven't added it to QEMU. This means the extension won't work correctly for 32-bit guests. This should be a small fix to add the CSR once the spec is updated.
All testing of this implementation has been done by using the baremetal Xvisor Hypervisor. We are able to run two Linux guests (that's all I have tried) as guests.
At the moment this spec is in a draft state and is subject to change. As QEMU is extreamly useful in early bring up I think it makes sense for QEMU to support non-frozen extensions. I would like to decide with this series how QEMU will handle all future non-frozen extensions. That is a standard way that QEMU users can test future RISC-V extensions while still understanding things will change. One idea is just to disable it by default, another option is to maybe use the
Kconfigto make it a compile time option which developers can use. Should we also display a warning when running non-frozen extensions?Thanks to Anup for doing the initial port of Xvisor. The port is avaliable here:
https://github.com/avpatel/xvisor-next and will run on QEMU.Also thanks to Atish for implementing the SBI call support in Xvisor and for lots of help debugging.
To run this yourself:
1. Apply this patch series to QEMU. The latest branch can be found here:
https://github.com/alistair23/qemu/tree/mainline/alistair/riscv-hyp-work.next
2. Get the version of OpenSBI that supports the H extension. This can be found here:
https://github.com/riscv/opensbi/tree/hyp_ext_changes_v1
3. Build the next release of Xvisor. It is available here:
https://github.com/avpatel/xvisor-next
4. Make sure you build the Xvisor tests, see here for details:
https://github.com/avpatel/xvisor-next/tree/master/tests/riscv/virt64/linux
5. Run QEMU:./riscv64-softmmu/qemu-system-riscv64 -nographic \ -machine virt -cpu rv64,h=true\ -serial mon:stdio -serial null -m 4G \ -device loader,file=vmm.bin,addr=0x80200000 \ -kernel fw_jump.elf \ -initrd vmm-disk-linux.img \ -append "vmm.console=uart@10000000 vmm.bootcmd=\"vfs mount initrd /;vfs run /boot.xscript;vfs cat /system/banner.txt\""Once you get to the prompt you can start the geust by running:
guest kick guest0
You can then bind to the serial port using:
vserial bind guest0/uart0
Then you can start Linux using:
autoexecThis was all tested with the mainline 5.1 kernel. I don't know if it will work on older kernels.
So far all of the QEMU work has been tested on Xvisor.
Known Issues/TODO:
- Add mstatush to support 32-bit Hypervisors
- Fix the random hang that sometimes appears when running a Hypervisor guest (~5%)There is also on going work from Anup to port KVM.
We have code complete implementation of RISC-V KVM kernel module and RISC-V KVMTOOL. Currently, we are debugging KVM on QEMU and we will send-out RFC PATCHES for KVM in June/July 2019.
The KVM RISC-V kernel module is available inriscv_kvm_v1branch at: https://github.com/avpatel/linux.git
The KVMTOOL RISC-V port is available inriscv_v1branch of
https://github.com/avpatel/kvmtool.gitThere is very early work on a Xen port as well which is avaliable here:
https://github.com/alistair23/xen/tree/alistair/riscv-portChanges from RFC:
- Rebase on latest master
- Add floating point changes from Hypervisor extensionAlistair Francis (27):
target/riscv: Don't set write permissions on dirty PTEs target/riscv: Add the Hypervisor extension target/riscv: Add the virtulisation mode target/riscv: Add the force HS exception mode target/riscv: Add the Hypervisor CSRs to CPUState target/riscv: Dump Hypervisor registers if enabled target/riscv: Remove strict perm checking for CSR R/W target/riscv: Create function to test if FP is enabled target/riscv: Add support for background interrupt setting target/riscv: Add Hypervisor CSR access functions target/riscv: Add background CSRs accesses target/riscv: Add background register swapping function target/ricsv: Flush the TLB on virtulisation mode changes target/riscv: Generate illegal instruction on WFI when V=1
riscv: plic: Remove unused interrupt functions
riscv: plic: Always set sip.SEIP bit for HS target/riscv: Add hypvervisor trap support target/riscv: Add Hypervisor trap return support target/riscv: Add hfence instructions
target/riscv: Disable guest FP support based on backgrond status target/riscv: Mark both sstatus and bsstatus as dirty target/riscv: Respect MPRV and SPRV for floating point ops target/riscv: Allow specifying MMU stage target/riscv: Allow specifying number of MMU stages target/riscv: Implement second stage MMU target/riscv: Call the second stage MMU in virtualisation mode target/riscv: Allow enabling the Hypervisor extensionhw/riscv/sifive_plic.c | 24 +- include/hw/riscv/sifive_plic.h | 3 - target/riscv/cpu.c | 31 ++ target/riscv/cpu.h | 32 +- target/riscv/cpu_bits.h | 32 +- target/riscv/cpu_helper.c | 443 ++++++++++++++++-- target/riscv/csr.c | 216 ++++++++- target/riscv/insn32.decode | 23 +- .../riscv/insn_trans/trans_privileged.inc.c | 40 ++ target/riscv/op_helper.c | 71 ++- target/riscv/translate.c | 26 + 11 files changed, 839 insertions(+), 102 deletions(-)
Quoted 7 lines
Quoted 3 lines
--
2.21.0tested with Linux kernel v5.2 and related opensbi/xvisor branches described above.
Tested-by Chih-Min Chao <[email protected]>
Thanks for testing!!!
Unfortunately I don't think this is going to be merged. It is too late to have this in the 4.1 release and I'm about to start working on upgrading this to the v0.4 Hypervisor spec. So hopefully we can have that in 4.2.
Alistair
Quoted 1 line
Quoted 9 lines · Alistair Francis
At the moment this spec is in a draft state and is subject to change. As QEMU is extreamly useful in early bring up I think it makes sense for QEMU to support non-frozen extensions. I would like to decide with this series how QEMU will handle all future non-frozen extensions. That is a standard way that QEMU users can test future RISC-V extensions while still understanding things will change. One idea is just to disable it by default, another option is to maybe use the
Kconfigto make it a compile time option which developers can use. Should we also display a warning when running non-frozen extensions?
We had an instance of this for Arm (though in fact the relevant patches to QEMU didn't end up getting into master before the spec was finalized in the end). My suggestion would be at minimum:
* by default non-frozen extensions should not be provided
* they should be enabled via a command line option (cpu property) whose name starts with "x-", which is our standard way of flagging properties that are experimental and subject to change or removal in future QEMU versions
That way end-users know they're doing something non-standard that won't necessarily be supported in future by newer versions of QEMU, and if people copy recipes/commandlines/random guest images off old blog posts there'll be a hint that there's a reason why they don't work on newer QEMU that adheres to the final spec.
thanks
-- PMM
Quoted 12 lines · Peter Maydell
Quoted 9 lines · Alistair Francis
At the moment this spec is in a draft state and is subject to change. As QEMU is extreamly useful in early bring up I think it makes sense for QEMU to support non-frozen extensions. I would like to decide with this series how QEMU will handle all future non-frozen extensions. That is a standard way that QEMU users can test future RISC-V extensions while still understanding things will change. One idea is just to disable it by default, another option is to maybe use the
Kconfigto make it a compile time option which developers can use. Should we also display a warning when running non-frozen extensions?
Quoted 5 lines
We had an instance of this for Arm (though in fact the relevant patches to QEMU didn't end up getting into master before the spec was finalized in the end). My suggestion would be at minimum:
* by default non-frozen extensions should not be provided
Yep, these are off by default.
Quoted 4 lines
* they should be enabled via a command line option (cpu property) whose name starts with "x-", which is our standard way of flagging properties that are experimental and subject to change or removal in future QEMU versions
Sounds good, I'll rename the property in the next version.
Alistair
Quoted 10 lines
That way end-users know they're doing something non-standard that won't necessarily be supported in future by newer versions of QEMU, and if people copy recipes/commandlines/random guest images off old blog posts there'll be a hint that there's a reason why they don't work on newer QEMU that adheres to the final spec.
thanks
-- PMM
Quoted 26 lines · Alistair Francis
Quoted 2 lines · Peter Maydell
On Fri, 7 Jun 2019 at 23:03, Alistair Francis <[email protected]>
Quoted 1 line · wrote:
Quoted 1 line
At the moment this spec is in a draft state and is subject to change.
As
Quoted 5 lines
Quoted 5 lines
QEMU is extreamly useful in early bring up I think it makes sense for QEMU to support non-frozen extensions. I would like to decide with this series how QEMU will handle all future non-frozen extensions. That is a standard way that QEMU users can test future RISC-V extensions while still understanding things will change. One idea is just to disable it
by
Quoted 1 line
Quoted 1 line
default, another option is to maybe use the
Kconfigto make it acompile
Quoted 8 lines
Quoted 2 lines
time option which developers can use. Should we also display a warning when running non-frozen extensions?
We had an instance of this for Arm (though in fact the relevant patches to QEMU didn't end up getting into master before the spec was finalized in the end). My suggestion would be at minimum:
* by default non-frozen extensions should not be providedYep, these are off by default.
Quoted 20 lines
Quoted 4 lines
* they should be enabled via a command line option (cpu property) whose name starts with "x-", which is our standard way of flagging properties that are experimental and subject to change or removal in future QEMU versions
Sounds good, I'll rename the property in the next version.
Alistair
Quoted 10 lines
That way end-users know they're doing something non-standard that won't necessarily be supported in future by newer versions of QEMU, and if people copy recipes/commandlines/random guest images off old blog posts there'll be a hint that there's a reason why they don't work on newer QEMU that adheres to the final spec.
thanks
-- PMM
Hi Peter,
I agree that unfrozen extension should't be merged into master but I think the patch set is more like RFC version.
Some part of the patches have been merged in into master and will be available in 4.1. So my intention is to be
familiar with present h-extension by reviewing the Alistair's great work and help the work when new patch set are ready
for 4.2 cycle.
chihmin.chao