Skip to content

Add RISC-V Hypervisor Extension

Alistair Francis qemu-devel qemu-riscv 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest27 patches 5 repliesIn review

Changes from RFC from the cover letter

- Rebase on latest master
- Add floating point changes from Hypervisor extension

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/27target/riscv: Don't set write permissions on dirty PTEstarget/riscv/cpu_helper.c+2 -40 replies
2/27target/riscv: Add the Hypervisor extensiontarget/riscv/cpu.h+1 -00 replies
3/27target/riscv: Add the virtulisation modetarget/riscv/cpu.h target/riscv/cpu_bits.h target/riscv/cpu_helper.c+33 -00 replies
4/27target/riscv: Add the force HS exception modetarget/riscv/cpu.h target/riscv/cpu_bits.h target/riscv/cpu_helper.c+31 -00 replies
5/27target/riscv: Add the Hypervisor CSRs to CPUStatetarget/riscv/cpu.h+17 -00 replies
6/27target/riscv: Dump Hypervisor registers if enabledtarget/riscv/cpu.c+27 -00 replies
7/27target/riscv: Remove strict perm checking for CSR R/Wtarget/riscv/csr.c+1 -20 replies
8/27target/riscv: Create function to test if FP is enabledtarget/riscv/cpu.h target/riscv/cpu_helper.c target/riscv/csr.c+28 -100 replies
9/27target/riscv: Add support for background interrupt settingtarget/riscv/cpu_helper.c+17 -20 replies
10/27target/riscv: Add Hypervisor CSR access functionstarget/riscv/csr.c+68 -00 replies
11/27target/riscv: Add background CSRs accessestarget/riscv/cpu_bits.h target/riscv/csr.c+130 -00 replies
12/27target/riscv: Add background register swapping functiontarget/riscv/cpu.h target/riscv/cpu_bits.h target/riscv/cpu_helper.c+58 -00 replies
13/27target/ricsv: Flush the TLB on virtulisation mode changestarget/riscv/cpu_helper.c+5 -00 replies
14/27target/riscv: Generate illegal instruction on WFI when V=1target/riscv/op_helper.c+3 -20 replies
15/27riscv: plic: Remove unused interrupt functionshw/riscv/sifive_plic.c include/hw/riscv/sifive_plic.h+0 -150 replies
16/27riscv: plic: Always set sip.SEIP bit for HShw/riscv/sifive_plic.c+11 -10 replies
17/27target/riscv: Add hypvervisor trap supporttarget/riscv/cpu_bits.h target/riscv/cpu_helper.c target/riscv/csr.c+65 -140 replies
18/27target/riscv: Add Hypervisor trap return supporttarget/riscv/op_helper.c+54 -120 replies
19/27target/riscv: Add hfence instructionstarget/riscv/insn32.decode target/riscv/insn_trans/trans_privileged.inc.c+54 -90 replies
20/27target/riscv: Disable guest FP support based on backgrond statustarget/riscv/cpu_helper.c+3 -00 replies
21/27target/riscv: Mark both sstatus and bsstatus as dirtytarget/riscv/translate.c+12 -00 replies
22/27target/riscv: Respect MPRV and SPRV for floating point opstarget/riscv/translate.c+15 -10 replies
23/27target/riscv: Allow specifying MMU stagetarget/riscv/cpu_helper.c+27 -80 replies
24/27target/riscv: Allow specifying number of MMU stagestarget/riscv/cpu_helper.c+4 -30 replies
25/27target/riscv: Implement second stage MMUtarget/riscv/cpu_helper.c+78 -90 replies
26/27target/riscv: Call the second stage MMU in virtualisation modetarget/riscv/cpu_helper.c+99 -190 replies
27/27target/riscv: Allow enabling the Hypervisor extensiontarget/riscv/cpu.c target/riscv/cpu.h+5 -00 replies

Reviews trailers per patch

None of the 27 patches has a Reviewed-by or Acked-by.

PatchChih-Min Chao
SeriesT-b
1/27
2/27
3/27
4/27
5/27
6/27
7/27
8/27
9/27
10/27
11/27
12/27
13/27
14/27
15/27
16/27
17/27
18/27
19/27
20/27
21/27
22/27
23/27
24/27
25/27
26/27
27/27

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [Qemu-devel] [PATCH v1 00/27] Add RISC-V Hypervisor Extension

This patch series adds the RISC-V Hypervisor extension 0.3. This is the latest draft spec of the Hypervisor extension.

The Hypervisor extension is disabled by default, so this series should result in no changes to anyone using QEMU unless they enable the extension. The extention can be enabled with the -cpu property (see below).

At the moment the spec does not include information about the mstatush register. As it is not in the spec I haven't added it to QEMU. This means the extension won't work correctly for 32-bit guests. This should be a small fix to add the CSR once the spec is updated.

All testing of this implementation has been done by using the baremetal Xvisor Hypervisor. We are able to run two Linux guests (that's all I have tried) as guests.

At the moment this spec is in a draft state and is subject to change. As QEMU is extreamly useful in early bring up I think it makes sense for QEMU to support non-frozen extensions. I would like to decide with this series how QEMU will handle all future non-frozen extensions. That is a standard way that QEMU users can test future RISC-V extensions while still understanding things will change. One idea is just to disable it by default, another option is to maybe use the Kconfig to make it a compile time option which developers can use. Should we also display a warning when running non-frozen extensions?

Thanks to Anup for doing the initial port of Xvisor. The port is avaliable here:
https://github.com/avpatel/xvisor-next and will run on QEMU.

Also thanks to Atish for implementing the SBI call support in Xvisor and for lots of help debugging.

To run this yourself:
1. Apply this patch series to QEMU. The latest branch can be found here:
https://github.com/alistair23/qemu/tree/mainline/alistair/riscv-hyp-work.next
2. Get the version of OpenSBI that supports the H extension. This can be found here:
https://github.com/riscv/opensbi/tree/hyp_ext_changes_v1
3. Build the next release of Xvisor. It is available here:
https://github.com/avpatel/xvisor-next
4. Make sure you build the Xvisor tests, see here for details:
https://github.com/avpatel/xvisor-next/tree/master/tests/riscv/virt64/linux
5. Run QEMU:

     ./riscv64-softmmu/qemu-system-riscv64 -nographic \
       -machine virt -cpu rv64,h=true\
       -serial mon:stdio -serial null -m 4G \
       -device loader,file=vmm.bin,addr=0x80200000 \
       -kernel fw_jump.elf \
       -initrd vmm-disk-linux.img \
       -append "vmm.console=uart@10000000 vmm.bootcmd=\"vfs mount initrd /;vfs run /boot.xscript;vfs cat /system/banner.txt\""

Once you get to the prompt you can start the geust by running:
guest kick guest0
You can then bind to the serial port using:
vserial bind guest0/uart0
Then you can start Linux using:
autoexec

This was all tested with the mainline 5.1 kernel. I don't know if it will work on older kernels.

So far all of the QEMU work has been tested on Xvisor.

Known Issues/TODO:
- Add mstatush to support 32-bit Hypervisors
- Fix the random hang that sometimes appears when running a Hypervisor guest (~5%)

There is also on going work from Anup to port KVM.
We have code complete implementation of RISC-V KVM kernel module and RISC-V KVMTOOL. Currently, we are debugging KVM on QEMU and we will send-out RFC PATCHES for KVM in June/July 2019.
The KVM RISC-V kernel module is available in riscv_kvm_v1 branch at: https://github.com/avpatel/linux.git
The KVMTOOL RISC-V port is available in riscv_v1 branch of
https://github.com/avpatel/kvmtool.git

There is very early work on a Xen port as well which is avaliable here:
https://github.com/alistair23/xen/tree/alistair/riscv-port

Changes from RFC:
- Rebase on latest master
- Add floating point changes from Hypervisor extension

Alistair Francis (27):
target/riscv: Don't set write permissions on dirty PTEs target/riscv: Add the Hypervisor extension target/riscv: Add the virtulisation mode target/riscv: Add the force HS exception mode target/riscv: Add the Hypervisor CSRs to CPUState target/riscv: Dump Hypervisor registers if enabled target/riscv: Remove strict perm checking for CSR R/W target/riscv: Create function to test if FP is enabled target/riscv: Add support for background interrupt setting target/riscv: Add Hypervisor CSR access functions target/riscv: Add background CSRs accesses target/riscv: Add background register swapping function target/ricsv: Flush the TLB on virtulisation mode changes target/riscv: Generate illegal instruction on WFI when V=1
riscv: plic: Remove unused interrupt functions
riscv: plic: Always set sip.SEIP bit for HS target/riscv: Add hypvervisor trap support target/riscv: Add Hypervisor trap return support target/riscv: Add hfence instructions
target/riscv: Disable guest FP support based on backgrond status target/riscv: Mark both sstatus and bsstatus as dirty target/riscv: Respect MPRV and SPRV for floating point ops target/riscv: Allow specifying MMU stage target/riscv: Allow specifying number of MMU stages target/riscv: Implement second stage MMU target/riscv: Call the second stage MMU in virtualisation mode target/riscv: Allow enabling the Hypervisor extension

 hw/riscv/sifive_plic.c                        |  24 +-
 include/hw/riscv/sifive_plic.h                |   3 -
 target/riscv/cpu.c                            |  31 ++
 target/riscv/cpu.h                            |  32 +-
 target/riscv/cpu_bits.h                       |  32 +-
 target/riscv/cpu_helper.c                     | 443 ++++++++++++++++--
 target/riscv/csr.c                            | 216 ++++++++-
 target/riscv/insn32.decode                    |  23 +-
 .../riscv/insn_trans/trans_privileged.inc.c   |  40 ++
 target/riscv/op_helper.c                      |  71 ++-
 target/riscv/translate.c                      |  26 +
 11 files changed, 839 insertions(+), 102 deletions(-)
Signature · 1 line
2.21.0