Skip to contentSkip to conversation

RISC-V: Hypervisor prep work part 2

PATCH In review Alistair Francis 5 participants 17 messages Aug 23, 2019 – Jan 21, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v4 cover RISC-V: Hypervisor prep work part 2

The first three patches are ones that I have pulled out of my original Hypervisor series at an attempt to reduce the number of patches in the series.

These three patches all make sense without the Hypervisor series so can be merged seperatley and will reduce the review burden of the next version of the patches.

The fource patch is a prep patch for the new v0.4 Hypervisor spec.

The fifth patch is unreleated to Hypervisor that I'm just slipping in here because it seems easier then sending it by itself.

The final two patches are issues I discovered while adding the v0.4 Hypervisor extension.

v4:
- Drop MIP change patch
- Add a Floating Point fixup patch
v3:
- Change names of all GP registers
- Add two more patches
v2:
- Small corrections based on feedback
- Remove the CSR permission check patch

Alistair Francis (6):
target/riscv: Don't set write permissions on dirty PTEs
riscv: plic: Remove unused interrupt functions target/riscv: Create function to test if FP is enabled target/riscv: Update the Hypervisor CSRs to v0.4 target/riscv: Fix mstatus dirty mask target/riscv: Use TB_FLAGS_MSTATUS_FS for floating point

Atish Patra (1):
target/riscv: Use both register name and ABI name

 hw/riscv/sifive_plic.c         | 12 ------------
 include/hw/riscv/sifive_plic.h |  3 ---
 target/riscv/cpu.c             | 19 ++++++++++--------
 target/riscv/cpu.h             |  6 +++++-
 target/riscv/cpu_bits.h        | 35 +++++++++++++++++-----------------
 target/riscv/cpu_helper.c      | 16 ++++++++++++----
 target/riscv/csr.c             | 22 +++++++++++----------
 7 files changed, 58 insertions(+), 55 deletions(-)
Signature · 1 line
2.22.0
#1 lore raw

PATCH v4 1/7 target/riscv: Don't set write permissions on dirty PTEs

Setting write permission on dirty PTEs results in userspace inside a Hypervisor guest (VU) becoming corrupted. This appears to be because it ends up with write permission in the second stage translation in cases where we aren't doing a store.

---
 target/riscv/cpu_helper.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
target/riscv/cpu_helper.c +2 -4
index e32b6126af..f027be7f16 100644
@@ -340,10 +340,8 @@ restart:
             if ((pte & PTE_X)) {
                 *prot |= PAGE_EXEC;
             }
-            /* add write permission on stores or if the page is already dirty,
-               so that we TLB miss on later writes to update the dirty bit */
-            if ((pte & PTE_W) &&
-                    (access_type == MMU_DATA_STORE || (pte & PTE_D))) {
+            /* add write permission on stores */
+            if ((pte & PTE_W) && (access_type == MMU_DATA_STORE)) {
                 *prot |= PAGE_WRITE;
             }
             return TRANSLATE_SUCCESS;
Signature · 1 line
2.22.0
#2 lore raw

PATCH v4 2/7 riscv: plic: Remove unused interrupt functions

In reply to Alistair Francis
---
 hw/riscv/sifive_plic.c         | 12 ------------
 include/hw/riscv/sifive_plic.h |  3 ---
 2 files changed, 15 deletions(-)
hw/riscv/sifive_plic.c +0 -12
index 64a1a10380..98e4304b66 100644
@@ -162,18 +162,6 @@ static void sifive_plic_update(SiFivePLICState *plic)
     }
 }
 
-void sifive_plic_raise_irq(SiFivePLICState *plic, uint32_t irq)
-{
-    sifive_plic_set_pending(plic, irq, true);
-    sifive_plic_update(plic);
-}
-
-void sifive_plic_lower_irq(SiFivePLICState *plic, uint32_t irq)
-{
-    sifive_plic_set_pending(plic, irq, false);
-    sifive_plic_update(plic);
-}
-
 static uint32_t sifive_plic_claim(SiFivePLICState *plic, uint32_t addrid)
 {
     int i, j;
include/hw/riscv/sifive_plic.h +0 -3
index b0edba2884..4421e81249 100644
@@ -69,9 +69,6 @@ typedef struct SiFivePLICState {
     uint32_t aperture_size;
 } SiFivePLICState;
 
-void sifive_plic_raise_irq(SiFivePLICState *plic, uint32_t irq);
-void sifive_plic_lower_irq(SiFivePLICState *plic, uint32_t irq);
-
 DeviceState *sifive_plic_create(hwaddr addr, char *hart_config,
     uint32_t num_sources, uint32_t num_priorities,
     uint32_t priority_base, uint32_t pending_base,
Signature · 1 line
2.22.0
#3 lore raw

PATCH v4 3/7 target/riscv: Create function to test if FP is enabled

In reply to Alistair Francis

Let's create a function that tests if floating point support is enabled. We can then protect all floating point operations based on if they are enabled.

This patch so far doesn't change anything, it's just preparing for the Hypervisor support for floating point operations.

---
 target/riscv/cpu.h        |  6 +++++-
 target/riscv/cpu_helper.c | 10 ++++++++++
 target/riscv/csr.c        | 20 +++++++++++---------
 3 files changed, 26 insertions(+), 10 deletions(-)
target/riscv/cpu.h +5 -1
index 240b31e2eb..eb7b5b0af3 100644
@@ -255,6 +255,7 @@ void riscv_cpu_do_interrupt(CPUState *cpu);
 int riscv_cpu_gdb_read_register(CPUState *cpu, uint8_t *buf, int reg);
 int riscv_cpu_gdb_write_register(CPUState *cpu, uint8_t *buf, int reg);
 bool riscv_cpu_exec_interrupt(CPUState *cs, int interrupt_request);
+bool riscv_cpu_fp_enabled(CPURISCVState *env);
 int riscv_cpu_mmu_index(CPURISCVState *env, bool ifetch);
 hwaddr riscv_cpu_get_phys_page_debug(CPUState *cpu, vaddr addr);
 void  riscv_cpu_do_unaligned_access(CPUState *cs, vaddr addr,
@@ -298,7 +299,10 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
 #ifdef CONFIG_USER_ONLY
     *flags = TB_FLAGS_MSTATUS_FS;
 #else
-    *flags = cpu_mmu_index(env, 0) | (env->mstatus & MSTATUS_FS);
+    *flags = cpu_mmu_index(env, 0);
+    if (riscv_cpu_fp_enabled(env)) {
+        *flags |= env->mstatus & MSTATUS_FS;
+    }
 #endif
 }
 
target/riscv/cpu_helper.c +10 -0
index f027be7f16..225e407cff 100644
@@ -71,6 +71,16 @@ bool riscv_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
 
 #if !defined(CONFIG_USER_ONLY)
 
+/* Return true is floating point support is currently enabled */
+bool riscv_cpu_fp_enabled(CPURISCVState *env)
+{
+    if (env->mstatus & MSTATUS_FS) {
+        return true;
+    }
+
+    return false;
+}
+
 int riscv_cpu_claim_interrupts(RISCVCPU *cpu, uint32_t interrupts)
 {
     CPURISCVState *env = &cpu->env;
target/riscv/csr.c +11 -9
index e0d4586760..2789215b5e 100644
@@ -46,7 +46,7 @@ void riscv_set_csr_ops(int csrno, riscv_csr_operations *ops)
 static int fs(CPURISCVState *env, int csrno)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
 #endif
@@ -108,7 +108,7 @@ static int pmp(CPURISCVState *env, int csrno)
 static int read_fflags(CPURISCVState *env, int csrno, target_ulong *val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
 #endif
@@ -119,7 +119,7 @@ static int read_fflags(CPURISCVState *env, int csrno, target_ulong *val)
 static int write_fflags(CPURISCVState *env, int csrno, target_ulong val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
     env->mstatus |= MSTATUS_FS;
@@ -131,7 +131,7 @@ static int write_fflags(CPURISCVState *env, int csrno, target_ulong val)
 static int read_frm(CPURISCVState *env, int csrno, target_ulong *val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
 #endif
@@ -142,7 +142,7 @@ static int read_frm(CPURISCVState *env, int csrno, target_ulong *val)
 static int write_frm(CPURISCVState *env, int csrno, target_ulong val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
     env->mstatus |= MSTATUS_FS;
@@ -154,7 +154,7 @@ static int write_frm(CPURISCVState *env, int csrno, target_ulong val)
 static int read_fcsr(CPURISCVState *env, int csrno, target_ulong *val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
 #endif
@@ -166,7 +166,7 @@ static int read_fcsr(CPURISCVState *env, int csrno, target_ulong *val)
 static int write_fcsr(CPURISCVState *env, int csrno, target_ulong val)
 {
 #if !defined(CONFIG_USER_ONLY)
-    if (!env->debugger && !(env->mstatus & MSTATUS_FS)) {
+    if (!env->debugger && !riscv_cpu_fp_enabled(env)) {
         return -1;
     }
     env->mstatus |= MSTATUS_FS;
@@ -307,6 +307,7 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 {
     target_ulong mstatus = env->mstatus;
     target_ulong mask = 0;
+    int dirty;
 
     /* flush tlb on mstatus fields that affect VM */
     if (env->priv_ver <= PRIV_VERSION_1_09_1) {
@@ -340,8 +341,9 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 
     mstatus = (mstatus & ~mask) | (val & mask);
 
-    int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
-                ((mstatus & MSTATUS_XS) == MSTATUS_XS);
+    dirty = (riscv_cpu_fp_enabled(env) &&
+             ((mstatus & MSTATUS_FS) == MSTATUS_FS)) |
+            ((mstatus & MSTATUS_XS) == MSTATUS_XS);
     mstatus = set_field(mstatus, MSTATUS_SD, dirty);
     env->mstatus = mstatus;
 
Signature · 1 line
2.22.0
#4 lore raw

Re: [Qemu-devel] [PATCH v4 3/7] target/riscv: Create function to test if FP is enabled

Hi,

Quoted 18 lines · Alistair Francis

Let's create a function that tests if floating point support is enabled. We can then protect all floating point operations based on if they are enabled.

This patch so far doesn't change anything, it's just preparing for the Hypervisor support for floating point operations.

---
 target/riscv/cpu.h        |  6 +++++-
 target/riscv/cpu_helper.c | 10 ++++++++++
 target/riscv/csr.c        | 20 +++++++++++---------
 3 files changed, 26 insertions(+), 10 deletions(-)

[ snip ]

Quoted 4 lines
target/riscv/csr.c +0 -0
index e0d4586760..2789215b5e 100644

[ snip ]

Quoted 9 lines
target/riscv/csr.c +1 -0
@@ -307,6 +307,7 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 {
     target_ulong mstatus = env->mstatus;
     target_ulong mask = 0;
+    int dirty;
 
     /* flush tlb on mstatus fields that affect VM */
     if (env->priv_ver <= PRIV_VERSION_1_09_1) {

target/riscv/csr.c static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val) · line 348

Quoted 10 lines
target/riscv/csr.c +3 -2
@@ -340,8 +341,9 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 
     mstatus = (mstatus & ~mask) | (val & mask);
 
-    int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
-                ((mstatus & MSTATUS_XS) == MSTATUS_XS);
+    dirty = (riscv_cpu_fp_enabled(env) &&
+             ((mstatus & MSTATUS_FS) == MSTATUS_FS)) |
+            ((mstatus & MSTATUS_XS) == MSTATUS_XS);
     mstatus = set_field(mstatus, MSTATUS_SD, dirty);
     env->mstatus = mstatus;

This patch, and more precisely the above two hunks broke qemu-system-riscv64. More precisely, when running a Debian sid system inside QEMU, sshd hangs during key exchange.

Reverting this commit "fixes" the issue up to the following commit which breaks things again:

| commit bdce1a5c6d512257f83b6b6831bee2c975643bbd
| Author: Alistair Francis <[email protected]>
| Date:   Fri Aug 23 08:21:25 2019 -0700
| 
|     target/riscv: Use TB_FLAGS_MSTATUS_FS for floating point
| 
|     Use the TB_FLAGS_MSTATUS_FS macro when enabling floating point in the tb
|     flags.
| 
|     Signed-off-by: Alistair Francis <[email protected]>
|     Reviewed-by: Palmer Dabbelt <[email protected]>
|     Signed-off-by: Palmer Dabbelt <[email protected]>

I wonder if the issue is related to the fact that MSTATUS_FS and thus TB_FLAGS_MSTATUS_FS actually consist in 2 bits and are not a simple flag.

Overall I am able to get QEMU v4.2 working again by applying the following patch:

target/riscv/cpu.h +1 -1
index e59343e13c..f0ff57e27a 100644
@@ -295,7 +295,7 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
 #else
     *flags = cpu_mmu_index(env, 0);
     if (riscv_cpu_fp_enabled(env)) {
-        *flags |= TB_FLAGS_MSTATUS_FS;
+        *flags |= env->mstatus & MSTATUS_FS;
     }
 #endif
 }
target/riscv/csr.c +2 -4
index da02f9f0b1..1754c6c575 100644
@@ -307,7 +307,6 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 {
     target_ulong mstatus = env->mstatus;
     target_ulong mask = 0;
-    int dirty;
 
     /* flush tlb on mstatus fields that affect VM */
     if (env->priv_ver <= PRIV_VERSION_1_09_1) {
@@ -341,9 +340,8 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 
     mstatus = (mstatus & ~mask) | (val & mask);
 
-    dirty = (riscv_cpu_fp_enabled(env) &&
-             ((mstatus & MSTATUS_FS) == MSTATUS_FS)) |
-            ((mstatus & MSTATUS_XS) == MSTATUS_XS);
+    int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
+                ((mstatus & MSTATUS_XS) == MSTATUS_XS);
     mstatus = set_field(mstatus, MSTATUS_SD, dirty);
     env->mstatus = mstatus;
 
Signature · 2 lines
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
[email protected]                 http://www.aurel32.net
#5 lore raw

Re: [Qemu-devel] [PATCH v4 3/7] target/riscv: Create function to test if FP is enabled

Quoted 19 lines · Aurelien Jarno
Quoted 4 lines
target/riscv/csr.c +0 -0
index e0d4586760..2789215b5e 100644

[ snip ]

Quoted 12 lines
target/riscv/csr.c +1 -0
@@ -307,6 +307,7 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 {
     target_ulong mstatus = env->mstatus;
     target_ulong mask = 0;
+    int dirty;
 
     /* flush tlb on mstatus fields that affect VM */
     if (env->priv_ver <= PRIV_VERSION_1_09_1) {
@@ -340,8 +341,9 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 
     mstatus = (mstatus & ~mask) | (val & mask);
 
Quoted 11 lines

target/riscv/csr.c static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val) · line 346

Quoted 7 lines
target/riscv/csr.c +3 -2
-    int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
-                ((mstatus & MSTATUS_XS) == MSTATUS_XS);
+    dirty = (riscv_cpu_fp_enabled(env) &&
+             ((mstatus & MSTATUS_FS) == MSTATUS_FS)) |
+            ((mstatus & MSTATUS_XS) == MSTATUS_XS);
     mstatus = set_field(mstatus, MSTATUS_SD, dirty);
     env->mstatus = mstatus;

This patch, and more precisely the above two hunks broke qemu-system-riscv64. More precisely, when running a Debian sid system inside QEMU, sshd hangs during key exchange.

The problem is that at this stage, mstatus != env->status. Prior to that patch, dirty was computed exclusively on the new mstatus status, after the update by val. With this patch, riscv_cpu_fp_enabled() refers to the old value of mstatus. Therefore when FS is changed from "Off" (FS = 00) to "Dirty" (FS == 11), the SD bit is not set.

Signature · 2 lines
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
[email protected]                 http://www.aurel32.net
#6 lore raw

Re: [Qemu-devel] [PATCH v4 3/7] target/riscv: Create function to test if FP is enabled

Quoted 33 lines · Aurelien Jarno
Quoted 19 lines · Aurelien Jarno
Quoted 4 lines
target/riscv/csr.c +0 -0
index e0d4586760..2789215b5e 100644

[ snip ]

Quoted 12 lines
target/riscv/csr.c +1 -0
@@ -307,6 +307,7 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 {
     target_ulong mstatus = env->mstatus;
     target_ulong mask = 0;
+    int dirty;
 
     /* flush tlb on mstatus fields that affect VM */
     if (env->priv_ver <= PRIV_VERSION_1_09_1) {
@@ -340,8 +341,9 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 
     mstatus = (mstatus & ~mask) | (val & mask);
 
Quoted 11 lines

target/riscv/csr.c static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val) · line 346

Quoted 7 lines
target/riscv/csr.c +3 -2
-    int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
-                ((mstatus & MSTATUS_XS) == MSTATUS_XS);
+    dirty = (riscv_cpu_fp_enabled(env) &&
+             ((mstatus & MSTATUS_FS) == MSTATUS_FS)) |
+            ((mstatus & MSTATUS_XS) == MSTATUS_XS);
     mstatus = set_field(mstatus, MSTATUS_SD, dirty);
     env->mstatus = mstatus;

This patch, and more precisely the above two hunks broke qemu-system-riscv64. More precisely, when running a Debian sid system inside QEMU, sshd hangs during key exchange.

Quoted 5 lines

The problem is that at this stage, mstatus != env->status. Prior to that patch, dirty was computed exclusively on the new mstatus status, after the update by val. With this patch, riscv_cpu_fp_enabled() refers to the old value of mstatus. Therefore when FS is changed from "Off" (FS = 00) to "Dirty" (FS == 11), the SD bit is not set.

Thanks for reporting this!

Can you try this branch (it should be a PR to mainline QEMU soon) and let me know if that fixes the issue?

https://github.com/palmer-dabbelt/qemu/commits/for-master

Alistair

Quoted 4 lines

--
Aurelien Jarno GPG: 4096R/1DDD8C9B
[email protected] http://www.aurel32.net

#7 lore raw

Re: [Qemu-devel] [PATCH v4 3/7] target/riscv: Create function to test if FP is enabled

Hi,

Quoted 40 lines · Alistair Francis
Quoted 33 lines · Aurelien Jarno
Quoted 19 lines · Aurelien Jarno
Quoted 4 lines
target/riscv/csr.c +0 -0
index e0d4586760..2789215b5e 100644

[ snip ]

Quoted 12 lines
target/riscv/csr.c +1 -0
@@ -307,6 +307,7 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 {
     target_ulong mstatus = env->mstatus;
     target_ulong mask = 0;
+    int dirty;
 
     /* flush tlb on mstatus fields that affect VM */
     if (env->priv_ver <= PRIV_VERSION_1_09_1) {
@@ -340,8 +341,9 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 
     mstatus = (mstatus & ~mask) | (val & mask);
 
Quoted 11 lines

target/riscv/csr.c static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val) · line 346

Quoted 7 lines
target/riscv/csr.c +3 -2
-    int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
-                ((mstatus & MSTATUS_XS) == MSTATUS_XS);
+    dirty = (riscv_cpu_fp_enabled(env) &&
+             ((mstatus & MSTATUS_FS) == MSTATUS_FS)) |
+            ((mstatus & MSTATUS_XS) == MSTATUS_XS);
     mstatus = set_field(mstatus, MSTATUS_SD, dirty);
     env->mstatus = mstatus;

This patch, and more precisely the above two hunks broke qemu-system-riscv64. More precisely, when running a Debian sid system inside QEMU, sshd hangs during key exchange.

Quoted 5 lines

The problem is that at this stage, mstatus != env->status. Prior to that patch, dirty was computed exclusively on the new mstatus status, after the update by val. With this patch, riscv_cpu_fp_enabled() refers to the old value of mstatus. Therefore when FS is changed from "Off" (FS = 00) to "Dirty" (FS == 11), the SD bit is not set.

Quoted 6 lines

Thanks for reporting this!

Can you try this branch (it should be a PR to mainline QEMU soon) and let me know if that fixes the issue?

https://github.com/palmer-dabbelt/qemu/commits/for-master

Thanks for the patchset. I confirm this fixes the issue.

Aurelien

Signature · 2 lines
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
[email protected]                 http://www.aurel32.net
#8 lore raw

Re: [Qemu-devel] [PATCH v4 3/7] target/riscv: Create function to test if FP is enabled

Quoted 44 lines · Aurelien Jarno

Hi,

Quoted 40 lines · Alistair Francis
Quoted 38 lines · Aurelien Jarno
Quoted 19 lines · Aurelien Jarno
Quoted 4 lines
target/riscv/csr.c +0 -0
index e0d4586760..2789215b5e 100644

[ snip ]

Quoted 12 lines
target/riscv/csr.c +1 -0
@@ -307,6 +307,7 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 {
     target_ulong mstatus = env->mstatus;
     target_ulong mask = 0;
+    int dirty;
 
     /* flush tlb on mstatus fields that affect VM */
     if (env->priv_ver <= PRIV_VERSION_1_09_1) {
@@ -340,8 +341,9 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
 
     mstatus = (mstatus & ~mask) | (val & mask);
 
Quoted 11 lines

target/riscv/csr.c static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val) · line 346

Quoted 7 lines
target/riscv/csr.c +3 -2
-    int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
-                ((mstatus & MSTATUS_XS) == MSTATUS_XS);
+    dirty = (riscv_cpu_fp_enabled(env) &&
+             ((mstatus & MSTATUS_FS) == MSTATUS_FS)) |
+            ((mstatus & MSTATUS_XS) == MSTATUS_XS);
     mstatus = set_field(mstatus, MSTATUS_SD, dirty);
     env->mstatus = mstatus;

This patch, and more precisely the above two hunks broke qemu-system-riscv64. More precisely, when running a Debian sid system inside QEMU, sshd hangs during key exchange.

The problem is that at this stage, mstatus != env->status. Prior to that patch, dirty was computed exclusively on the new mstatus status, after the update by val. With this patch, riscv_cpu_fp_enabled() refers to the old value of mstatus. Therefore when FS is changed from "Off" (FS = 00) to "Dirty" (FS == 11), the SD bit is not set.

Quoted 8 lines
Quoted 6 lines

Thanks for reporting this!

Can you try this branch (it should be a PR to mainline QEMU soon) and let me know if that fixes the issue?

https://github.com/palmer-dabbelt/qemu/commits/for-master

Thanks for the patchset. I confirm this fixes the issue.

Great! Sorry we were so slow in replying to you, I was traveling. Hopefully this is pushed to master soon.

Alistair

Quoted 6 lines

Aurelien

--
Aurelien Jarno GPG: 4096R/1DDD8C9B
[email protected] http://www.aurel32.net

#9 lore raw

PATCH v4 4/7 target/riscv: Update the Hypervisor CSRs to v0.4

In reply to Alistair Francis

Update the Hypervisor CSR addresses to match the v0.4 spec.

---
 target/riscv/cpu_bits.h | 35 ++++++++++++++++++-----------------
 1 file changed, 18 insertions(+), 17 deletions(-)
target/riscv/cpu_bits.h +18 -17
index 11f971ad5d..e99834856c 100644
@@ -173,6 +173,24 @@
 #define CSR_SPTBR           0x180
 #define CSR_SATP            0x180
 
+/* Hpervisor CSRs */
+#define CSR_HSTATUS         0x600
+#define CSR_HEDELEG         0x602
+#define CSR_HIDELEG         0x603
+#define CSR_HCOUNTERNEN     0x606
+#define CSR_HGATP           0x680
+
+#if defined(TARGET_RISCV32)
+#define HGATP_MODE           SATP32_MODE
+#define HGATP_VMID           SATP32_ASID
+#define HGATP_PPN            SATP32_PPN
+#endif
+#if defined(TARGET_RISCV64)
+#define HGATP_MODE           SATP64_MODE
+#define HGATP_VMID           SATP64_ASID
+#define HGATP_PPN            SATP64_PPN
+#endif
+
 /* Physical Memory Protection */
 #define CSR_PMPCFG0         0x3a0
 #define CSR_PMPCFG1         0x3a1
@@ -206,23 +224,6 @@
 #define CSR_DPC             0x7b1
 #define CSR_DSCRATCH        0x7b2
 
-/* Hpervisor CSRs */
-#define CSR_HSTATUS         0xa00
-#define CSR_HEDELEG         0xa02
-#define CSR_HIDELEG         0xa03
-#define CSR_HGATP           0xa80
-
-#if defined(TARGET_RISCV32)
-#define HGATP_MODE           SATP32_MODE
-#define HGATP_ASID           SATP32_ASID
-#define HGATP_PPN            SATP32_PPN
-#endif
-#if defined(TARGET_RISCV64)
-#define HGATP_MODE           SATP64_MODE
-#define HGATP_ASID           SATP64_ASID
-#define HGATP_PPN            SATP64_PPN
-#endif
-
 /* Performance Counters */
 #define CSR_MHPMCOUNTER3    0xb03
 #define CSR_MHPMCOUNTER4    0xb04
Signature · 1 line
2.22.0
#10 lore raw

PATCH v4 5/7 target/riscv: Use both register name and ABI name

In reply to Alistair Francis

From: Atish Patra <[email protected]>

Use both the generic register name and ABI name for the general purpose registers and floating point registers.

---
 target/riscv/cpu.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)
target/riscv/cpu.c +11 -8
index 6d52f97d7c..f13e298a36 100644
@@ -34,17 +34,20 @@
 static const char riscv_exts[26] = "IEMAFDQCLBJTPVNSUHKORWXYZG";
 
 const char * const riscv_int_regnames[] = {
-  "zero", "ra", "sp",  "gp",  "tp", "t0", "t1", "t2",
-  "s0",   "s1", "a0",  "a1",  "a2", "a3", "a4", "a5",
-  "a6",   "a7", "s2",  "s3",  "s4", "s5", "s6", "s7",
-  "s8",   "s9", "s10", "s11", "t3", "t4", "t5", "t6"
+  "x0/zero", "x1/ra",  "x2/sp",  "x3/gp",  "x4/tp",  "x5/t0",   "x6/t1",
+  "x7/t2",   "x8/s0",  "x9/s1",  "x10/a0", "x11/a1", "x12/a2",  "x13/a3",
+  "x14/a4",  "x15/a5", "x16/a6", "x17/a7", "x18/s2", "x19/s3",  "x20/s4",
+  "x21/s5",  "x22/s6", "x23/s7", "x24/s8", "x25/s9", "x26/s10", "x27/s11",
+  "x28/t3",  "x29/t4", "x30/t5", "x31/t6"
 };
 
 const char * const riscv_fpr_regnames[] = {
-  "ft0", "ft1", "ft2",  "ft3",  "ft4", "ft5", "ft6",  "ft7",
-  "fs0", "fs1", "fa0",  "fa1",  "fa2", "fa3", "fa4",  "fa5",
-  "fa6", "fa7", "fs2",  "fs3",  "fs4", "fs5", "fs6",  "fs7",
-  "fs8", "fs9", "fs10", "fs11", "ft8", "ft9", "ft10", "ft11"
+  "f0/ft0",   "f1/ft1",  "f2/ft2",   "f3/ft3",   "f4/ft4",  "f5/ft5",
+  "f6/ft6",   "f7/ft7",  "f8/fs0",   "f9/fs1",   "f10/fa0", "f11/fa1",
+  "f12/fa2",  "f13/fa3", "f14/fa4",  "f15/fa5",  "f16/fa6", "f17/fa7",
+  "f18/fs2",  "f19/fs3", "f20/fs4",  "f21/fs5",  "f22/fs6", "f23/fs7",
+  "f24/fs8",  "f25/fs9", "f26/fs10", "f27/fs11", "f28/ft8", "f29/ft9",
+  "f30/ft10", "f31/ft11"
 };
 
 const char * const riscv_excp_names[] = {
Signature · 1 line
2.22.0
#11 lore raw

PATCH v4 6/7 target/riscv: Fix mstatus dirty mask

In reply to Alistair Francis
---
 target/riscv/csr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/riscv/csr.c +1 -1
index 2789215b5e..f767ad24be 100644
@@ -335,7 +335,7 @@ static int write_mstatus(CPURISCVState *env, int csrno, target_ulong val)
              * RV32: MPV and MTL are not in mstatus. The current plan is to
              * add them to mstatush. For now, we just don't support it.
              */
-            mask |= MSTATUS_MPP | MSTATUS_MPV;
+            mask |= MSTATUS_MTL | MSTATUS_MPV;
 #endif
     }
 
Signature · 1 line
2.22.0
#12 lore raw

PATCH v4 7/7 target/riscv: Use TB_FLAGS_MSTATUS_FS for floating point

In reply to Alistair Francis

Use the TB_FLAGS_MSTATUS_FS macro when enabling floating point in the tb flags.

---
 target/riscv/cpu.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/riscv/cpu.h +1 -1
index eb7b5b0af3..0347be453b 100644
@@ -301,7 +301,7 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
 #else
     *flags = cpu_mmu_index(env, 0);
     if (riscv_cpu_fp_enabled(env)) {
-        *flags |= env->mstatus & MSTATUS_FS;
+        *flags |= TB_FLAGS_MSTATUS_FS;
     }
 #endif
 }
Signature · 1 line
2.22.0
#13 lore raw

Re: [Qemu-devel] [PATCH v4 7/7] target/riscv: Use TB_FLAGS_MSTATUS_FS for floating point

Quoted 14 lines · Alistair Francis

Use the TB_FLAGS_MSTATUS_FS macro when enabling floating point in the tb flags.

---
 target/riscv/cpu.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/riscv/cpu.h +0 -0
index eb7b5b0af3..0347be453b 100644

target/riscv/cpu.h static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc, · line 306

Quoted 7 lines
target/riscv/cpu.h +1 -1
@@ -301,7 +301,7 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
 #else
     *flags = cpu_mmu_index(env, 0);
     if (riscv_cpu_fp_enabled(env)) {
-        *flags |= env->mstatus & MSTATUS_FS;
+        *flags |= TB_FLAGS_MSTATUS_FS;
     }
 #endif

The old code was setting the bit in flags only if it was also set in env->mstatus; the new code sets the bit unconditionally -- deliberate change ?

thanks
-- PMM

#14 lore raw

Re: [Qemu-devel] [PATCH v4 7/7] target/riscv: Use TB_FLAGS_MSTATUS_FS for floating point

Quoted 11 lines · Peter Maydell
Quoted 9 lines · Alistair Francis

Use the TB_FLAGS_MSTATUS_FS macro when enabling floating point in the tb flags.

---
 target/riscv/cpu.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines

target/riscv/cpu.h static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc, · line 306

Quoted 12 lines
target/riscv/cpu.h +1 -1
index eb7b5b0af3..0347be453b 100644
@@ -301,7 +301,7 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
 #else
     *flags = cpu_mmu_index(env, 0);
     if (riscv_cpu_fp_enabled(env)) {
-        *flags |= env->mstatus & MSTATUS_FS;
+        *flags |= TB_FLAGS_MSTATUS_FS;
     }
 #endif

The old code was setting the bit in flags only if it was also set in env->mstatus; the new code sets the bit unconditionally -- deliberate change ?

Yes it is deliberate as the riscv_cpu_fp_enabled() function already does the check. The function contains the exact same & operation inside of it.

Alistair

Quoted 3 lines

thanks
-- PMM

Reviewed-by on v4 7/7
#15 lore raw

Re: [Qemu-devel] [PATCH v4 7/7] target/riscv: Use TB_FLAGS_MSTATUS_FS for floating point

In reply to Alistair Francis
Quoted 13 lines · Alistair Francis

Use the TB_FLAGS_MSTATUS_FS macro when enabling floating point in the tb flags.

---
 target/riscv/cpu.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/riscv/cpu.h +0 -0
index eb7b5b0af3..0347be453b 100644

target/riscv/cpu.h static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc, · line 304

Quoted 5 lines
target/riscv/cpu.h +1 -1
@@ -301,7 +301,7 @@ static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc,
 #else
     *flags = cpu_mmu_index(env, 0);
     if (riscv_cpu_fp_enabled(env)) {
-        *flags |= env->mstatus & MSTATUS_FS;
+        *flags |= TB_FLAGS_MSTATUS_FS;

I thought this was a functional change, but it's not: fp_enabled() checks mstatus already.

target/riscv/cpu.h static inline void cpu_get_tb_cpu_state(CPURISCVState *env, target_ulong *pc, · line 307

Quoted 3 lines
target/riscv/cpu.h +0 -0
     }
 #endif
 }
#16 lore raw
In reply to Alistair Francis
Quoted 40 lines · Alistair Francis

The first three patches are ones that I have pulled out of my original Hypervisor series at an attempt to reduce the number of patches in the series.

These three patches all make sense without the Hypervisor series so can be merged seperatley and will reduce the review burden of the next version of the patches.

The fource patch is a prep patch for the new v0.4 Hypervisor spec.

The fifth patch is unreleated to Hypervisor that I'm just slipping in here because it seems easier then sending it by itself.

The final two patches are issues I discovered while adding the v0.4 Hypervisor extension.

v4:
- Drop MIP change patch
- Add a Floating Point fixup patch
v3:
- Change names of all GP registers
- Add two more patches
v2:
- Small corrections based on feedback
- Remove the CSR permission check patch

Alistair Francis (6):
target/riscv: Don't set write permissions on dirty PTEs
riscv: plic: Remove unused interrupt functions target/riscv: Create function to test if FP is enabled target/riscv: Update the Hypervisor CSRs to v0.4 target/riscv: Fix mstatus dirty mask target/riscv: Use TB_FLAGS_MSTATUS_FS for floating point

Atish Patra (1):
target/riscv: Use both register name and ABI name

Quoted 8 lines
 hw/riscv/sifive_plic.c         | 12 ------------
 include/hw/riscv/sifive_plic.h |  3 ---
 target/riscv/cpu.c             | 19 ++++++++++--------
 target/riscv/cpu.h             |  6 +++++-
 target/riscv/cpu_bits.h        | 35 +++++++++++++++++-----------------
 target/riscv/cpu_helper.c      | 16 ++++++++++++----
 target/riscv/csr.c             | 22 +++++++++++----------
 7 files changed, 58 insertions(+), 55 deletions(-)

Aside from that PTE patch, I've applied

target/riscv: Use both register name and ABI name target/riscv: Fix mstatus dirty mask target/riscv: Use TB_FLAGS_MSTATUS_FS for floating point

on top of Bin's patch set, as the rest had made it into for-master. Like we talked about at lunch, I'm not sure the PTE one is actually correct -- it might just be paranoia, though.