Skip to contentSkip to conversation

RISC-V: Fix Hypervisor guest user space

PATCH Applied Alistair Francis 5 participants 11 messages Mar 26, 2020 – Jun 27, 2020 RISC-V

lore
Share & tools
lore
#0 lore raw

cover RISC-V: Fix Hypervisor guest user space

This series fixes two bugs in the RISC-V two stage lookup implementation. This fixes the Hypervisor userspace failing to start.

Alistair Francis (2):
riscv: Don't use stage-2 PTE lookup protection flags
riscv: AND stage-1 and stage-2 protection flags

 target/riscv/cpu_helper.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
Signature · 1 line
2.26.0
#1 lore raw

PATCH 1/2 riscv: Don't use stage-2 PTE lookup protection flags

When doing the fist of a two stage lookup (Hypervisor extensions) don't set the current protection flags from the second stage lookup of the base address PTE.

---
 target/riscv/cpu_helper.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
target/riscv/cpu_helper.c +2 -1
index d3ba9efb02..f36d184b7b 100644
@@ -452,10 +452,11 @@ restart:
         hwaddr pte_addr;
 
         if (two_stage && first_stage) {
+            int vbase_prot;
             hwaddr vbase;
 
             /* Do the second stage translation on the base PTE address. */
-            get_physical_address(env, &vbase, prot, base, access_type,
+            get_physical_address(env, &vbase, &vbase_prot, base, access_type,
                                  mmu_idx, false, true);
 
             pte_addr = vbase + idx * ptesize;
Signature · 1 line
2.26.0
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH for 5.0 v1 1/2] riscv: Don't use stage-2 PTE lookup protection flags

Quoted 17 lines · Alistair Francis

When doing the fist of a two stage lookup (Hypervisor extensions) don't set the current protection flags from the second stage lookup of the base address PTE.

---
 target/riscv/cpu_helper.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
target/riscv/cpu_helper.c +0 -0
index d3ba9efb02..f36d184b7b 100644
@@ -452,10 +452,11 @@ restart:
         hwaddr pte_addr;
 

target/riscv/cpu_helper.c restart: · line 462

Quoted 10 lines
target/riscv/cpu_helper.c +2 -1
         if (two_stage && first_stage) {
+            int vbase_prot;
             hwaddr vbase;
 
             /* Do the second stage translation on the base PTE address. */
-            get_physical_address(env, &vbase, prot, base, access_type,
+            get_physical_address(env, &vbase, &vbase_prot, base, access_type,
                                  mmu_idx, false, true);
 
             pte_addr = vbase + idx * ptesize;

Certainly stage2 pte lookup has nothing to do with the original lookup, so using a new variable for prot is correct.

So as far as this minimal patch,

However, this bit of code doesn't look right:

(1) Similarly, what has the original access_type got to do with the PTE lookup? Seems like this should be MMU_DATA_LOAD always.

(2) Why is the get_physical_address return value ignored? On failure, surely this should be some sort of PTE lookup failure.

(3) Do we need to validate vbase_prot for write before updating the PTE for Access or Dirty? That seems like a loop-hole to allow silent modification of hypervisor read-only memory.

I do wonder if it might be easier to manage all of this by using additional TLBs to handle the stage2 and physical address spaces. That's probably too invasive for this stage of development though.

r~

#3 lore raw

Re: [PATCH for 5.0 v1 1/2] riscv: Don't use stage-2 PTE lookup protection flags

Quoted 33 lines · Richard Henderson
Quoted 17 lines · Alistair Francis

When doing the fist of a two stage lookup (Hypervisor extensions) don't set the current protection flags from the second stage lookup of the base address PTE.

---
 target/riscv/cpu_helper.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
target/riscv/cpu_helper.c +0 -0
index d3ba9efb02..f36d184b7b 100644
@@ -452,10 +452,11 @@ restart:
         hwaddr pte_addr;
 

target/riscv/cpu_helper.c restart: · line 462

Quoted 10 lines
target/riscv/cpu_helper.c +2 -1
         if (two_stage && first_stage) {
+            int vbase_prot;
             hwaddr vbase;
 
             /* Do the second stage translation on the base PTE address. */
-            get_physical_address(env, &vbase, prot, base, access_type,
+            get_physical_address(env, &vbase, &vbase_prot, base, access_type,
                                  mmu_idx, false, true);
 
             pte_addr = vbase + idx * ptesize;

Certainly stage2 pte lookup has nothing to do with the original lookup, so using a new variable for prot is correct.

Quoted 5 lines

So as far as this minimal patch,

However, this bit of code doesn't look right:

Thanks for the comments here. Coming back to this after a while.

Quoted 3 lines

(1) Similarly, what has the original access_type got to do with the PTE lookup? Seems like this should be MMU_DATA_LOAD always.

Fixed in master now

Quoted 3 lines

(2) Why is the get_physical_address return value ignored? On failure, surely this should be some sort of PTE lookup failure.

Also fixed in master now

Quoted 4 lines

(3) Do we need to validate vbase_prot for write before updating the PTE for Access or Dirty? That seems like a loop-hole to allow silent modification of hypervisor read-only memory.

That's a good point.

Updating the accessed bit seems correct to me as we did access it and that doesn't then provide write permissions.

Updating the dirty bit would provide write permissions, but we would only change the dirty bit on a store and vbase_prot is now always a load.

If the PTE was already dirty then we might incorrectly provide write access though.

Quoted 4 lines

I do wonder if it might be easier to manage all of this by using additional TLBs to handle the stage2 and physical address spaces. That's probably too invasive for this stage of development though.

Do you mean change riscv_cpu_mmu_index() to take into account virtulisation and have more then the current 3 (M, S and U) MMU indexes?

Alistair

Quoted 3 lines

r~

#4 lore raw

Re: [PATCH for 5.0 v1 1/2] riscv: Don't use stage-2 PTE lookup protection flags

Quoted 8 lines · Alistair Francis
Quoted 3 lines

(3) Do we need to validate vbase_prot for write before updating the PTE for Access or Dirty? That seems like a loop-hole to allow silent modification of hypervisor read-only memory.

That's a good point.

Updating the accessed bit seems correct to me as we did access it and that doesn't then provide write permissions.

I guess my first question is: Does the stage2 hypervisor pte provide read-only memory?

If not, all of this is moot.

However, if it does, consider:

(1) The guest os creates a stage1 page table with a leaf table within the read-only memory. This is obviously hokey.

(2) The guest os accesses a virtual address that utilizes the aforementioned PTE, the hardware (qemu) updates the accessed bit.

(3) The read-only page has now been modified. Oops.

Quoted 7 lines
Quoted 3 lines

I do wonder if it might be easier to manage all of this by using additional TLBs to handle the stage2 and physical address spaces. That's probably too invasive for this stage of development though.

Do you mean change riscv_cpu_mmu_index() to take into account virtulisation and have more then the current 3 (M, S and U) MMU indexes?

I had been thinking that you might be able to use some form of mmu-indexed load/lookup instead of address_space_ldq. Which would require 1 mmuidx that is physically mapped (same as M?) and another that uses only the hypervisor's second stage lookup.

r~

#5 lore raw

PATCH 2/2 riscv: AND stage-1 and stage-2 protection flags

In reply to Alistair Francis

Take the result of stage-1 and stage-2 page table walks and AND the two protection flags together. This way we require both to set permissions instead of just stage-2.

---
 target/riscv/cpu_helper.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
target/riscv/cpu_helper.c +5 -3
index f36d184b7b..50e13a064f 100644
@@ -707,7 +707,7 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
 #ifndef CONFIG_USER_ONLY
     vaddr im_address;
     hwaddr pa = 0;
-    int prot;
+    int prot, prot2;
     bool pmp_violation = false;
     bool m_mode_two_stage = false;
     bool hs_mode_two_stage = false;
@@ -757,13 +757,15 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
             /* Second stage lookup */
             im_address = pa;
 
-            ret = get_physical_address(env, &pa, &prot, im_address,
+            ret = get_physical_address(env, &pa, &prot2, im_address,
                                        access_type, mmu_idx, false, true);
 
             qemu_log_mask(CPU_LOG_MMU,
                     "%s 2nd-stage address=%" VADDR_PRIx " ret %d physical "
                     TARGET_FMT_plx " prot %d\n",
-                    __func__, im_address, ret, pa, prot);
+                    __func__, im_address, ret, pa, prot2);
+
+            prot &= prot2;
 
             if (riscv_feature(env, RISCV_FEATURE_PMP) &&
                 (ret == TRANSLATE_SUCCESS) &&
Signature · 1 line
2.26.0
Reviewed-by on v1 2/2
#6 lore raw

Re: [PATCH for 5.0 v1 2/2] riscv: AND stage-1 and stage-2 protection flags

Quoted 31 lines · Alistair Francis

Take the result of stage-1 and stage-2 page table walks and AND the two protection flags together. This way we require both to set permissions instead of just stage-2.

---
 target/riscv/cpu_helper.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
target/riscv/cpu_helper.c +2 -2
index f36d184b7b..50e13a064f 100644
@@ -707,7 +707,7 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
 #ifndef CONFIG_USER_ONLY
     vaddr im_address;
     hwaddr pa = 0;
-    int prot;
+    int prot, prot2;
     bool pmp_violation = false;
     bool m_mode_two_stage = false;
     bool hs_mode_two_stage = false;
@@ -757,13 +757,15 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
             /* Second stage lookup */
             im_address = pa;
 
-            ret = get_physical_address(env, &pa, &prot, im_address,
+            ret = get_physical_address(env, &pa, &prot2, im_address,
                                        access_type, mmu_idx, false, true);
 

target/riscv/cpu_helper.c bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size, · line 771

Quoted 10 lines
target/riscv/cpu_helper.c +3 -1
             qemu_log_mask(CPU_LOG_MMU,
                     "%s 2nd-stage address=%" VADDR_PRIx " ret %d physical "
                     TARGET_FMT_plx " prot %d\n",
-                    __func__, im_address, ret, pa, prot);
+                    __func__, im_address, ret, pa, prot2);
+
+            prot &= prot2;
 
             if (riscv_feature(env, RISCV_FEATURE_PMP) &&
                 (ret == TRANSLATE_SUCCESS) &&

Whee! Yes, I think this is what you've been looking for.

r~

#7 lore raw

Re: [PATCH for 5.0 v1 2/2] riscv: AND stage-1 and stage-2 protection flags

Quoted 40 lines · Richard Henderson
Quoted 38 lines · Alistair Francis

Take the result of stage-1 and stage-2 page table walks and AND the two protection flags together. This way we require both to set permissions instead of just stage-2.

---
 target/riscv/cpu_helper.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
target/riscv/cpu_helper.c +5 -3
index f36d184b7b..50e13a064f 100644
@@ -707,7 +707,7 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
 #ifndef CONFIG_USER_ONLY
     vaddr im_address;
     hwaddr pa = 0;
-    int prot;
+    int prot, prot2;
     bool pmp_violation = false;
     bool m_mode_two_stage = false;
     bool hs_mode_two_stage = false;
@@ -757,13 +757,15 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
             /* Second stage lookup */
             im_address = pa;
 
-            ret = get_physical_address(env, &pa, &prot, im_address,
+            ret = get_physical_address(env, &pa, &prot2, im_address,
                                        access_type, mmu_idx, false, true);
 
             qemu_log_mask(CPU_LOG_MMU,
                     "%s 2nd-stage address=%" VADDR_PRIx " ret %d physical "
                     TARGET_FMT_plx " prot %d\n",
-                    __func__, im_address, ret, pa, prot);
+                    __func__, im_address, ret, pa, prot2);
+
+            prot &= prot2;
 
Quoted 5 lines

target/riscv/cpu_helper.c bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size, · line 771

Quoted 3 lines
target/riscv/cpu_helper.c +0 -0
             if (riscv_feature(env, RISCV_FEATURE_PMP) &&
                 (ret == TRANSLATE_SUCCESS) &&

Whee! Yes, I think this is what you've been looking for.

Yep!

I actually tried this ages ago, but it didn't work without the first path so it never fixed the problem.

Quoted 1 line

Thanks

Alistair

Quoted 3 lines

r~

maintainer
#8 lore raw
In reply to Alistair Francis
Quoted 9 lines · Alistair Francis

This series fixes two bugs in the RISC-V two stage lookup implementation. This fixes the Hypervisor userspace failing to start.

Alistair Francis (2):
riscv: Don't use stage-2 PTE lookup protection flags
riscv: AND stage-1 and stage-2 protection flags

 target/riscv/cpu_helper.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

Thanks, these are in the queue.

Tested-by
#9 lore raw

Hi Palmer,

Quoted 5 lines · Palmer Dabbelt
Quoted 3 lines · Alistair Francis

This series fixes two bugs in the RISC-V two stage lookup implementation. This fixes the Hypervisor userspace failing to start.

Quoted 9 lines
Quoted 6 lines

Alistair Francis (2):
riscv: Don't use stage-2 PTE lookup protection flags
riscv: AND stage-1 and stage-2 protection flags

 target/riscv/cpu_helper.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

Thanks, these are in the queue.

I have tested this patch series on latest QEMU master without "target/riscv: Don't set write permissions on dirty PTEs" workaround patch. It works fine now.

Please drop the work-around patch "target/riscv: Don't set write permissions on dirty PTEs" from your for-next.

Regards,
Anup

#10 lore raw
In reply to Alistair Francis
Quoted 7 lines · Alistair Francis

This series fixes two bugs in the RISC-V two stage lookup implementation. This fixes the Hypervisor userspace failing to start.

Alistair Francis (2):
riscv: Don't use stage-2 PTE lookup protection flags
riscv: AND stage-1 and stage-2 protection flags

Applied to the RISC-V tree for 5.1

Alistair

Quoted 7 lines
 target/riscv/cpu_helper.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

--
2.26.0