Skip to contentSkip to conversation

RISC-V: KVM: Write hgatp register with valid mode bits

PATCH Superseded [email protected] 4 participants 10 messages Aug 18, 2025 – Aug 21, 2025

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 RISC-V: KVM: Write hgatp register with valid mode bits

From: Fangyu Yu <[email protected]>

According to the RISC-V Privileged Architecture Spec, when MODE=Bare is selected,software must write zero to the remaining fields of hgatp.

We have detected the valid mode supported by the HW before, So using a valid mode to detect how many vmid bits are supported.

---
Changes in v2:
- Fixed build error since kvm_riscv_gstage_mode() has been modified.

---
 arch/riscv/kvm/vmid.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/riscv/kvm/vmid.c +2 -1
index 3b426c800480..5f33625f4070 100644
@@ -14,6 +14,7 @@
 #include <linux/smp.h>
 #include <linux/kvm_host.h>
 #include <asm/csr.h>
+#include <asm/kvm_mmu.h>
 #include <asm/kvm_tlb.h>
 #include <asm/kvm_vmid.h>
 
@@ -28,7 +29,7 @@ void __init kvm_riscv_gstage_vmid_detect(void)
 
 	/* Figure-out number of VMID bits in HW */
 	old = csr_read(CSR_HGATP);
-	csr_write(CSR_HGATP, old | HGATP_VMID);
+	csr_write(CSR_HGATP, (kvm_riscv_gstage_mode << HGATP_MODE_SHIFT) | HGATP_VMID);
 	vmid_bits = csr_read(CSR_HGATP);
 	vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
 	vmid_bits = fls_long(vmid_bits);
Signature · 1 line
2.49.0
#1 lore raw
Quoted 8 lines · On Mon, Aug 18, 2025 at 1:42 PM <[email protected]> wrote:

From: Fangyu Yu <[email protected]>

According to the RISC-V Privileged Architecture Spec, when MODE=Bare is selected,software must write zero to the remaining fields of hgatp.

We have detected the valid mode supported by the HW before, So using a valid mode to detect how many vmid bits are supported.

Good catch! It's a bug. The code seems copied from asids_init(), whose old value is not bare mode. For real hardware, it would cause problems, but the qemu buggy code hides the problem.

It needs a tag: Fixes: fd7bb4a251df ("RISC-V: KVM: Implement VMID allocator")

Others, Reviewed-by: Guo Ren <[email protected]>

Quoted 34 lines

---
Changes in v2:
- Fixed build error since kvm_riscv_gstage_mode() has been modified.

---
 arch/riscv/kvm/vmid.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/riscv/kvm/vmid.c +2 -2
index 3b426c800480..5f33625f4070 100644
@@ -14,6 +14,7 @@
 #include <linux/smp.h>
 #include <linux/kvm_host.h>
 #include <asm/csr.h>
+#include <asm/kvm_mmu.h>
 #include <asm/kvm_tlb.h>
 #include <asm/kvm_vmid.h>
 
@@ -28,7 +29,7 @@ void __init kvm_riscv_gstage_vmid_detect(void)
 
        /* Figure-out number of VMID bits in HW */
        old = csr_read(CSR_HGATP);
-       csr_write(CSR_HGATP, old | HGATP_VMID);
+       csr_write(CSR_HGATP, (kvm_riscv_gstage_mode << HGATP_MODE_SHIFT) | HGATP_VMID);
        vmid_bits = csr_read(CSR_HGATP);
        vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
        vmid_bits = fls_long(vmid_bits);
--

2.49.0

Signature · 2 lines
Best Regards
 Guo Ren
Reviewed-by
#2 lore raw
Quoted 10 lines · Guo Ren
Quoted 8 lines · On Mon, Aug 18, 2025 at 1:42 PM <[email protected]> wrote:

From: Fangyu Yu <[email protected]>

According to the RISC-V Privileged Architecture Spec, when MODE=Bare is selected,software must write zero to the remaining fields of hgatp.

We have detected the valid mode supported by the HW before, So using a valid mode to detect how many vmid bits are supported.

Quoted 7 lines

Good catch! It's a bug. The code seems copied from asids_init(), whose old value is not bare mode. For real hardware, it would cause problems, but the qemu buggy code hides the problem.

It needs a tag: Fixes: fd7bb4a251df ("RISC-V: KVM: Implement VMID allocator")

Others, Reviewed-by: Guo Ren <[email protected]>

Sorry for the typo:

                                                             ^^
Quoted 40 lines
Quoted 33 lines

---
Changes in v2:
- Fixed build error since kvm_riscv_gstage_mode() has been modified.

---
 arch/riscv/kvm/vmid.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/riscv/kvm/vmid.c +2 -2
index 3b426c800480..5f33625f4070 100644
@@ -14,6 +14,7 @@
 #include <linux/smp.h>
 #include <linux/kvm_host.h>
 #include <asm/csr.h>
+#include <asm/kvm_mmu.h>
 #include <asm/kvm_tlb.h>
 #include <asm/kvm_vmid.h>
 
@@ -28,7 +29,7 @@ void __init kvm_riscv_gstage_vmid_detect(void)
 
        /* Figure-out number of VMID bits in HW */
        old = csr_read(CSR_HGATP);
-       csr_write(CSR_HGATP, old | HGATP_VMID);
+       csr_write(CSR_HGATP, (kvm_riscv_gstage_mode << HGATP_MODE_SHIFT) | HGATP_VMID);
        vmid_bits = csr_read(CSR_HGATP);
        vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
        vmid_bits = fls_long(vmid_bits);
--
Quoted 2 lines

2.49.0

--
Best Regards
Guo Ren

Signature · 2 lines
Best Regards
 Guo Ren
#3 lore raw

PATCH 1/1 RISC-V KVM: Remove unnecessary HGATP csr_read

From: "Guo Ren (Alibaba DAMO Academy)" <[email protected]>

The HGATP has been set to zero in gstage_mode_detect(), so there is no need to save the old context. Unify the code convention with gstage_mode_detect().

---
This cleanup is based on yufang.yu's vmid fixup patch.

---
 arch/riscv/kvm/vmid.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
arch/riscv/kvm/vmid.c +1 -4
index 5f33625f4070..abb1c2bf2542 100644
@@ -25,15 +25,12 @@ static DEFINE_SPINLOCK(vmid_lock);
 
 void __init kvm_riscv_gstage_vmid_detect(void)
 {
-	unsigned long old;
-
 	/* Figure-out number of VMID bits in HW */
-	old = csr_read(CSR_HGATP);
 	csr_write(CSR_HGATP, (kvm_riscv_gstage_mode << HGATP_MODE_SHIFT) | HGATP_VMID);
 	vmid_bits = csr_read(CSR_HGATP);
 	vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
 	vmid_bits = fls_long(vmid_bits);
-	csr_write(CSR_HGATP, old);
+	csr_write(CSR_HGATP, 0);
 
 	/* We polluted local TLB so flush all guest TLB */
 	kvm_riscv_local_hfence_gvma_all();
Signature · 1 line
2.40.1
#4 lore raw

v4 cover Fixup & optimize hgatp mode & vmid detect functions

In reply to Guo Ren

From: "Guo Ren (Alibaba DAMO Academy)" <[email protected]>

Here are serval fixup & optmizitions for hgatp detect according to the RISC-V Privileged Architecture Spec.

---
Changes in v4:
- Involve ("RISC-V: KVM: Prevent HGATP_MODE_BARE passed"), which explain why gstage_mode_detect needs reset HGATP to zero.

Changes in v3:
- Add "Fixes" tag.
- Involve("RISC-V: KVM: Remove unnecessary HGATP csr_read"), which depends on patch 1.

Changes in v2:
- Fixed build error since kvm_riscv_gstage_mode() has been modified.
---

Fangyu Yu (1):
RISC-V: KVM: Write hgatp register with valid mode bits

Guo Ren (Alibaba DAMO Academy) (2):
RISC-V: KVM: Remove unnecessary HGATP csr_read
RISC-V: KVM: Prevent HGATP_MODE_BARE passed

 arch/riscv/kvm/gstage.c | 27 ++++++++++++++++++++++++---
 arch/riscv/kvm/main.c   | 35 +++++++++++++++++------------------
 arch/riscv/kvm/vmid.c   |  8 +++-----
 3 files changed, 44 insertions(+), 26 deletions(-)
Signature · 1 line
2.40.1
#5 lore raw

PATCH v4 1/3 RISC-V: KVM: Write hgatp register with valid mode bits

From: Fangyu Yu <[email protected]>

According to the RISC-V Privileged Architecture Spec, when MODE=Bare is selected,software must write zero to the remaining fields of hgatp.

We have detected the valid mode supported by the HW before, So using a valid mode to detect how many vmid bits are supported.

---
 arch/riscv/kvm/vmid.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/riscv/kvm/vmid.c +2 -1
index 3b426c800480..5f33625f4070 100644
@@ -14,6 +14,7 @@
 #include <linux/smp.h>
 #include <linux/kvm_host.h>
 #include <asm/csr.h>
+#include <asm/kvm_mmu.h>
 #include <asm/kvm_tlb.h>
 #include <asm/kvm_vmid.h>
 
@@ -28,7 +29,7 @@ void __init kvm_riscv_gstage_vmid_detect(void)
 
 	/* Figure-out number of VMID bits in HW */
 	old = csr_read(CSR_HGATP);
-	csr_write(CSR_HGATP, old | HGATP_VMID);
+	csr_write(CSR_HGATP, (kvm_riscv_gstage_mode << HGATP_MODE_SHIFT) | HGATP_VMID);
 	vmid_bits = csr_read(CSR_HGATP);
 	vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
 	vmid_bits = fls_long(vmid_bits);
Signature · 1 line
2.40.1
#6 lore raw

PATCH v4 2/3 RISC-V: KVM: Remove unnecessary HGATP csr_read

In reply to [email protected]

From: "Guo Ren (Alibaba DAMO Academy)" <[email protected]>

The HGATP has been set to zero in gstage_mode_detect(), so there is no need to save the old context. Unify the code convention with gstage_mode_detect().

---
 arch/riscv/kvm/vmid.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
arch/riscv/kvm/vmid.c +1 -4
index 5f33625f4070..abb1c2bf2542 100644
@@ -25,15 +25,12 @@ static DEFINE_SPINLOCK(vmid_lock);
 
 void __init kvm_riscv_gstage_vmid_detect(void)
 {
-	unsigned long old;
-
 	/* Figure-out number of VMID bits in HW */
-	old = csr_read(CSR_HGATP);
 	csr_write(CSR_HGATP, (kvm_riscv_gstage_mode << HGATP_MODE_SHIFT) | HGATP_VMID);
 	vmid_bits = csr_read(CSR_HGATP);
 	vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
 	vmid_bits = fls_long(vmid_bits);
-	csr_write(CSR_HGATP, old);
+	csr_write(CSR_HGATP, 0);
 
 	/* We polluted local TLB so flush all guest TLB */
 	kvm_riscv_local_hfence_gvma_all();
Signature · 1 line
2.40.1
#7 lore raw

PATCH v4 3/3 RISC-V: KVM: Prevent HGATP_MODE_BARE passed

In reply to [email protected]

From: "Guo Ren (Alibaba DAMO Academy)" <[email protected]>

urrent kvm_riscv_gstage_mode_detect() assumes H-extension must have HGATP_MODE_SV39X4/SV32X4 at least, but the spec allows H-extension with HGATP_MODE_BARE alone. The KVM depends on !HGATP_MODE_BARE at least, so enhance the gstage-mode-detect to block HGATP_MODE_BARE.

Move gstage-mode-check closer to gstage-mode-detect to prevent unnecessary init.

---
 arch/riscv/kvm/gstage.c | 27 ++++++++++++++++++++++++---
 arch/riscv/kvm/main.c   | 35 +++++++++++++++++------------------
 2 files changed, 41 insertions(+), 21 deletions(-)
arch/riscv/kvm/gstage.c +24 -3
index 24c270d6d0e2..b67d60d722c2 100644
@@ -321,7 +321,7 @@ void __init kvm_riscv_gstage_mode_detect(void)
 	if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV57X4) {
 		kvm_riscv_gstage_mode = HGATP_MODE_SV57X4;
 		kvm_riscv_gstage_pgd_levels = 5;
-		goto skip_sv48x4_test;
+		goto done;
 	}
 
 	/* Try Sv48x4 G-stage mode */
@@ -329,10 +329,31 @@ void __init kvm_riscv_gstage_mode_detect(void)
 	if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV48X4) {
 		kvm_riscv_gstage_mode = HGATP_MODE_SV48X4;
 		kvm_riscv_gstage_pgd_levels = 4;
+		goto done;
 	}
-skip_sv48x4_test:
 
+	/* Try Sv39x4 G-stage mode */
+	csr_write(CSR_HGATP, HGATP_MODE_SV39X4 << HGATP_MODE_SHIFT);
+	if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV39X4) {
+		kvm_riscv_gstage_mode = HGATP_MODE_SV39X4;
+		kvm_riscv_gstage_pgd_levels = 3;
+		goto done;
+	}
+#else /* CONFIG_32BIT */
+	/* Try Sv32x4 G-stage mode */
+	csr_write(CSR_HGATP, HGATP_MODE_SV32X4 << HGATP_MODE_SHIFT);
+	if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV32X4) {
+		kvm_riscv_gstage_mode = HGATP_MODE_SV32X4;
+		kvm_riscv_gstage_pgd_levels = 2;
+		goto done;
+	}
+#endif
+
+	/* KVM depends on !HGATP_MODE_OFF */
+	kvm_riscv_gstage_mode = HGATP_MODE_OFF;
+	kvm_riscv_gstage_pgd_levels = 0;
+
+done:
 	csr_write(CSR_HGATP, 0);
 	kvm_riscv_local_hfence_gvma_all();
-#endif
 }
arch/riscv/kvm/main.c +17 -18
index 67c876de74ef..8ee7aaa74ddc 100644
@@ -93,6 +93,23 @@ static int __init riscv_kvm_init(void)
 		return rc;
 
 	kvm_riscv_gstage_mode_detect();
+	switch (kvm_riscv_gstage_mode) {
+	case HGATP_MODE_SV32X4:
+		str = "Sv32x4";
+		break;
+	case HGATP_MODE_SV39X4:
+		str = "Sv39x4";
+		break;
+	case HGATP_MODE_SV48X4:
+		str = "Sv48x4";
+		break;
+	case HGATP_MODE_SV57X4:
+		str = "Sv57x4";
+		break;
+	default:
+		return -ENODEV;
+	}
+	kvm_info("using %s G-stage page table format\n", str);
 
 	kvm_riscv_gstage_vmid_detect();
 
@@ -135,24 +152,6 @@ static int __init riscv_kvm_init(void)
 			 (rc) ? slist : "no features");
 	}
 
-	switch (kvm_riscv_gstage_mode) {
-	case HGATP_MODE_SV32X4:
-		str = "Sv32x4";
-		break;
-	case HGATP_MODE_SV39X4:
-		str = "Sv39x4";
-		break;
-	case HGATP_MODE_SV48X4:
-		str = "Sv48x4";
-		break;
-	case HGATP_MODE_SV57X4:
-		str = "Sv57x4";
-		break;
-	default:
-		return -ENODEV;
-	}
-	kvm_info("using %s G-stage page table format\n", str);
-
 	kvm_info("VMID %ld bits available\n", kvm_riscv_gstage_vmid_bits());
 
 	if (kvm_riscv_aia_available())
Signature · 1 line
2.40.1
Reviewed-by on v2 1/1
#8 lore raw
In reply to [email protected]
Quoted 9 lines · On Mon, Aug 18, 2025 at 01:42:07PM +0800, [email protected] wrote:

From: Fangyu Yu <[email protected]>

According to the RISC-V Privileged Architecture Spec, when MODE=Bare is selected,software must write zero to the remaining fields of hgatp.

We have detected the valid mode supported by the HW before, So using a valid mode to detect how many vmid bits are supported.

With Fixed tag, feel free to add:

Quoted 37 lines

---
Changes in v2:
- Fixed build error since kvm_riscv_gstage_mode() has been modified.

---
 arch/riscv/kvm/vmid.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/riscv/kvm/vmid.c +2 -1
index 3b426c800480..5f33625f4070 100644
@@ -14,6 +14,7 @@
 #include <linux/smp.h>
 #include <linux/kvm_host.h>
 #include <asm/csr.h>
+#include <asm/kvm_mmu.h>
 #include <asm/kvm_tlb.h>
 #include <asm/kvm_vmid.h>
 
@@ -28,7 +29,7 @@ void __init kvm_riscv_gstage_vmid_detect(void)
 
 	/* Figure-out number of VMID bits in HW */
 	old = csr_read(CSR_HGATP);
-	csr_write(CSR_HGATP, old | HGATP_VMID);
+	csr_write(CSR_HGATP, (kvm_riscv_gstage_mode << HGATP_MODE_SHIFT) | HGATP_VMID);
 	vmid_bits = csr_read(CSR_HGATP);
 	vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
 	vmid_bits = fls_long(vmid_bits);
Signature · 7 lines
2.49.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Reviewed-by on v2 1/1
#9 lore raw
In reply to [email protected]
Quoted 8 lines · On 8/18/2025 1:42 PM, [email protected] wrote:

From: Fangyu Yu <[email protected]>

According to the RISC-V Privileged Architecture Spec, when MODE=Bare is selected,software must write zero to the remaining fields of hgatp.

We have detected the valid mode supported by the HW before, So using a valid mode to detect how many vmid bits are supported.

Quoted 8 lines

---
Changes in v2:
- Fixed build error since kvm_riscv_gstage_mode() has been modified.

---
  arch/riscv/kvm/vmid.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

Thanks,
Nutty

Quoted 21 lines
arch/riscv/kvm/vmid.c +2 -1
index 3b426c800480..5f33625f4070 100644
@@ -14,6 +14,7 @@
  #include <linux/smp.h>
  #include <linux/kvm_host.h>
  #include <asm/csr.h>
+#include <asm/kvm_mmu.h>
  #include <asm/kvm_tlb.h>
  #include <asm/kvm_vmid.h>
 
@@ -28,7 +29,7 @@ void __init kvm_riscv_gstage_vmid_detect(void)
 
  	/* Figure-out number of VMID bits in HW */
  	old = csr_read(CSR_HGATP);
-	csr_write(CSR_HGATP, old | HGATP_VMID);
+	csr_write(CSR_HGATP, (kvm_riscv_gstage_mode << HGATP_MODE_SHIFT) | HGATP_VMID);
  	vmid_bits = csr_read(CSR_HGATP);
  	vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
  	vmid_bits = fls_long(vmid_bits);