Skip to contentSkip to conversation

RISC-V: KVM: provide UAPI for host SATP mode

PATCH Superseded Daniel Henrique Barboza 4 participants 7 messages Jul 5, 2023

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RISC-V: KVM: provide UAPI for host SATP mode

KVM userspaces need to be aware of the host SATP to allow them to advertise it back to the guest OS.

Since this information is used to build the guest FDT we can't wait for the SATP reg to be readable. We just need to read the SATP mode, thus we can use the existing 'satp_mode' global that represents the SATP reg with MODE set and both ASID and PPN cleared. E.g. for a 32 bit host running with sv32 satp_mode is 0x80000000, for a 64 bit host running sv57 satp_mode is 0xa000000000000000, and so on.

Add a new userspace virtual config register 'satp_mode' to allow userspace to read the current SATP mode the host is using with GET_ONE_REG API before spinning the vcpu.

'satp_mode' can't be changed via KVM, so SET_ONE_REG is allowed as long as userspace writes the existing 'satp_mode'.

---
 arch/riscv/include/asm/csr.h      | 2 ++
 arch/riscv/include/uapi/asm/kvm.h | 1 +
 arch/riscv/kvm/vcpu.c             | 7 +++++++
 3 files changed, 10 insertions(+)
arch/riscv/include/asm/csr.h +2 -0
index b6acb7ed115f..be6e5c305e5b 100644
@@ -46,6 +46,7 @@
 #ifndef CONFIG_64BIT
 #define SATP_PPN	_AC(0x003FFFFF, UL)
 #define SATP_MODE_32	_AC(0x80000000, UL)
+#define SATP_MODE_SHIFT	31
 #define SATP_ASID_BITS	9
 #define SATP_ASID_SHIFT	22
 #define SATP_ASID_MASK	_AC(0x1FF, UL)
@@ -54,6 +55,7 @@
 #define SATP_MODE_39	_AC(0x8000000000000000, UL)
 #define SATP_MODE_48	_AC(0x9000000000000000, UL)
 #define SATP_MODE_57	_AC(0xa000000000000000, UL)
+#define SATP_MODE_SHIFT	60
 #define SATP_ASID_BITS	16
 #define SATP_ASID_SHIFT	44
 #define SATP_ASID_MASK	_AC(0xFFFF, UL)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index f92790c9481a..0493c078e64e 100644
@@ -54,6 +54,7 @@ struct kvm_riscv_config {
 	unsigned long marchid;
 	unsigned long mimpid;
 	unsigned long zicboz_block_size;
+	unsigned long satp_mode;
 };
 
 /* CORE registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
arch/riscv/kvm/vcpu.c +7 -0
index 8bd9f2a8a0b9..b31acf923802 100644
@@ -313,6 +313,9 @@ static int kvm_riscv_vcpu_get_reg_config(struct kvm_vcpu *vcpu,
 	case KVM_REG_RISCV_CONFIG_REG(mimpid):
 		reg_val = vcpu->arch.mimpid;
 		break;
+	case KVM_REG_RISCV_CONFIG_REG(satp_mode):
+		reg_val = satp_mode >> SATP_MODE_SHIFT;
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -395,6 +398,10 @@ static int kvm_riscv_vcpu_set_reg_config(struct kvm_vcpu *vcpu,
 		else
 			return -EBUSY;
 		break;
+	case KVM_REG_RISCV_CONFIG_REG(satp_mode):
+		if (reg_val != (satp_mode >> SATP_MODE_SHIFT))
+			return -EINVAL;
+		break;
 	default:
 		return -EINVAL;
 	}
Signature · 1 line
2.41.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 84 lines · Daniel Henrique Barboza

KVM userspaces need to be aware of the host SATP to allow them to advertise it back to the guest OS.

Since this information is used to build the guest FDT we can't wait for the SATP reg to be readable. We just need to read the SATP mode, thus we can use the existing 'satp_mode' global that represents the SATP reg with MODE set and both ASID and PPN cleared. E.g. for a 32 bit host running with sv32 satp_mode is 0x80000000, for a 64 bit host running sv57 satp_mode is 0xa000000000000000, and so on.

Add a new userspace virtual config register 'satp_mode' to allow userspace to read the current SATP mode the host is using with GET_ONE_REG API before spinning the vcpu.

'satp_mode' can't be changed via KVM, so SET_ONE_REG is allowed as long as userspace writes the existing 'satp_mode'.

---
 arch/riscv/include/asm/csr.h      | 2 ++
 arch/riscv/include/uapi/asm/kvm.h | 1 +
 arch/riscv/kvm/vcpu.c             | 7 +++++++
 3 files changed, 10 insertions(+)
arch/riscv/include/asm/csr.h +2 -0
index b6acb7ed115f..be6e5c305e5b 100644
@@ -46,6 +46,7 @@
 #ifndef CONFIG_64BIT
 #define SATP_PPN	_AC(0x003FFFFF, UL)
 #define SATP_MODE_32	_AC(0x80000000, UL)
+#define SATP_MODE_SHIFT	31
 #define SATP_ASID_BITS	9
 #define SATP_ASID_SHIFT	22
 #define SATP_ASID_MASK	_AC(0x1FF, UL)
@@ -54,6 +55,7 @@
 #define SATP_MODE_39	_AC(0x8000000000000000, UL)
 #define SATP_MODE_48	_AC(0x9000000000000000, UL)
 #define SATP_MODE_57	_AC(0xa000000000000000, UL)
+#define SATP_MODE_SHIFT	60
 #define SATP_ASID_BITS	16
 #define SATP_ASID_SHIFT	44
 #define SATP_ASID_MASK	_AC(0xFFFF, UL)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index f92790c9481a..0493c078e64e 100644
@@ -54,6 +54,7 @@ struct kvm_riscv_config {
 	unsigned long marchid;
 	unsigned long mimpid;
 	unsigned long zicboz_block_size;
+	unsigned long satp_mode;
 };
 
 /* CORE registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
arch/riscv/kvm/vcpu.c +7 -0
index 8bd9f2a8a0b9..b31acf923802 100644
@@ -313,6 +313,9 @@ static int kvm_riscv_vcpu_get_reg_config(struct kvm_vcpu *vcpu,
 	case KVM_REG_RISCV_CONFIG_REG(mimpid):
 		reg_val = vcpu->arch.mimpid;
 		break;
+	case KVM_REG_RISCV_CONFIG_REG(satp_mode):
+		reg_val = satp_mode >> SATP_MODE_SHIFT;
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -395,6 +398,10 @@ static int kvm_riscv_vcpu_set_reg_config(struct kvm_vcpu *vcpu,
 		else
 			return -EBUSY;
 		break;
+	case KVM_REG_RISCV_CONFIG_REG(satp_mode):
+		if (reg_val != (satp_mode >> SATP_MODE_SHIFT))
+			return -EINVAL;
+		break;
 	default:
 		return -EINVAL;
 	}
Signature · 1 line
2.41.0
#2 lore raw
In reply to Daniel Henrique Barboza
Quoted 4 lines · Daniel Henrique Barboza

KVM userspaces need to be aware of the host SATP to allow them to advertise it back to the guest OS.

Since this information is used to build the guest FDT we can't wait for

The thing is the "mmu-type" property in the FDT is never used: the kernel will probe the hardware and choose the largest available mode, or use "no4lvl"/"no5lvl" from the command line to restrict this mode. And FYI the current mode is exposed through cpuinfo. @Conor Can we deprecate this node or something similar?

Just a remark, not sure that helps :)

Quoted 77 lines

the SATP reg to be readable. We just need to read the SATP mode, thus we can use the existing 'satp_mode' global that represents the SATP reg with MODE set and both ASID and PPN cleared. E.g. for a 32 bit host running with sv32 satp_mode is 0x80000000, for a 64 bit host running sv57 satp_mode is 0xa000000000000000, and so on.

Add a new userspace virtual config register 'satp_mode' to allow userspace to read the current SATP mode the host is using with GET_ONE_REG API before spinning the vcpu.

'satp_mode' can't be changed via KVM, so SET_ONE_REG is allowed as long as userspace writes the existing 'satp_mode'.

---
  arch/riscv/include/asm/csr.h      | 2 ++
  arch/riscv/include/uapi/asm/kvm.h | 1 +
  arch/riscv/kvm/vcpu.c             | 7 +++++++
  3 files changed, 10 insertions(+)
arch/riscv/include/asm/csr.h +2 -0
index b6acb7ed115f..be6e5c305e5b 100644
@@ -46,6 +46,7 @@
  #ifndef CONFIG_64BIT
  #define SATP_PPN	_AC(0x003FFFFF, UL)
  #define SATP_MODE_32	_AC(0x80000000, UL)
+#define SATP_MODE_SHIFT	31
  #define SATP_ASID_BITS	9
  #define SATP_ASID_SHIFT	22
  #define SATP_ASID_MASK	_AC(0x1FF, UL)
@@ -54,6 +55,7 @@
  #define SATP_MODE_39	_AC(0x8000000000000000, UL)
  #define SATP_MODE_48	_AC(0x9000000000000000, UL)
  #define SATP_MODE_57	_AC(0xa000000000000000, UL)
+#define SATP_MODE_SHIFT	60
  #define SATP_ASID_BITS	16
  #define SATP_ASID_SHIFT	44
  #define SATP_ASID_MASK	_AC(0xFFFF, UL)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index f92790c9481a..0493c078e64e 100644
@@ -54,6 +54,7 @@ struct kvm_riscv_config {
  	unsigned long marchid;
  	unsigned long mimpid;
  	unsigned long zicboz_block_size;
+	unsigned long satp_mode;
  };
 
  /* CORE registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
arch/riscv/kvm/vcpu.c +7 -0
index 8bd9f2a8a0b9..b31acf923802 100644
@@ -313,6 +313,9 @@ static int kvm_riscv_vcpu_get_reg_config(struct kvm_vcpu *vcpu,
  	case KVM_REG_RISCV_CONFIG_REG(mimpid):
  		reg_val = vcpu->arch.mimpid;
  		break;
+	case KVM_REG_RISCV_CONFIG_REG(satp_mode):
+		reg_val = satp_mode >> SATP_MODE_SHIFT;
+		break;
  	default:
  		return -EINVAL;
  	}
@@ -395,6 +398,10 @@ static int kvm_riscv_vcpu_set_reg_config(struct kvm_vcpu *vcpu,
  		else
  			return -EBUSY;
  		break;
+	case KVM_REG_RISCV_CONFIG_REG(satp_mode):
+		if (reg_val != (satp_mode >> SATP_MODE_SHIFT))
+			return -EINVAL;
+		break;
  	default:
  		return -EINVAL;
  	}
#3 lore raw
Quoted 11 lines · Alexandre Ghiti
Quoted 4 lines · Daniel Henrique Barboza

KVM userspaces need to be aware of the host SATP to allow them to advertise it back to the guest OS.

Since this information is used to build the guest FDT we can't wait for

The thing is the "mmu-type" property in the FDT is never used: the kernel will probe the hardware and choose the largest available mode, or use "no4lvl"/"no5lvl" from the command line to restrict this mode. And FYI the current mode is exposed through cpuinfo. @Conor Can we deprecate this node or something similar?

Just a remark, not sure that helps :)

It does, thanks. I am aware that the current mode is exposed through cpuinfo. mvendorid/marchid/mimpid is also exposed there. As far as I understand we should rely on KVM to provide all CPU related info to configure a vcpu though.

A little background of where I'm coming from. One of the QEMU KVM cpu types (host) doesn't have an assigned satp_mode. The FDT creation of the 'virt' board relies on that info being present, and the result is that the board will segfault. I sent a fix for it that I hope will be queued shortly:

https://lore.kernel.org/qemu-devel/20230630100811.287315-3-dbarboza at ventanamicro.com/

Thus, if it's decided that the satp_mode FDT is deprecated, we can ignore this patch altogether. Thanks,

Daniel

Quoted 78 lines
Quoted 78 lines

the SATP reg to be readable. We just need to read the SATP mode, thus we can use the existing 'satp_mode' global that represents the SATP reg with MODE set and both ASID and PPN cleared. E.g. for a 32 bit host running with sv32 satp_mode is 0x80000000, for a 64 bit host running sv57 satp_mode is 0xa000000000000000, and so on.

Add a new userspace virtual config register 'satp_mode' to allow userspace to read the current SATP mode the host is using with GET_ONE_REG API before spinning the vcpu.

'satp_mode' can't be changed via KVM, so SET_ONE_REG is allowed as long as userspace writes the existing 'satp_mode'.

---

? arch/riscv/include/asm/csr.h????? | 2 ++
? arch/riscv/include/uapi/asm/kvm.h | 1 +
? arch/riscv/kvm/vcpu.c???????????? | 7 +++++++

? 3 files changed, 10 insertions(+)

arch/riscv/include/asm/csr.h +0 -0
index b6acb7ed115f..be6e5c305e5b 100644
@@ -46,6 +46,7 @@

? #ifndef CONFIG_64BIT
? #define SATP_PPN??? _AC(0x003FFFFF, UL)
? #define SATP_MODE_32??? _AC(0x80000000, UL)
+#define SATP_MODE_SHIFT??? 31
? #define SATP_ASID_BITS??? 9
? #define SATP_ASID_SHIFT??? 22
? #define SATP_ASID_MASK??? _AC(0x1FF, UL)

arch/riscv/include/asm/csr.h +1 -0
@@ -54,6 +55,7 @@
 ? #define SATP_MODE_39??? _AC(0x8000000000000000, UL)
 ? #define SATP_MODE_48??? _AC(0x9000000000000000, UL)
 ? #define SATP_MODE_57??? _AC(0xa000000000000000, UL)
+#define SATP_MODE_SHIFT??? 60

? #define SATP_ASID_BITS??? 16
? #define SATP_ASID_SHIFT??? 44
? #define SATP_ASID_MASK??? _AC(0xFFFF, UL)

arch/riscv/include/uapi/asm/kvm.h +1 -0
index f92790c9481a..0493c078e64e 100644
@@ -54,6 +54,7 @@ struct kvm_riscv_config {
 ????? unsigned long marchid;
 ????? unsigned long mimpid;
 ????? unsigned long zicboz_block_size;
+??? unsigned long satp_mode;
 ? };

? /* CORE registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */

arch/riscv/kvm/vcpu.c +0 -0
index 8bd9f2a8a0b9..b31acf923802 100644
@@ -313,6 +313,9 @@ static int kvm_riscv_vcpu_get_reg_config(struct kvm_vcpu *vcpu,

????? case KVM_REG_RISCV_CONFIG_REG(mimpid):
????????? reg_val = vcpu->arch.mimpid;
????????? break;
+??? case KVM_REG_RISCV_CONFIG_REG(satp_mode):
+??????? reg_val = satp_mode >> SATP_MODE_SHIFT;
+??????? break;
????? default:
????????? return -EINVAL;
????? }

arch/riscv/kvm/vcpu.c +0 -0
@@ -395,6 +398,10 @@ static int kvm_riscv_vcpu_set_reg_config(struct kvm_vcpu *vcpu,

????????? else
????????????? return -EBUSY;
????????? break;
+??? case KVM_REG_RISCV_CONFIG_REG(satp_mode):
+??????? if (reg_val != (satp_mode >> SATP_MODE_SHIFT)) +??????????? return -EINVAL;
+??????? break;
????? default:
????????? return -EINVAL;
????? }

#4 lore raw
Quoted 24 lines · Daniel Henrique Barboza
Quoted 11 lines · Alexandre Ghiti
Quoted 4 lines · Daniel Henrique Barboza

KVM userspaces need to be aware of the host SATP to allow them to advertise it back to the guest OS.

Since this information is used to build the guest FDT we can't wait for

The thing is the "mmu-type" property in the FDT is never used: the kernel will probe the hardware and choose the largest available mode, or use "no4lvl"/"no5lvl" from the command line to restrict this mode. And FYI the current mode is exposed through cpuinfo. @Conor Can we deprecate this node or something similar?

Just a remark, not sure that helps :)

It does, thanks. I am aware that the current mode is exposed through cpuinfo. mvendorid/marchid/mimpid is also exposed there. As far as I understand we should rely on KVM to provide all CPU related info to configure a vcpu though.

A little background of where I'm coming from. One of the QEMU KVM cpu types (host) doesn't have an assigned satp_mode. The FDT creation of the 'virt' board relies on that info being present, and the result is that the board will segfault. I sent a fix for it that I hope will be queued shortly:

Quoted 4 lines

https://lore.kernel.org/qemu-devel/20230630100811.287315-3-dbarboza at ventanamicro.com/

Thus, if it's decided that the satp_mode FDT is deprecated, we can ignore this patch altogether. Thanks,

We'll eventually want the ability to get and set vsatp.mode from the VMM, so we'll want KVM_REG_RISCV_CONFIG_REG(satp_mode) anyway. For now, since we only support CPU passthrough with KVM, it's a convenient way to read the host's mode (while PPC qemu-kvm does read cpuinfo, I'm not aware of any other qemu-kvm doing that).

Thanks,
drew

#5 lore raw

Hey,

Quoted 6 lines · Andrew Jones
Quoted 5 lines · Daniel Henrique Barboza
Quoted 4 lines · Alexandre Ghiti
Quoted 3 lines · Daniel Henrique Barboza

KVM userspaces need to be aware of the host SATP to allow them to advertise it back to the guest OS.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line

Since this information is used to build the guest FDT we can't wait for

The thing is the "mmu-type" property in the FDT is never used:
the kernel will probe the hardware and choose the largest available mode, or use "no4lvl"/"no5lvl" from the command line to restrict this mode. And FYI the current mode is exposed through cpuinfo. @Conor Can we deprecate this node or something similar?

I'd be loathe to deprecate it just because Linux doesn't use it. I know nothing really of the BSDs or other operating systems, but from a quick check of FreeBSD:

sys/riscv/riscv/mp_machdep.c
400-static bool
401-cpu_check_mmu(u_int id __unused, phandle_t node, u_int addr_size __unused,
402- pcell_t *reg __unused)
403-{
404- char type[32];
405-
406- /* Check if this hart supports MMU. */
407: if (OF_getprop(node, "mmu-type", (void *)type, sizeof(type)) == -1 ||
408- strncmp(type, "riscv,none", 10) == 0)
409- return (false);
410-
411- return (true);
412-}

Similarly in U-Boot:

	mmu = dev_read_string(dev, "mmu-type");
	if (mmu)
		info->features |= BIT(CPU_FEAT_MMU);

(@Drew, I remember why now that the property is optional - nommu exists)

Seems like you should indeed propagate this to the DT you give to guests.

Cheers,
Conor.

Quoted 23 lines
Quoted 12 lines
Quoted 2 lines

Just a remark, not sure that helps :)

It does, thanks. I am aware that the current mode is exposed through cpuinfo. mvendorid/marchid/mimpid is also exposed there. As far as I understand we should rely on KVM to provide all CPU related info to configure a vcpu though.

A little background of where I'm coming from. One of the QEMU KVM cpu types (host) doesn't have an assigned satp_mode. The FDT creation of the 'virt' board relies on that info being present, and the result is that the board will segfault. I sent a fix for it that I hope will be queued shortly:

Quoted 4 lines

https://lore.kernel.org/qemu-devel/20230630100811.287315-3-dbarboza at ventanamicro.com/

Thus, if it's decided that the satp_mode FDT is deprecated, we can ignore this patch altogether. Thanks,

We'll eventually want the ability to get and set vsatp.mode from the VMM, so we'll want KVM_REG_RISCV_CONFIG_REG(satp_mode) anyway. For now, since we only support CPU passthrough with KVM, it's a convenient way to read the host's mode (while PPC qemu-kvm does read cpuinfo, I'm not aware of any other qemu-kvm doing that).

Quoted 7 lines

Thanks,
drew

_______________________________________________
linux-riscv mailing list
linux-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kvm-riscv/attachments/20230705/b0d88724/attachment.sig>