Skip to contentSkip to conversation

Tweaks to the kvm-arm.mode= early cmdline parsing

PATCH Applied Will Deacon 3 participants 8 messages May 1, 2024 – May 8, 2024

Subsystems (2)
lore
Share & tools
lore

Will Deacon

author maintainer
#0 lore raw

cover Tweaks to the kvm-arm.mode= early cmdline parsing

Hi folks,

Here are a couple of tweaks to the early parsing of the 'kvm-arm.mode=' cmdline option so that:

1. kvm-arm.mode=nvhe overrides an earlier arm64_sw.hvhe=1 option
2. Protected KVM initialises in hVHE if VH is available in the CPU

This is useful for Android, where we'd like the default cmdline to work on all CPUs (i.e. with and without VH) but using hVHE by default where available and finally allowing nVHE to be forced by the bootloader appending additional arguments.

The patches could probably be squashed, but I kept them separate because they're doing different things and this code is quite fiddly when you start reasoning about the possible interactions between all of the options.

Cheers,

Will

Cc Marc Zyngier <[email protected]>
Cc Oliver Upton <[email protected]>

--->8

Will Deacon (2):
KVM: arm64: Fix hvhe/nvhe early alias parsing
KVM: arm64: Use hVHE in pKVM by default on CPUs with VHE support

 arch/arm64/kernel/pi/idreg-override.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Signature · 1 line
2.45.0.rc0.197.gbae5840b3b-goog
author maintainer
#1 lore raw

PATCH 1/2 KVM: arm64: Fix hvhe/nvhe early alias parsing

Booting a kernel with "arm64_sw.hvhe=1 kvm-arm.mode=nvhe" on the command-line results in KVM initialising using hVHE, whereas one might expect the latter option to override the former.

Fix this by adding "arm64_sw.hvhe=0" to the alias expansion for "kvm-arm.mode=nvhe".

---
 arch/arm64/kernel/pi/idreg-override.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kernel/pi/idreg-override.c +1 -1
index aad399796e81..39c9253fcf23 100644
@@ -209,7 +209,7 @@ static const struct {
 	char	alias[FTR_ALIAS_NAME_LEN];
 	char	feature[FTR_ALIAS_OPTION_LEN];
 } aliases[] __initconst = {
-	{ "kvm_arm.mode=nvhe",		"id_aa64mmfr1.vh=0" },
+	{ "kvm_arm.mode=nvhe",		"arm64_sw.hvhe=0 id_aa64mmfr1.vh=0" },
 	{ "kvm_arm.mode=protected",	"id_aa64mmfr1.vh=0" },
 	{ "arm64.nosve",		"id_aa64pfr0.sve=0" },
 	{ "arm64.nosme",		"id_aa64pfr1.sme=0" },
Signature · 1 line
2.45.0.rc0.197.gbae5840b3b-goog
#2 lore raw

Re: [PATCH 1/2] KVM: arm64: Fix hvhe/nvhe early alias parsing

Quoted 6 lines · Will Deacon

Booting a kernel with "arm64_sw.hvhe=1 kvm-arm.mode=nvhe" on the command-line results in KVM initialising using hVHE, whereas one might expect the latter option to override the former.

Fix this by adding "arm64_sw.hvhe=0" to the alias expansion for "kvm-arm.mode=nvhe".

Hmm, I wonder if it'd be better to just evaluate the sanitised VH field in hvhe_possible(). Otherwise I worry about keeping aliases in sync when new command line options come along.

This is similar to what we had before commit 35876f35f482 ("arm64:
cpufeature: Add helper to test for CPU feature overrides") w/ the added use of the sanitised reg.

Thoughts?

arch/arm64/kernel/cpufeature.c +6 -0
index 56583677c1f2..3bd5f00a8db3 100644
@@ -2138,6 +2138,12 @@ static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap,
 static bool hvhe_possible(const struct arm64_cpu_capabilities *entry,
 			  int __unused)
 {
+	u64 val = read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1);
+
+	/* No VHE? Then no hVHE for you either. */
+	if (!SYS_FIELD_GET(ID_AA64MMFR1_EL1, VH, val))
+		return false;
+
 	return arm64_test_sw_feature_override(ARM64_SW_FEATURE_OVERRIDE_HVHE);
 }
 
Signature · 2 lines
Thanks,
Oliver
author maintainer
#3 lore raw

Re: [PATCH 1/2] KVM: arm64: Fix hvhe/nvhe early alias parsing

Hey Oliver,

Quoted 12 lines · Oliver Upton
Quoted 6 lines · Will Deacon

Booting a kernel with "arm64_sw.hvhe=1 kvm-arm.mode=nvhe" on the command-line results in KVM initialising using hVHE, whereas one might expect the latter option to override the former.

Fix this by adding "arm64_sw.hvhe=0" to the alias expansion for "kvm-arm.mode=nvhe".

Hmm, I wonder if it'd be better to just evaluate the sanitised VH field in hvhe_possible(). Otherwise I worry about keeping aliases in sync when new command line options come along.

Quoted 5 lines

This is similar to what we had before commit 35876f35f482 ("arm64:
cpufeature: Add helper to test for CPU feature overrides") w/ the added use of the sanitised reg.

Thoughts?

I think that goes wonky when you have the arguments the other way around:

	"kvm-arm.mode=nvhe arm64_sw.hvhe=1"

would end up using nVHE.

Will

Acked-by
#4 lore raw

Re: [PATCH 1/2] KVM: arm64: Fix hvhe/nvhe early alias parsing

Quoted 19 lines · Will Deacon

Hey Oliver,

Quoted 16 lines · Oliver Upton
Quoted 6 lines · Will Deacon

Booting a kernel with "arm64_sw.hvhe=1 kvm-arm.mode=nvhe" on the command-line results in KVM initialising using hVHE, whereas one might expect the latter option to override the former.

Fix this by adding "arm64_sw.hvhe=0" to the alias expansion for "kvm-arm.mode=nvhe".

Hmm, I wonder if it'd be better to just evaluate the sanitised VH field in hvhe_possible(). Otherwise I worry about keeping aliases in sync when new command line options come along.

This is similar to what we had before commit 35876f35f482 ("arm64:
cpufeature: Add helper to test for CPU feature overrides") w/ the added use of the sanitised reg.

Quoted 7 lines
Quoted 1 line

Thoughts?

I think that goes wonky when you have the arguments the other way around:

	"kvm-arm.mode=nvhe arm64_sw.hvhe=1"

would end up using nVHE.

Right... What I was hoping to get at is a simple set of arguments to test protected + nVHE, even on systems that support VHE. Although I suppose:

"kvm-arm.mode=protected id_aa64mmfr1.vh=0 arm64_sw.hvhe=0"

isn't that bad and would preserve precedence of later args. So, FWIW:

Signature · 2 lines
Thanks,
Oliver
author maintainer
#5 lore raw

PATCH 2/2 KVM: arm64: Use hVHE in pKVM by default on CPUs with VHE support

In reply to Will Deacon

The early command line parsing treats "kvm-arm.mode=protected" as an alias for "id_aa64mmfr1.vh=0", forcing the use of nVHE so that the host kernel runs at EL1 with the pKVM hypervisor at EL2.

With the introduction of hVHE support in ad744e8cb346 ("arm64: Allow arm64_sw.hvhe on command line"), the hypervisor can run using the EL2+0 translation regime. This is interesting for unusual CPUs that have VH stuck to 1, but also because it opens the possibility of a hypervisor "userspace" in the distant future which could be used to isolate vCPU contexts in the hypervisor (see Marc's talk from KVM Forum 2022 [1]).

Repaint the "kvm-arm.mode=protected" alias to map to "arm64_sw.hvhe=1", which will use hVHE on CPUs that support it and remain with nVHE otherwise.

[1] https://www.youtube.com/watch?v=1F_Mf2j9eIo

---
 arch/arm64/kernel/pi/idreg-override.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kernel/pi/idreg-override.c +1 -1
index 39c9253fcf23..c20549e43a77 100644
@@ -210,7 +210,7 @@ static const struct {
 	char	feature[FTR_ALIAS_OPTION_LEN];
 } aliases[] __initconst = {
 	{ "kvm_arm.mode=nvhe",		"arm64_sw.hvhe=0 id_aa64mmfr1.vh=0" },
-	{ "kvm_arm.mode=protected",	"id_aa64mmfr1.vh=0" },
+	{ "kvm_arm.mode=protected",	"arm64_sw.hvhe=1" },
 	{ "arm64.nosve",		"id_aa64pfr0.sve=0" },
 	{ "arm64.nosme",		"id_aa64pfr1.sme=0" },
 	{ "arm64.nobti",		"id_aa64pfr1.bt=0" },
Signature · 1 line
2.45.0.rc0.197.gbae5840b3b-goog
Acked-by on v1 2/2
#6 lore raw

Re: [PATCH 2/2] KVM: arm64: Use hVHE in pKVM by default on CPUs with VHE support

Quoted 11 lines · Will Deacon

The early command line parsing treats "kvm-arm.mode=protected" as an alias for "id_aa64mmfr1.vh=0", forcing the use of nVHE so that the host kernel runs at EL1 with the pKVM hypervisor at EL2.

With the introduction of hVHE support in ad744e8cb346 ("arm64: Allow arm64_sw.hvhe on command line"), the hypervisor can run using the EL2+0 translation regime. This is interesting for unusual CPUs that have VH stuck to 1, but also because it opens the possibility of a hypervisor "userspace" in the distant future which could be used to isolate vCPU contexts in the hypervisor (see Marc's talk from KVM Forum 2022 [1]).

Quoted 6 lines

Repaint the "kvm-arm.mode=protected" alias to map to "arm64_sw.hvhe=1", which will use hVHE on CPUs that support it and remain with nVHE otherwise.

[1] https://www.youtube.com/watch?v=1F_Mf2j9eIo

Signature · 2 lines
Thanks,
Oliver
maintainer Applied
#7 lore raw
In reply to Will Deacon
Quoted 6 lines · Will Deacon

Here are a couple of tweaks to the early parsing of the 'kvm-arm.mode=' cmdline option so that:

1. kvm-arm.mode=nvhe overrides an earlier arm64_sw.hvhe=1 option
2. Protected KVM initialises in hVHE if VH is available in the CPU

Quoted 6 lines

This is useful for Android, where we'd like the default cmdline to work on all CPUs (i.e. with and without VH) but using hVHE by default where available and finally allowing nVHE to be forced by the bootloader appending additional arguments.

[...]

Applied to next, thanks!

[1/2] KVM: arm64: Fix hvhe/nvhe early alias parsing

      commit: 3c142f9d02b992aec5d96b82917e4cc07850c4df
[2/2] KVM: arm64: Use hVHE in pKVM by default on CPUs with VHE support
      commit: 5053c3f0519cd4c746577e3a6a7756f7c04b03dd

Cheers,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.