Skip to contentSkip to conversation

KVM: arm64: Prevent vcpu_has_ptrauth from generating OOL functions

PATCH In review Marc Zyngier 3 participants 5 messages Jul 22, 2020 – Jul 23, 2020

Subsystems (3)
lore
Share & tools
lore

Marc Zyngier

author maintainer
#0 lore raw

PATCH 1/1 KVM: arm64: Prevent vcpu_has_ptrauth from generating OOL functions

So far, vcpu_has_ptrauth() is implemented in terms of system_supports_*_auth() calls, which are declared "inline". In some specific conditions (clang and SCS), the "inline" very much turns into an "out of line", which leads to a fireworks when this predicate is evaluated on a non-VHE system (right at the beginning of __hyp_handle_ptrauth).

Instead, make sure vcpu_has_ptrauth gets expanded inline by directly using the cpus_have_final_cap() helpers, which are __always_inline, generate much better code, and are the only thing that make sense when running at EL2 on a nVHE system.

Fixes 29eb5a3c57f7 ("KVM: arm64: Handle PtrAuth traps early")
---
 arch/arm64/include/asm/kvm_host.h | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_host.h +8 -3
index 147064314abf..a8278f6873e6 100644
@@ -391,9 +391,14 @@ struct kvm_vcpu_arch {
 #define vcpu_has_sve(vcpu) (system_supports_sve() && \
 			    ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_SVE))
 
-#define vcpu_has_ptrauth(vcpu)	((system_supports_address_auth() || \
-				  system_supports_generic_auth()) && \
-				 ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_PTRAUTH))
+#ifdef CONFIG_ARM64_PTR_AUTH
+#define vcpu_has_ptrauth(vcpu)						\
+	((cpus_have_final_cap(ARM64_HAS_ADDRESS_AUTH) ||		\
+	  cpus_have_final_cap(ARM64_HAS_GENERIC_AUTH)) &&		\
+	 (vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_PTRAUTH)
+#else
+#define vcpu_has_ptrauth(vcpu)		false
+#endif
 
 #define vcpu_gp_regs(v)		(&(v)->arch.ctxt.gp_regs)
 
Signature · 1 line
2.28.0.rc0.142.g3c755180ce-goog
Reviewed-by Tested-by on v1 1/1
#1 lore raw
Quoted 11 lines · Marc Zyngier

So far, vcpu_has_ptrauth() is implemented in terms of system_supports_*_auth() calls, which are declared "inline". In some specific conditions (clang and SCS), the "inline" very much turns into an "out of line", which leads to a fireworks when this predicate is evaluated on a non-VHE system (right at the beginning of __hyp_handle_ptrauth).

Instead, make sure vcpu_has_ptrauth gets expanded inline by directly using the cpus_have_final_cap() helpers, which are __always_inline, generate much better code, and are the only thing that make sense when running at EL2 on a nVHE system.

Quoted 4 lines
Fixes 29eb5a3c57f7 ("KVM: arm64: Handle PtrAuth traps early")

Thank you for the quick fix! I have booted a mainline kernel with this patch with Shadow Call Stack enabled and verified that using KVM no longer causes a panic.

For the future, is there an easy way to tell which type of system I am using (nVHE or VHE)? I am new to the arm64 KVM world but it is something that I am going to continue to test with various clang technologies now that I have actual hardware capable of it that can run a mainline kernel.

Cheers,
Nathan

Quoted 29 lines
---
 arch/arm64/include/asm/kvm_host.h | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_host.h +8 -3
index 147064314abf..a8278f6873e6 100644
@@ -391,9 +391,14 @@ struct kvm_vcpu_arch {
 #define vcpu_has_sve(vcpu) (system_supports_sve() && \
 			    ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_SVE))
 
-#define vcpu_has_ptrauth(vcpu)	((system_supports_address_auth() || \
-				  system_supports_generic_auth()) && \
-				 ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_PTRAUTH))
+#ifdef CONFIG_ARM64_PTR_AUTH
+#define vcpu_has_ptrauth(vcpu)						\
+	((cpus_have_final_cap(ARM64_HAS_ADDRESS_AUTH) ||		\
+	  cpus_have_final_cap(ARM64_HAS_GENERIC_AUTH)) &&		\
+	 (vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_PTRAUTH)
+#else
+#define vcpu_has_ptrauth(vcpu)		false
+#endif
 
 #define vcpu_gp_regs(v)		(&(v)->arch.ctxt.gp_regs)
 
Signature · 1 line
2.28.0.rc0.142.g3c755180ce-goog
reviewer
#2 lore raw

Hi Nathan,

Quoted 2 lines · Nathan Chancellor

For the future, is there an easy way to tell which type of system I am using (nVHE or VHE)?

afaict the easiest way is looking at the kernel log and you will find something like "{VHE,Hyp} mode initialized successfully". I can get the following message on my *VHE* box:

 # cat /var/log/dmesg | grep kvm
[    4.896295] kvm [1]: IPA Size Limit: 48bits
[    4.896339] [...]
[    4.899407] kvm [1]: VHE mode initialized successfully
                        ^^^

Have a look at kvm_arch_init(). With VHE, the host kernel is running at EL2 (aka Hyp mode).

Thanks,
Zenghui

author maintainer
#3 lore raw
In reply to Nathan Chancellor

Hi Nathan,

Quoted 12 lines · Nathan Chancellor
Quoted 11 lines · Marc Zyngier

So far, vcpu_has_ptrauth() is implemented in terms of system_supports_*_auth() calls, which are declared "inline". In some specific conditions (clang and SCS), the "inline" very much turns into an "out of line", which leads to a fireworks when this predicate is evaluated on a non-VHE system (right at the beginning of __hyp_handle_ptrauth).

Instead, make sure vcpu_has_ptrauth gets expanded inline by directly using the cpus_have_final_cap() helpers, which are __always_inline, generate much better code, and are the only thing that make sense when running at EL2 on a nVHE system.

Quoted 8 lines
Quoted 4 lines
Fixes 29eb5a3c57f7 ("KVM: arm64: Handle PtrAuth traps early")

Thank you for the quick fix! I have booted a mainline kernel with this patch with Shadow Call Stack enabled and verified that using KVM no longer causes a panic.

Great! I'll try and ferry this to mainline as quickly as possible.

Quoted 8 lines

For the future, is there an easy way to tell which type of system I am using (nVHE or VHE)? I am new to the arm64 KVM world but it is something that I am going to continue to test with various clang technologies now that I have actual hardware capable of it that can run a mainline kernel.

ARMv8.0 CPUs are only capable of running non-VHE. So if you have something based on older ARM CPUs (such as A57, A72, A53, A73, A35...), or licensee CPUs (ThunderX, XGene, EMag...), this will only run non-VHE (the host kernel runs at EL1, while the hypervisor runs at EL2.

From ARMv8.1 onward, VHE is normally present, and the host kernel can run at EL2 directly. ARM CPUs include A55, A65, A75, A76, A77, N1, while licensee CPUs include TX2, Kunpeng 920, and probably some more.

As pointed out by Zenghui in another email, KVM shows which mode it is using. Even without KVM, the kernel prints very early on:

[    0.000000] CPU features: detected: Virtualization Host Extensions

Note that this is only a performance difference, and that most features that are supported by the CPU can be used by KVM in either mode.

Thanks again,

        M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#4 lore raw
Quoted 21 lines · Marc Zyngier

Hi Nathan,

Quoted 18 lines · Nathan Chancellor
Quoted 16 lines · Marc Zyngier

So far, vcpu_has_ptrauth() is implemented in terms of system_supports_*_auth()
calls, which are declared "inline". In some specific conditions (clang and SCS), the "inline" very much turns into an "out of line", which leads to a fireworks when this predicate is evaluated on a non-VHE system (right at the beginning of __hyp_handle_ptrauth).

Instead, make sure vcpu_has_ptrauth gets expanded inline by directly using the cpus_have_final_cap() helpers, which are __always_inline, generate much better code, and are the only thing that make sense when running at EL2 on a nVHE system.

Fixes 29eb5a3c57f7 ("KVM: arm64: Handle PtrAuth traps early")
Quoted 5 lines
Quoted 3 lines

Thank you for the quick fix! I have booted a mainline kernel with this patch with Shadow Call Stack enabled and verified that using KVM no longer causes a panic.

Great! I'll try and ferry this to mainline as quickly as possible.

Awesome, I will keep an eye out.

Quoted 25 lines
Quoted 8 lines

For the future, is there an easy way to tell which type of system I am using (nVHE or VHE)? I am new to the arm64 KVM world but it is something that I am going to continue to test with various clang technologies now that I have actual hardware capable of it that can run a mainline kernel.

ARMv8.0 CPUs are only capable of running non-VHE. So if you have something based on older ARM CPUs (such as A57, A72, A53, A73, A35...), or licensee CPUs (ThunderX, XGene, EMag...), this will only run non-VHE (the host kernel runs at EL1, while the hypervisor runs at EL2.

From ARMv8.1 onward, VHE is normally present, and the host kernel can run at EL2 directly. ARM CPUs include A55, A65, A75, A76, A77, N1, while licensee CPUs include TX2, Kunpeng 920, and probably some more.

As pointed out by Zenghui in another email, KVM shows which mode it is using. Even without KVM, the kernel prints very early on:

[    0.000000] CPU features: detected: Virtualization Host Extensions
Quoted 9 lines

Note that this is only a performance difference, and that most features that are supported by the CPU can be used by KVM in either mode.

Thanks again,

        M.
Signature · 1 line
Jazz is not dead. It just smells funny...

Excellent, thank you both for the in-depth explanation. Hopefully my test farm continues to grow so I can stay on top of testing this stuff.

Cheers,
Nathan