Skip to contentSkip to conversation

Add ARMv8.3 pointer authentication for kvm guest

PATCH Superseded Amit Daniel Kachhap 5 participants 41 messages Feb 19, 2019 – Mar 5, 2019

Subsystems (5)
Documentation Arm (32-bit) KVM arm64 +1 more subsystems

v6 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

v6 cover Add ARMv8.3 pointer authentication for kvm guest

Hi,

This patch series adds pointer authentication support for KVM guest and is based on top of Linux 5.0-rc6. The basic patches in this series was originally posted by Mark Rutland earlier[1,2] and contains some history of this work.

Extension Overview:
=============================================

The ARMv8.3 pointer authentication extension adds functionality to detect modification of pointer values, mitigating certain classes of attack such as stack smashing, and making return oriented programming attacks harder.

The extension introduces the concept of a pointer authentication code (PAC), which is stored in some upper bits of pointers. Each PAC is derived from the original pointer, another 64-bit value (e.g. the stack pointer), and a secret 128-bit key.

New instructions are added which can be used to:

* Insert a PAC into a pointer
* Strip a PAC from a pointer
* Authenticate and strip a PAC from a pointer

The detailed description of ARMv8.3 pointer authentication support in userspace/kernel and can be found in Kristina's generic pointer authentication patch series[3].

KVM guest work:
==============================================

If pointer authentication is enabled for KVM guests then the new PAC instructions will not trap to EL2. If not then they may be ignored if in HINT region or trapped in EL2 as illegal instruction. Since KVM guest vcpu runs as a thread so they have a key initialized which will be used by PAC. When world switch happens between host and guest then this key is exchanged.

There were some review comments by Christoffer Dall in the original series [1,2,3] and this patch series tries to implement them.

The current v6 patch series contains most of the suggestions by James Morse, Kristina, Julien and Dave.

This patch series is based on just a single patch from Dave Martin [8] which add control checks for accessing sys registers.

Changes since v5 [7]: Major changes are listed below.

* Split hcr_el2 and mdcr_el2 save/restore in two patches.
* Reverted back save/restore of sys-reg keys as done in V4 [5]. There was suggestion by James Morse to use ptrauth utilities in a single place in arm core and use them from kvm. However this change deviates from the existing sys-reg implementations and is not scalable.
* Invoked the key switch C functions from __guest_enter/__guest_exit assembly.
* Host key save is now done inside vcpu_load.
* Reverted back masking of cpufeature ID registers for ptrauth when disabled from userpace.
* Reset of ptrauth key registers not done conditionally.
* Code and Documentation cleanup.

Changes since v4 [6]: Several suggestions from James Morse
* Move host registers to be saved/restored inside struct kvm_cpu_context.
* Similar to hcr_el2, save/restore mdcr_el2 register also.
* Added save routines for ptrauth keys in generic arm core and use them during KVM context switch.
* Defined a GCC attribute __no_ptrauth which discards generating ptrauth instructions in a function. This is taken from Kristina's earlier kernel pointer authentication support patches [4].
* Dropped a patch to mask cpufeature when not enabled from userspace and now only key registers are masked from register list.

Changes since v3 [5]:
* Use pointer authentication only when VHE is present as ARM8.3 implies ARM8.1 features to be present.
* Added lazy context handling of ptrauth instructions from V2 version again.
* Added more details in Documentation.

Changes since v2 [1,2]:
* Allow host and guest to have different HCR_EL2 settings and not just constant value HCR_HOST_VHE_FLAGS or HCR_HOST_NVHE_FLAGS.
* Optimise the reading of HCR_EL2 in host/guest switch by fetching it once during KVM initialisation state and using it later.
* Context switch pointer authentication keys when switching between guest and host. Pointer authentication was enabled in a lazy context earlier[2] and is removed now to make it simple. However it can be revisited later if there is significant performance issue.
* Added a userspace option to choose pointer authentication.
* Based on the userspace option, ptrauth cpufeature will be visible.
* Based on the userspace option, ptrauth key registers will be accessible.
* A small document is added on how to enable pointer authentication from userspace KVM API.

Looking for feedback and comments.

Thanks,
Amit

[1]: https://lore.kernel.org/lkml/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://lkml.org/lkml/2018/12/7/666
[4]: https://lore.kernel.org/lkml/[email protected]/
[5]: https://lkml.org/lkml/2018/10/17/594
[6]: https://lkml.org/lkml/2018/12/18/80
[7]: https://lkml.org/lkml/2019/1/28/49
[8]: https://lore.kernel.org/linux-arm-kernel/[email protected]/

Linux (5.0-rc6 based):

Amit Daniel Kachhap (5):
arm64/kvm: preserve host HCR_EL2 value
arm64/kvm: preserve host MDCR_EL2 value
arm64/kvm: context-switch ptrauth registers
arm64/kvm: add a userspace option to enable pointer authentication arm64/kvm: control accessibility of ptrauth key registers

 Documentation/arm64/pointer-authentication.txt |  13 ++-
 Documentation/virtual/kvm/api.txt              |   4 +
 arch/arm/include/asm/kvm_host.h                |   4 +-
 arch/arm64/include/asm/kvm_asm.h               |   2 +
 arch/arm64/include/asm/kvm_emulate.h           |  22 ++---
 arch/arm64/include/asm/kvm_host.h              |  45 ++++++++--
 arch/arm64/include/asm/kvm_hyp.h               |   9 +-
 arch/arm64/include/uapi/asm/kvm.h              |   1 +
 arch/arm64/kernel/traps.c                      |   1 +
 arch/arm64/kvm/debug.c                         |  28 ++----
 arch/arm64/kvm/guest.c                         |   2 +-
 arch/arm64/kvm/handle_exit.c                   |  21 +++--
 arch/arm64/kvm/hyp/Makefile                    |   1 +
 arch/arm64/kvm/hyp/entry.S                     |  17 ++++
 arch/arm64/kvm/hyp/ptrauth-sr.c                | 115 +++++++++++++++++++++++++
 arch/arm64/kvm/hyp/switch.c                    |  40 ++++-----
 arch/arm64/kvm/hyp/sysreg-sr.c                 |  27 +++++-
 arch/arm64/kvm/hyp/tlb.c                       |   6 +-
 arch/arm64/kvm/reset.c                         |   3 +
 arch/arm64/kvm/sys_regs.c                      |  66 +++++++++++---
 include/uapi/linux/kvm.h                       |   1 +
 virt/kvm/arm/arm.c                             |   4 +-
 22 files changed, 338 insertions(+), 94 deletions(-)
 create mode 100644 arch/arm64/kvm/hyp/ptrauth-sr.c

kvmtool:

Repo: git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git

Amit Daniel Kachhap (1):
arm/kvm: arm64: Add a vcpu feature for pointer authentication

 arm/aarch32/include/kvm/kvm-cpu-arch.h    | 1 +
 arm/aarch64/include/asm/kvm.h             | 1 +
 arm/aarch64/include/kvm/kvm-config-arch.h | 4 +++-
 arm/aarch64/include/kvm/kvm-cpu-arch.h    | 1 +
 arm/include/arm-common/kvm-config-arch.h  | 1 +
 arm/kvm-cpu.c                             | 6 ++++++
 include/linux/kvm.h                       | 1 +
 7 files changed, 14 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.4
#1 lore raw

PATCH v6 1/6 arm64/kvm: preserve host HCR_EL2 value

From: Mark Rutland <[email protected]>

When restoring HCR_EL2 for the host, KVM uses HCR_HOST_VHE_FLAGS, which is a constant value. This works today, as the host HCR_EL2 value is always the same, but this will get in the way of supporting extensions that require HCR_EL2 bits to be set conditionally for the host.

To allow such features to work without KVM having to explicitly handle every possible host feature combination, this patch has KVM save/restore for the host HCR when switching to/from a guest HCR. The saving of the register is done once during cpu hypervisor initialization state and is just restored after switch from guest.

For fetching HCR_EL2 during kvm initialisation, a hyp call is made using kvm_call_hyp and is helpful in NHVE case.

For the hyp TLB maintenance code, __tlb_switch_to_host_vhe() is updated to toggle the TGE bit with a RMW sequence, as we already do in __tlb_switch_to_guest_vhe().

The value of hcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

[Added __cpu_copy_hyp_conf, hcr_el2 field in struct kvm_cpu_context]
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>
---
 arch/arm/include/asm/kvm_host.h      |  2 ++
 arch/arm64/include/asm/kvm_asm.h     |  2 ++
 arch/arm64/include/asm/kvm_emulate.h | 22 +++++++++++-----------
 arch/arm64/include/asm/kvm_host.h    | 13 ++++++++++++-
 arch/arm64/include/asm/kvm_hyp.h     |  2 +-
 arch/arm64/kvm/guest.c               |  2 +-
 arch/arm64/kvm/hyp/switch.c          | 23 +++++++++++++----------
 arch/arm64/kvm/hyp/sysreg-sr.c       | 21 ++++++++++++++++++++-
 arch/arm64/kvm/hyp/tlb.c             |  6 +++++-
 virt/kvm/arm/arm.c                   |  1 +
 10 files changed, 68 insertions(+), 26 deletions(-)
arch/arm/include/asm/kvm_host.h +2 -0
index ca56537..05706b4 100644
@@ -273,6 +273,8 @@ static inline void __cpu_init_stage2(void)
 	kvm_call_hyp(__init_stage2_translation);
 }
 
+static inline void __cpu_copy_hyp_conf(void) {}
+
 static inline int kvm_arch_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 {
 	return 0;
arch/arm64/include/asm/kvm_asm.h +2 -0
index f5b79e9..8acd73f 100644
@@ -80,6 +80,8 @@ extern void __vgic_v3_init_lrs(void);
 
 extern u32 __kvm_get_mdcr_el2(void);
 
+extern void __kvm_populate_host_regs(void);
+
 /* Home-grown __this_cpu_{ptr,read} variants that always work at HYP */
 #define __hyp_this_cpu_ptr(sym)						\
 	({								\
arch/arm64/include/asm/kvm_emulate.h +11 -11
index 506386a..0dbe795 100644
@@ -50,25 +50,25 @@ void kvm_inject_pabt32(struct kvm_vcpu *vcpu, unsigned long addr);
 
 static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
 {
-	return !(vcpu->arch.hcr_el2 & HCR_RW);
+	return !(vcpu->arch.ctxt.hcr_el2 & HCR_RW);
 }
 
 static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
 {
-	vcpu->arch.hcr_el2 = HCR_GUEST_FLAGS;
+	vcpu->arch.ctxt.hcr_el2 = HCR_GUEST_FLAGS;
 	if (is_kernel_in_hyp_mode())
-		vcpu->arch.hcr_el2 |= HCR_E2H;
+		vcpu->arch.ctxt.hcr_el2 |= HCR_E2H;
 	if (cpus_have_const_cap(ARM64_HAS_RAS_EXTN)) {
 		/* route synchronous external abort exceptions to EL2 */
-		vcpu->arch.hcr_el2 |= HCR_TEA;
+		vcpu->arch.ctxt.hcr_el2 |= HCR_TEA;
 		/* trap error record accesses */
-		vcpu->arch.hcr_el2 |= HCR_TERR;
+		vcpu->arch.ctxt.hcr_el2 |= HCR_TERR;
 	}
 	if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB))
-		vcpu->arch.hcr_el2 |= HCR_FWB;
+		vcpu->arch.ctxt.hcr_el2 |= HCR_FWB;
 
 	if (test_bit(KVM_ARM_VCPU_EL1_32BIT, vcpu->arch.features))
-		vcpu->arch.hcr_el2 &= ~HCR_RW;
+		vcpu->arch.ctxt.hcr_el2 &= ~HCR_RW;
 
 	/*
 	 * TID3: trap feature register accesses that we virtualise.
@@ -76,22 +76,22 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
 	 * are currently virtualised.
 	 */
 	if (!vcpu_el1_is_32bit(vcpu))
-		vcpu->arch.hcr_el2 |= HCR_TID3;
+		vcpu->arch.ctxt.hcr_el2 |= HCR_TID3;
 }
 
 static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
 {
-	return (unsigned long *)&vcpu->arch.hcr_el2;
+	return (unsigned long *)&vcpu->arch.ctxt.hcr_el2;
 }
 
 static inline void vcpu_clear_wfe_traps(struct kvm_vcpu *vcpu)
 {
-	vcpu->arch.hcr_el2 &= ~HCR_TWE;
+	vcpu->arch.ctxt.hcr_el2 &= ~HCR_TWE;
 }
 
 static inline void vcpu_set_wfe_traps(struct kvm_vcpu *vcpu)
 {
-	vcpu->arch.hcr_el2 |= HCR_TWE;
+	vcpu->arch.ctxt.hcr_el2 |= HCR_TWE;
 }
 
 static inline unsigned long vcpu_get_vsesr(struct kvm_vcpu *vcpu)
arch/arm64/include/asm/kvm_host.h +12 -1
index 7732d0b..1b2e05b 100644
@@ -203,6 +203,8 @@ struct kvm_cpu_context {
 		u32 copro[NR_COPRO_REGS];
 	};
 
+	/* HYP host/guest configuration */
+	u64 hcr_el2;
 	struct kvm_vcpu *__hyp_running_vcpu;
 };
 
@@ -212,7 +214,6 @@ struct kvm_vcpu_arch {
 	struct kvm_cpu_context ctxt;
 
 	/* HYP configuration */
-	u64 hcr_el2;
 	u32 mdcr_el2;
 
 	/* Exception Information */
@@ -458,6 +459,16 @@ int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
 
 static inline void __cpu_init_stage2(void) {}
 
+/**
+ * __cpu_copy_hyp_conf - copy the boot hyp configuration registers
+ *
+ * It is called once per-cpu during CPU hyp initialisation.
+ */
+static inline void __cpu_copy_hyp_conf(void)
+{
+	kvm_call_hyp(__kvm_populate_host_regs);
+}
+
 /* Guest/host FPSIMD coordination helpers */
 int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu);
arch/arm64/include/asm/kvm_hyp.h +1 -1
index a80a7ef..6e65cad 100644
@@ -151,7 +151,7 @@ void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
 bool __fpsimd_enabled(void);
 
 void activate_traps_vhe_load(struct kvm_vcpu *vcpu);
-void deactivate_traps_vhe_put(void);
+void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu);
 
 u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
 void __noreturn __hyp_do_panic(unsigned long, ...);
arch/arm64/kvm/guest.c +1 -1
index dd436a5..e2f0268 100644
@@ -345,7 +345,7 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
 int __kvm_arm_vcpu_get_events(struct kvm_vcpu *vcpu,
 			      struct kvm_vcpu_events *events)
 {
-	events->exception.serror_pending = !!(vcpu->arch.hcr_el2 & HCR_VSE);
+	events->exception.serror_pending = !!(vcpu->arch.ctxt.hcr_el2 & HCR_VSE);
 	events->exception.serror_has_esr = cpus_have_const_cap(ARM64_HAS_RAS_EXTN);
 
 	if (events->exception.serror_pending && events->exception.serror_has_esr)
arch/arm64/kvm/hyp/switch.c +13 -10
index b0b1478..006bd33 100644
@@ -126,7 +126,7 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
 
 static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
 {
-	u64 hcr = vcpu->arch.hcr_el2;
+	u64 hcr = vcpu->arch.ctxt.hcr_el2;
 
 	write_sysreg(hcr, hcr_el2);
 
@@ -139,10 +139,10 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
 		__activate_traps_nvhe(vcpu);
 }
 
-static void deactivate_traps_vhe(void)
+static void deactivate_traps_vhe(struct kvm_cpu_context *host_ctxt)
 {
 	extern char vectors[];	/* kernel exception vectors */
-	write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
+	write_sysreg(host_ctxt->hcr_el2, hcr_el2);
 
 	/*
 	 * ARM erratum 1165522 requires the actual execution of the above
@@ -155,7 +155,7 @@ static void deactivate_traps_vhe(void)
 	write_sysreg(vectors, vbar_el1);
 }
 
-static void __hyp_text __deactivate_traps_nvhe(void)
+static void __hyp_text __deactivate_traps_nvhe(struct kvm_cpu_context *host_ctxt)
 {
 	u64 mdcr_el2 = read_sysreg(mdcr_el2);
 
@@ -165,25 +165,28 @@ static void __hyp_text __deactivate_traps_nvhe(void)
 	mdcr_el2 |= MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT;
 
 	write_sysreg(mdcr_el2, mdcr_el2);
-	write_sysreg(HCR_HOST_NVHE_FLAGS, hcr_el2);
+	write_sysreg(host_ctxt->hcr_el2, hcr_el2);
 	write_sysreg(CPTR_EL2_DEFAULT, cptr_el2);
 }
 
 static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
 {
+	struct kvm_cpu_context *host_ctxt;
+
+	host_ctxt = vcpu->arch.host_cpu_context;
 	/*
 	 * If we pended a virtual abort, preserve it until it gets
 	 * cleared. See D1.14.3 (Virtual Interrupts) for details, but
 	 * the crucial bit is "On taking a vSError interrupt,
 	 * HCR_EL2.VSE is cleared to 0."
 	 */
-	if (vcpu->arch.hcr_el2 & HCR_VSE)
-		vcpu->arch.hcr_el2 = read_sysreg(hcr_el2);
+	if (vcpu->arch.ctxt.hcr_el2 & HCR_VSE)
+		vcpu->arch.ctxt.hcr_el2 = read_sysreg(hcr_el2);
 
 	if (has_vhe())
-		deactivate_traps_vhe();
+		deactivate_traps_vhe(host_ctxt);
 	else
-		__deactivate_traps_nvhe();
+		__deactivate_traps_nvhe(host_ctxt);
 }
 
 void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
@@ -191,7 +194,7 @@ void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
 	__activate_traps_common(vcpu);
 }
 
-void deactivate_traps_vhe_put(void)
+void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu)
 {
 	u64 mdcr_el2 = read_sysreg(mdcr_el2);
 
arch/arm64/kvm/hyp/sysreg-sr.c +20 -1
index 68d6f7c..68ddc0f 100644
@@ -21,6 +21,7 @@
 #include <asm/kvm_asm.h>
 #include <asm/kvm_emulate.h>
 #include <asm/kvm_hyp.h>
+#include <asm/kvm_mmu.h>
 
 /*
  * Non-VHE: Both host and guest must save everything.
@@ -294,7 +295,7 @@ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
 	if (!has_vhe())
 		return;
 
-	deactivate_traps_vhe_put();
+	deactivate_traps_vhe_put(vcpu);
 
 	__sysreg_save_el1_state(guest_ctxt);
 	__sysreg_save_user_state(guest_ctxt);
@@ -316,3 +317,21 @@ void __hyp_text __kvm_enable_ssbs(void)
 	"msr	sctlr_el2, %0"
 	: "=&r" (tmp) : "L" (SCTLR_ELx_DSSBS));
 }
+
+/**
+ * __kvm_populate_host_regs - Stores host register values
+ *
+ * This function acts as a function handler parameter for kvm_call_hyp and
+ * may be called from EL1 exception level to fetch the register value.
+ */
+void __hyp_text __kvm_populate_host_regs(void)
+{
+	struct kvm_cpu_context *host_ctxt;
+
+	if (has_vhe())
+		host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
+	else
+		host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);
+
+	host_ctxt->hcr_el2 = read_sysreg(hcr_el2);
+}
arch/arm64/kvm/hyp/tlb.c +5 -1
index 76c3086..c5e7144 100644
@@ -86,12 +86,16 @@ static hyp_alternate_select(__tlb_switch_to_guest,
 static void __hyp_text __tlb_switch_to_host_vhe(struct kvm *kvm,
 						struct tlb_inv_context *cxt)
 {
+	u64 val;
+
 	/*
 	 * We're done with the TLB operation, let's restore the host's
 	 * view of HCR_EL2.
 	 */
 	write_sysreg(0, vttbr_el2);
-	write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
+	val = read_sysreg(hcr_el2);
+	val |= HCR_TGE;
+	write_sysreg(val, hcr_el2);
 	isb();
 
 	if (cpus_have_const_cap(ARM64_WORKAROUND_1165522)) {
virt/kvm/arm/arm.c +1 -0
index 9e350fd3..8e18f7f 100644
@@ -1328,6 +1328,7 @@ static void cpu_hyp_reinit(void)
 		cpu_init_hyp_mode(NULL);
 
 	kvm_arm_init_debug();
+	__cpu_copy_hyp_conf();
 
 	if (vgic_present)
 		kvm_vgic_init_cpu_hardware();
Signature · 1 line
2.7.4
reviewer
#2 lore raw

Re: [PATCH v6 1/6] arm64/kvm: preserve host HCR_EL2 value

Hi,

Quoted 29 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When restoring HCR_EL2 for the host, KVM uses HCR_HOST_VHE_FLAGS, which is a constant value. This works today, as the host HCR_EL2 value is always the same, but this will get in the way of supporting extensions that require HCR_EL2 bits to be set conditionally for the host.

To allow such features to work without KVM having to explicitly handle every possible host feature combination, this patch has KVM save/restore for the host HCR when switching to/from a guest HCR. The saving of the register is done once during cpu hypervisor initialization state and is just restored after switch from guest.

For fetching HCR_EL2 during kvm initialisation, a hyp call is made using kvm_call_hyp and is helpful in NHVE case.

For the hyp TLB maintenance code, __tlb_switch_to_host_vhe() is updated to toggle the TGE bit with a RMW sequence, as we already do in __tlb_switch_to_guest_vhe().

The value of hcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

[Added __cpu_copy_hyp_conf, hcr_el2 field in struct kvm_cpu_context]
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>

[...]

arch/arm64/include/asm/kvm_host.h int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, · line 467

Quoted 6 lines
arch/arm64/include/asm/kvm_host.h +6 -0
+/**
+ * __cpu_copy_hyp_conf - copy the boot hyp configuration registers
+ *
+ * It is called once per-cpu during CPU hyp initialisation.
+ */
+static inline void __cpu_copy_hyp_conf(void)

I think this would be better named as something like:

cpu_init_host_ctxt()

... as that makes it a bit clearer as to what is being initialized.

[...]

Quoted 4 lines
arch/arm64/kvm/hyp/sysreg-sr.c +4 -0
+/**
+ * __kvm_populate_host_regs - Stores host register values
+ *
+ * This function acts as a function handler parameter for kvm_call_hyp and

arch/arm64/kvm/hyp/sysreg-sr.c void __hyp_text __kvm_enable_ssbs(void) · line 334

Quoted 10 lines
arch/arm64/kvm/hyp/sysreg-sr.c +10 -0
+ * may be called from EL1 exception level to fetch the register value.
+ */
+void __hyp_text __kvm_populate_host_regs(void)
+{
+	struct kvm_cpu_context *host_ctxt;
+
+	if (has_vhe())
+		host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
+	else
+		host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);

Do we need the has_vhe() check here?

Can't we always do:

	host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);

... regardless of VHE? Or is that broken for VHE somehow?

Thanks,
Mark.

#3 lore raw

Re: [PATCH v6 1/6] arm64/kvm: preserve host HCR_EL2 value

Quoted 67 lines · Mark Rutland

Hi,

Quoted 29 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When restoring HCR_EL2 for the host, KVM uses HCR_HOST_VHE_FLAGS, which is a constant value. This works today, as the host HCR_EL2 value is always the same, but this will get in the way of supporting extensions that require HCR_EL2 bits to be set conditionally for the host.

To allow such features to work without KVM having to explicitly handle every possible host feature combination, this patch has KVM save/restore for the host HCR when switching to/from a guest HCR. The saving of the register is done once during cpu hypervisor initialization state and is just restored after switch from guest.

For fetching HCR_EL2 during kvm initialisation, a hyp call is made using kvm_call_hyp and is helpful in NHVE case.

For the hyp TLB maintenance code, __tlb_switch_to_host_vhe() is updated to toggle the TGE bit with a RMW sequence, as we already do in __tlb_switch_to_guest_vhe().

The value of hcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

[Added __cpu_copy_hyp_conf, hcr_el2 field in struct kvm_cpu_context]
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>

[...]

arch/arm64/include/asm/kvm_host.h int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, · line 467

Quoted 6 lines
arch/arm64/include/asm/kvm_host.h +6 -0
+/**
+ * __cpu_copy_hyp_conf - copy the boot hyp configuration registers
+ *
+ * It is called once per-cpu during CPU hyp initialisation.
+ */
+static inline void __cpu_copy_hyp_conf(void)

I think this would be better named as something like:

cpu_init_host_ctxt()

... as that makes it a bit clearer as to what is being initialized.

[...]

Quoted 4 lines
arch/arm64/kvm/hyp/sysreg-sr.c +4 -0
+/**
+ * __kvm_populate_host_regs - Stores host register values
+ *
+ * This function acts as a function handler parameter for kvm_call_hyp and

arch/arm64/kvm/hyp/sysreg-sr.c void __hyp_text __kvm_enable_ssbs(void) · line 334

Quoted 10 lines
arch/arm64/kvm/hyp/sysreg-sr.c +10 -0
+ * may be called from EL1 exception level to fetch the register value.
+ */
+void __hyp_text __kvm_populate_host_regs(void)
+{
+	struct kvm_cpu_context *host_ctxt;
+
+	if (has_vhe())
+		host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
+	else
+		host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);

Do we need the has_vhe() check here?

Quoted 5 lines

Can't we always do:

	host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);

... regardless of VHE? Or is that broken for VHE somehow?

The whole point of __hyp_this_cpu_ptr is that it is always valid... See 85478bab40917 for details.

Thanks,

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#4 lore raw

Re: [PATCH v6 1/6] arm64/kvm: preserve host HCR_EL2 value

In reply to Mark Rutland

Hi,

Quoted 44 lines · Mark Rutland

Hi,

Quoted 29 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When restoring HCR_EL2 for the host, KVM uses HCR_HOST_VHE_FLAGS, which is a constant value. This works today, as the host HCR_EL2 value is always the same, but this will get in the way of supporting extensions that require HCR_EL2 bits to be set conditionally for the host.

To allow such features to work without KVM having to explicitly handle every possible host feature combination, this patch has KVM save/restore for the host HCR when switching to/from a guest HCR. The saving of the register is done once during cpu hypervisor initialization state and is just restored after switch from guest.

For fetching HCR_EL2 during kvm initialisation, a hyp call is made using kvm_call_hyp and is helpful in NHVE case.

For the hyp TLB maintenance code, __tlb_switch_to_host_vhe() is updated to toggle the TGE bit with a RMW sequence, as we already do in __tlb_switch_to_guest_vhe().

The value of hcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

[Added __cpu_copy_hyp_conf, hcr_el2 field in struct kvm_cpu_context]
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>

[...]

arch/arm64/include/asm/kvm_host.h int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, · line 467

Quoted 6 lines
arch/arm64/include/asm/kvm_host.h +6 -0
+/**
+ * __cpu_copy_hyp_conf - copy the boot hyp configuration registers
+ *
+ * It is called once per-cpu during CPU hyp initialisation.
+ */
+static inline void __cpu_copy_hyp_conf(void)

I think this would be better named as something like:

Quoted 3 lines

cpu_init_host_ctxt()

... as that makes it a bit clearer as to what is being initialized.

ok, Agreed with the name.

Quoted 20 lines

[...]

Quoted 4 lines
arch/arm64/kvm/hyp/sysreg-sr.c +4 -0
+/**
+ * __kvm_populate_host_regs - Stores host register values
+ *
+ * This function acts as a function handler parameter for kvm_call_hyp and

arch/arm64/kvm/hyp/sysreg-sr.c void __hyp_text __kvm_enable_ssbs(void) · line 334

Quoted 10 lines
arch/arm64/kvm/hyp/sysreg-sr.c +10 -0
+ * may be called from EL1 exception level to fetch the register value.
+ */
+void __hyp_text __kvm_populate_host_regs(void)
+{
+	struct kvm_cpu_context *host_ctxt;
+
+	if (has_vhe())
+		host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
+	else
+		host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);

Do we need the has_vhe() check here?

Quoted 5 lines

Can't we always do:

	host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);

... regardless of VHE? Or is that broken for VHE somehow?

Yes it works fine for VHE. I missed this.

Thanks,
Amit

Quoted 4 lines

Thanks,
Mark.

#5 lore raw

Re: [PATCH v6 1/6] arm64/kvm: preserve host HCR_EL2 value

In reply to Amit Daniel Kachhap
Quoted 7 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When restoring HCR_EL2 for the host, KVM uses HCR_HOST_VHE_FLAGS, which is a constant value. This works today, as the host HCR_EL2 value is always the same, but this will get in the way of supporting extensions that require HCR_EL2 bits to be set conditionally for the host.

Quoted 8 lines

To allow such features to work without KVM having to explicitly handle every possible host feature combination, this patch has KVM save/restore for the host HCR when switching to/from a guest HCR. The saving of the register is done once during cpu hypervisor initialization state and is just restored after switch from guest.

For fetching HCR_EL2 during kvm initialisation, a hyp call is made using kvm_call_hyp and is helpful in NHVE case.

Minor nit: NVHE misspelled. This looks a bit like it's naming an arch feature rather than a kernel implementation detail though. Maybe write "non-VHE".

Quoted 57 lines

For the hyp TLB maintenance code, __tlb_switch_to_host_vhe() is updated to toggle the TGE bit with a RMW sequence, as we already do in __tlb_switch_to_guest_vhe().

The value of hcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

[Added __cpu_copy_hyp_conf, hcr_el2 field in struct kvm_cpu_context]
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>
---
 arch/arm/include/asm/kvm_host.h      |  2 ++
 arch/arm64/include/asm/kvm_asm.h     |  2 ++
 arch/arm64/include/asm/kvm_emulate.h | 22 +++++++++++-----------
 arch/arm64/include/asm/kvm_host.h    | 13 ++++++++++++-
 arch/arm64/include/asm/kvm_hyp.h     |  2 +-
 arch/arm64/kvm/guest.c               |  2 +-
 arch/arm64/kvm/hyp/switch.c          | 23 +++++++++++++----------
 arch/arm64/kvm/hyp/sysreg-sr.c       | 21 ++++++++++++++++++++-
 arch/arm64/kvm/hyp/tlb.c             |  6 +++++-
 virt/kvm/arm/arm.c                   |  1 +
 10 files changed, 68 insertions(+), 26 deletions(-)
arch/arm/include/asm/kvm_host.h +2 -0
index ca56537..05706b4 100644
@@ -273,6 +273,8 @@ static inline void __cpu_init_stage2(void)
 	kvm_call_hyp(__init_stage2_translation);
 }
 
+static inline void __cpu_copy_hyp_conf(void) {}
+
 static inline int kvm_arch_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 {
 	return 0;
arch/arm64/include/asm/kvm_asm.h +2 -0
index f5b79e9..8acd73f 100644
@@ -80,6 +80,8 @@ extern void __vgic_v3_init_lrs(void);
 
 extern u32 __kvm_get_mdcr_el2(void);
 
+extern void __kvm_populate_host_regs(void);
+
 /* Home-grown __this_cpu_{ptr,read} variants that always work at HYP */
 #define __hyp_this_cpu_ptr(sym)						\
 	({								\
arch/arm64/include/asm/kvm_emulate.h +0 -0
index 506386a..0dbe795 100644

arch/arm64/include/asm/kvm_emulate.h void kvm_inject_pabt32(struct kvm_vcpu *vcpu, unsigned long addr); · line 53

Quoted 5 lines
arch/arm64/include/asm/kvm_emulate.h +1 -1
@@ -50,25 +50,25 @@ void kvm_inject_pabt32(struct kvm_vcpu *vcpu, unsigned long addr);
 
 static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
 {
-	return !(vcpu->arch.hcr_el2 & HCR_RW);
+	return !(vcpu->arch.ctxt.hcr_el2 & HCR_RW);

Putting hcr_el2 into struct kvm_cpu_context creates a lot of splatter here, and I'm wondering whether it's really necessary. Otherwise, we could just put the per-vcpu guest HCR_EL2 value in struct kvm_vcpu_arch.

Is the *host* hcr_el2 value really different per-vcpu? That looks odd. I would have thought this is fixed across the system at KVM startup time.

Having a single global host hcr_el2 would also avoid the need for __kvm_populate_host_regs(): instead, we just decide what HCR_EL2 is to be ahead of time and set a global variable that we map into Hyp.

Or does the host HCR_EL2 need to vary at runtime for some reason I've missed?

[...]

+void __hyp_text __kvm_populate_host_regs(void)
+{
+ struct kvm_cpu_context *host_ctxt;
+
+ if (has_vhe())
+ host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
+ else
+ host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);

According to the comment by the definition of __hyp_this_cpu_ptr(), this always works at Hyp. I also see other calls with no fallback this_cpu_ptr() call like we have here.

So, can we simply always call __hyp_this_cpu_ptr() here?

(I'm not familiar with this, myself.)

Cheers
---Dave

#6 lore raw

Re: [PATCH v6 1/6] arm64/kvm: preserve host HCR_EL2 value

Hi,

Quoted 14 lines · Dave Martin
Quoted 13 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When restoring HCR_EL2 for the host, KVM uses HCR_HOST_VHE_FLAGS, which is a constant value. This works today, as the host HCR_EL2 value is always the same, but this will get in the way of supporting extensions that require HCR_EL2 bits to be set conditionally for the host.

To allow such features to work without KVM having to explicitly handle every possible host feature combination, this patch has KVM save/restore for the host HCR when switching to/from a guest HCR. The saving of the register is done once during cpu hypervisor initialization state and is just restored after switch from guest.

Quoted 6 lines
Quoted 2 lines

For fetching HCR_EL2 during kvm initialisation, a hyp call is made using kvm_call_hyp and is helpful in NHVE case.

Minor nit: NVHE misspelled. This looks a bit like it's naming an arch feature rather than a kernel implementation detail though. Maybe write "non-VHE".

yes.

Quoted 59 lines
Quoted 59 lines

For the hyp TLB maintenance code, __tlb_switch_to_host_vhe() is updated to toggle the TGE bit with a RMW sequence, as we already do in __tlb_switch_to_guest_vhe().

The value of hcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

[Added __cpu_copy_hyp_conf, hcr_el2 field in struct kvm_cpu_context]
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>
---
  arch/arm/include/asm/kvm_host.h      |  2 ++
  arch/arm64/include/asm/kvm_asm.h     |  2 ++
  arch/arm64/include/asm/kvm_emulate.h | 22 +++++++++++-----------
  arch/arm64/include/asm/kvm_host.h    | 13 ++++++++++++-
  arch/arm64/include/asm/kvm_hyp.h     |  2 +-
  arch/arm64/kvm/guest.c               |  2 +-
  arch/arm64/kvm/hyp/switch.c          | 23 +++++++++++++----------
  arch/arm64/kvm/hyp/sysreg-sr.c       | 21 ++++++++++++++++++++-
  arch/arm64/kvm/hyp/tlb.c             |  6 +++++-
  virt/kvm/arm/arm.c                   |  1 +
  10 files changed, 68 insertions(+), 26 deletions(-)
arch/arm/include/asm/kvm_host.h +2 -0
index ca56537..05706b4 100644
@@ -273,6 +273,8 @@ static inline void __cpu_init_stage2(void)
  	kvm_call_hyp(__init_stage2_translation);
  }
 
+static inline void __cpu_copy_hyp_conf(void) {}
+
  static inline int kvm_arch_vm_ioctl_check_extension(struct kvm *kvm, long ext)
  {
  	return 0;
arch/arm64/include/asm/kvm_asm.h +2 -0
index f5b79e9..8acd73f 100644
@@ -80,6 +80,8 @@ extern void __vgic_v3_init_lrs(void);
 
  extern u32 __kvm_get_mdcr_el2(void);
 
+extern void __kvm_populate_host_regs(void);
+
  /* Home-grown __this_cpu_{ptr,read} variants that always work at HYP */
  #define __hyp_this_cpu_ptr(sym)						\
  	({								\
arch/arm64/include/asm/kvm_emulate.h +0 -0
index 506386a..0dbe795 100644
@@ -50,25 +50,25 @@ void kvm_inject_pabt32(struct kvm_vcpu *vcpu, unsigned long addr);
 
Quoted 9 lines

arch/arm64/include/asm/kvm_emulate.h void kvm_inject_pabt32(struct kvm_vcpu *vcpu, unsigned long addr); · line 53

Quoted 4 lines
arch/arm64/include/asm/kvm_emulate.h +1 -1
  static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
  {
-	return !(vcpu->arch.hcr_el2 & HCR_RW);
+	return !(vcpu->arch.ctxt.hcr_el2 & HCR_RW);

Putting hcr_el2 into struct kvm_cpu_context creates a lot of splatter here, and I'm wondering whether it's really necessary. Otherwise, we could just put the per-vcpu guest HCR_EL2 value in struct kvm_vcpu_arch.

I did like that in V4 version [1] but comments were raised that this was repetition of hcr_el2 field in 2 places and may be avoided.

[1]: https://lkml.org/lkml/2019/1/4/433

Quoted 5 lines

Is the *host* hcr_el2 value really different per-vcpu? That looks odd. I would have thought this is fixed across the system at KVM startup time.

Quoted 7 lines

Having a single global host hcr_el2 would also avoid the need for __kvm_populate_host_regs(): instead, we just decide what HCR_EL2 is to be ahead of time and set a global variable that we map into Hyp.

Or does the host HCR_EL2 need to vary at runtime for some reason I've missed?

This patch basically makes host hcr_el2 not to use fixed values like HCR_HOST_NVHE_FLAGS/HCR_HOST_VHE_FLAGS during context switch and hence saves those values at boot time. This patch is just preparation to configure host hcr_el2 dynamically. However currently it is same for all cpus.

I suppose it is better to have host hcr_el2 as percpu to take care of heterogeneous systems. Currently even host mdcr_el2 is stored on percpu basis(arch/arm64/kvm/debug.c).

Quoted 12 lines

[...]

arch/arm64/kvm/hyp/sysreg-sr.c +8 -0
+void __hyp_text __kvm_populate_host_regs(void)
+{
+       struct kvm_cpu_context *host_ctxt;
+
+       if (has_vhe())
+               host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
+       else
+               host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);
Quoted 5 lines

According to the comment by the definition of __hyp_this_cpu_ptr(), this always works at Hyp. I also see other calls with no fallback this_cpu_ptr() call like we have here.

So, can we simply always call __hyp_this_cpu_ptr() here?

Yes i missed this.

Thanks,
Amit D

Quoted 6 lines

(I'm not familiar with this, myself.)

Cheers
---Dave

#7 lore raw

Re: [PATCH v6 1/6] arm64/kvm: preserve host HCR_EL2 value

In reply to Amit Daniel Kachhap

Hi Amit,

Quoted 29 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When restoring HCR_EL2 for the host, KVM uses HCR_HOST_VHE_FLAGS, which is a constant value. This works today, as the host HCR_EL2 value is always the same, but this will get in the way of supporting extensions that require HCR_EL2 bits to be set conditionally for the host.

To allow such features to work without KVM having to explicitly handle every possible host feature combination, this patch has KVM save/restore for the host HCR when switching to/from a guest HCR. The saving of the register is done once during cpu hypervisor initialization state and is just restored after switch from guest.

For fetching HCR_EL2 during kvm initialisation, a hyp call is made using kvm_call_hyp and is helpful in NHVE case.

For the hyp TLB maintenance code, __tlb_switch_to_host_vhe() is updated to toggle the TGE bit with a RMW sequence, as we already do in __tlb_switch_to_guest_vhe().

The value of hcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

arch/arm/include/asm/kvm_host.h +0 -0
index ca56537..05706b4 100644

arch/arm/include/asm/kvm_host.h static inline void __cpu_init_stage2(void) · line 277

Quoted 5 lines
arch/arm/include/asm/kvm_host.h +2 -0
@@ -273,6 +273,8 @@ static inline void __cpu_init_stage2(void)
 	kvm_call_hyp(__init_stage2_translation);
 }
 
+static inline void __cpu_copy_hyp_conf(void) {}
+

I agree Mark's suggestion of adding 'host_ctxt' in here makes it clearer what it is.

arch/arm64/include/asm/kvm_emulate.h

Quoted 4 lines
arch/arm64/include/asm/kvm_emulate.h +0 -0
index 506386a..0dbe795 100644

Hmmm, there is still a fair amount of churn due to moving the struct definition, but its easy enough to ignore as its mechanical. A preparatory patch that switched as may as possible to '*vcpu_hcr() = ' would cut the churn down some more, but I don't think its worth the extra effort.

arch/arm64/include/asm/kvm_hyp.h void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); · line 154

Quoted 10 lines
arch/arm64/include/asm/kvm_hyp.h +1 -1
index a80a7ef..6e65cad 100644
@@ -151,7 +151,7 @@ void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
 bool __fpsimd_enabled(void);
 
 void activate_traps_vhe_load(struct kvm_vcpu *vcpu);
-void deactivate_traps_vhe_put(void);
+void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu);

I've forgotten why this is needed. You don't add a user of vcpu to deactivate_traps_vhe_put() in this patch.

arch/arm64/kvm/hyp/switch.c void activate_traps_vhe_load(struct kvm_vcpu *vcpu) · line 198

Quoted 11 lines
arch/arm64/kvm/hyp/switch.c +1 -1
index b0b1478..006bd33 100644
@@ -191,7 +194,7 @@ void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
 
-void deactivate_traps_vhe_put(void)
+void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu)
 {
 	u64 mdcr_el2 = read_sysreg(mdcr_el2);
 

Why does deactivate_traps_vhe_put() need the vcpu?

arch/arm64/include/asm/kvm_host.h int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, · line 466

Quoted 13 lines
arch/arm64/include/asm/kvm_host.h +5 -0
index 7732d0b..1b2e05b 100644
@@ -458,6 +459,16 @@ int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
 
 static inline void __cpu_init_stage2(void) {}
 
+/**
+ * __cpu_copy_hyp_conf - copy the boot hyp configuration registers
+ *
+ * It is called once per-cpu during CPU hyp initialisation.
+ */

Is it just the boot cpu?

Quoted 12 lines
arch/arm64/include/asm/kvm_host.h +5 -0
+static inline void __cpu_copy_hyp_conf(void)
+{
+	kvm_call_hyp(__kvm_populate_host_regs);
+}
+
 
 
arch/arm64/kvm/hyp/sysreg-sr.c +0 -0
index 68d6f7c..68ddc0f 100644

arch/arm64/kvm/hyp/sysreg-sr.c line 24

Quoted 4 lines
arch/arm64/kvm/hyp/sysreg-sr.c +1 -0
@@ -21,6 +21,7 @@
 #include <asm/kvm_asm.h>
 #include <asm/kvm_emulate.h>
 #include <asm/kvm_hyp.h>
+#include <asm/kvm_mmu.h>

... what's kvm_mmu.h needed for?
The __hyp_this_cpu_ptr() you add comes from kvm_asm.h.

/me tries it.

Heh, hyp_symbol_addr(). kvm_asm.h should include this, but can't because the kvm_ksym_ref() dependency is the other-way round. This is just going to bite us somewhere else later!
If we want to fix it now, moving hyp_symbol_addr() to kvm_asm.h would fix it. It's generating adrp/add so the 'asm' label is fair, and it really should live with its EL1 counterpart kvm_ksym_ref().

Quoted 19 lines
arch/arm64/kvm/hyp/sysreg-sr.c +7 -1
@@ -294,7 +295,7 @@ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
 	if (!has_vhe())
 		return;
 
-	deactivate_traps_vhe_put();
+	deactivate_traps_vhe_put(vcpu);
 
 	__sysreg_save_el1_state(guest_ctxt);
 	__sysreg_save_user_state(guest_ctxt);
@@ -316,3 +317,21 @@ void __hyp_text __kvm_enable_ssbs(void)
 	"msr	sctlr_el2, %0"
 	: "=&r" (tmp) : "L" (SCTLR_ELx_DSSBS));
 }
+
+/**
+ * __kvm_populate_host_regs - Stores host register values
+ *
+ * This function acts as a function handler parameter for kvm_call_hyp and
+ * may be called from EL1 exception level to fetch the register value.

arch/arm64/kvm/hyp/sysreg-sr.c void __hyp_text __kvm_enable_ssbs(void) · line 334

Quoted 10 lines
arch/arm64/kvm/hyp/sysreg-sr.c +8 -0
+ */
+void __hyp_text __kvm_populate_host_regs(void)
+{
+	struct kvm_cpu_context *host_ctxt;
 
 
+	if (has_vhe())
+		host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
+	else
+		host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);

You can use __hyp_this_cpu_ptr() here, even on VHE.

For VHE the guts are the same and its simpler to use the same version in both cases.

__hyp_this_cpu_ptr(sym) == hyp_symbol_addr(sym) + tpidr_el2;

hyp_symbol_addr() here is just to guarantee the address is generated based on where we're executing from, not loaded from a literal pool which would give us the link-time address. (or whenever kaslr applied the relocations). This matters for non-VHE because the compiler can't know the code has an EL2 address as well as its link-time address.

This doesn't matter for VHE, as there is no additional different address.

(the other trickery is on non-VHE the tpidr_el2 value isn't actually the same as the hosts.. but on VHE it is)

virt/kvm/arm/arm.c static void cpu_hyp_reinit(void) · line 1331

Quoted 13 lines
arch/arm64/kvm/hyp/sysreg-sr.c +2 -0
+	host_ctxt->hcr_el2 = read_sysreg(hcr_el2);
+}
virt/kvm/arm/arm.c +1 -0
index 9e350fd3..8e18f7f 100644
@@ -1328,6 +1328,7 @@ static void cpu_hyp_reinit(void)
 		cpu_init_hyp_mode(NULL);
 
 	kvm_arm_init_debug();
+	__cpu_copy_hyp_conf();

Your commit message says:
| The saving of the register is done once during cpu hypervisor initialization state

But cpu_hyp_reinit() is called each time secondary CPUs come online. Its also called as part of the cpu-idle mechanism via hyp_init_cpu_pm_notifier(). cpu-idle can ask the firmware to power-off the CPU until an interrupt becomes pending for it. KVM's EL2 state disappears when this happens, these calls take care of setting it back up again. On Juno, this can happen tens of times a second, and this adds an extra call to EL2.

init_subsystems() would be the alternative place for this, but it wouldn't catch CPUs that came online after booting. I think you need something in cpu_hyp_reinit() or __cpu_copy_hyp_conf() to ensure it only happens once per CPU.

I think you can test whether the HCR_EL2 value is zero, assuming zero means uninitialised. A VHE system would always set E2H, and a non-VHE system has to set RW.

virt/kvm/arm/arm.c static void cpu_hyp_reinit(void)

Quoted 3 lines
virt/kvm/arm/arm.c +0 -0
 	if (vgic_present)
 		kvm_vgic_init_cpu_hardware();
 

Thanks,

James

Reviewed-by
#8 lore raw

Re: [PATCH v6 1/6] arm64/kvm: preserve host HCR_EL2 value

Hi Amit,

Quoted 36 lines · James Morse
Quoted 27 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>
When restoring HCR_EL2 for the host, KVM uses HCR_HOST_VHE_FLAGS, which is a constant value. This works today, as the host HCR_EL2 value is always the same, but this will get in the way of supporting extensions that require HCR_EL2 bits to be set conditionally for the host.

To allow such features to work without KVM having to explicitly handle every possible host feature combination, this patch has KVM save/restore for the host HCR when switching to/from a guest HCR. The saving of the register is done once during cpu hypervisor initialization state and is just restored after switch from guest.

For fetching HCR_EL2 during kvm initialisation, a hyp call is made using kvm_call_hyp and is helpful in NHVE case.

For the hyp TLB maintenance code, __tlb_switch_to_host_vhe() is updated to toggle the TGE bit with a RMW sequence, as we already do in __tlb_switch_to_guest_vhe().

The value of hcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

virt/kvm/arm/arm.c +0 -0
index 9e350fd3..8e18f7f 100644

virt/kvm/arm/arm.c static void cpu_hyp_reinit(void) · line 1331

Quoted 4 lines
virt/kvm/arm/arm.c +1 -0
@@ -1328,6 +1328,7 @@ static void cpu_hyp_reinit(void)
 		cpu_init_hyp_mode(NULL);
 
 	kvm_arm_init_debug();
+	__cpu_copy_hyp_conf();

Your commit message says:
| The saving of the register is done once during cpu hypervisor initialization state

Quoted 5 lines

But cpu_hyp_reinit() is called each time secondary CPUs come online. Its also called as part of the cpu-idle mechanism via hyp_init_cpu_pm_notifier(). cpu-idle can ask the firmware to power-off the CPU until an interrupt becomes pending for it. KVM's EL2 state disappears when this happens, these calls take care of setting it back up again. On Juno, this can happen tens of times a second, and this adds an extra call to EL2.

The bit I missed was the MDCR_EL2 copy is behind kvm_arm_init_debug(), so we already have an unnecessary EL2 call here, so its nothing new.

Assuming the deactivate_traps_vhe_put() vcpu isn't needed, and with Mark's comments addressed:

If we can avoid repeated calls to EL2 once we've got HCR_EL2+MDCR_EL2, even better!

Thanks,

James

#9 lore raw

Re: [PATCH v6 1/6] arm64/kvm: preserve host HCR_EL2 value

In reply to James Morse

Hi,

Quoted 35 lines · James Morse

Hi Amit,

Quoted 32 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When restoring HCR_EL2 for the host, KVM uses HCR_HOST_VHE_FLAGS, which is a constant value. This works today, as the host HCR_EL2 value is always the same, but this will get in the way of supporting extensions that require HCR_EL2 bits to be set conditionally for the host.

To allow such features to work without KVM having to explicitly handle every possible host feature combination, this patch has KVM save/restore for the host HCR when switching to/from a guest HCR. The saving of the register is done once during cpu hypervisor initialization state and is just restored after switch from guest.

For fetching HCR_EL2 during kvm initialisation, a hyp call is made using kvm_call_hyp and is helpful in NHVE case.

For the hyp TLB maintenance code, __tlb_switch_to_host_vhe() is updated to toggle the TGE bit with a RMW sequence, as we already do in __tlb_switch_to_guest_vhe().

The value of hcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

arch/arm/include/asm/kvm_host.h +0 -0
index ca56537..05706b4 100644
@@ -273,6 +273,8 @@ static inline void __cpu_init_stage2(void)
  	kvm_call_hyp(__init_stage2_translation);
  }
 
Quoted 4 lines

arch/arm/include/asm/kvm_host.h static inline void __cpu_init_stage2(void) · line 277

Quoted 2 lines
arch/arm/include/asm/kvm_host.h +2 -0
+static inline void __cpu_copy_hyp_conf(void) {}
+

I agree Mark's suggestion of adding 'host_ctxt' in here makes it clearer what it is.

ok.

Quoted 34 lines

arch/arm64/include/asm/kvm_emulate.h

Quoted 6 lines
arch/arm64/include/asm/kvm_emulate.h +0 -0
index 506386a..0dbe795 100644

Hmmm, there is still a fair amount of churn due to moving the struct definition, but its easy enough to ignore as its mechanical. A preparatory patch that switched as may as possible to '*vcpu_hcr() = ' would cut the churn down some more, but I don't think its worth the extra effort.

arch/arm64/include/asm/kvm_hyp.h void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); · line 154

Quoted 10 lines
arch/arm64/include/asm/kvm_hyp.h +1 -1
index a80a7ef..6e65cad 100644
@@ -151,7 +151,7 @@ void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
  bool __fpsimd_enabled(void);
 
  void activate_traps_vhe_load(struct kvm_vcpu *vcpu);
-void deactivate_traps_vhe_put(void);
+void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu);

I've forgotten why this is needed. You don't add a user of vcpu to deactivate_traps_vhe_put() in this patch.

Quoted 5 lines
arch/arm64/kvm/hyp/switch.c +0 -0
index b0b1478..006bd33 100644
@@ -191,7 +194,7 @@ void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
Quoted 7 lines

arch/arm64/kvm/hyp/switch.c void activate_traps_vhe_load(struct kvm_vcpu *vcpu) · line 200

Quoted 5 lines
arch/arm64/kvm/hyp/switch.c +1 -1
-void deactivate_traps_vhe_put(void)
+void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu)
  {
  	u64 mdcr_el2 = read_sysreg(mdcr_el2);
 

Why does deactivate_traps_vhe_put() need the vcpu?

vcpu is needed for the next patch which saves/restore mdcr_el2. I will add this in that patch.

Quoted 39 lines
Quoted 7 lines
arch/arm64/include/asm/kvm_host.h +0 -0
index 7732d0b..1b2e05b 100644

arch/arm64/include/asm/kvm_host.h int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, · line 466

Quoted 8 lines
arch/arm64/include/asm/kvm_host.h +5 -0
@@ -458,6 +459,16 @@ int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
 
  static inline void __cpu_init_stage2(void) {}
 
+/**
+ * __cpu_copy_hyp_conf - copy the boot hyp configuration registers
+ *
+ * It is called once per-cpu during CPU hyp initialisation.
+ */

Is it just the boot cpu?

Quoted 12 lines
arch/arm64/include/asm/kvm_host.h +5 -0
+static inline void __cpu_copy_hyp_conf(void)
+{
+	kvm_call_hyp(__kvm_populate_host_regs);
+}
+
 
 
arch/arm64/kvm/hyp/sysreg-sr.c +0 -0
index 68d6f7c..68ddc0f 100644

arch/arm64/kvm/hyp/sysreg-sr.c line 24

Quoted 4 lines
arch/arm64/kvm/hyp/sysreg-sr.c +1 -0
@@ -21,6 +21,7 @@
  #include <asm/kvm_asm.h>
  #include <asm/kvm_emulate.h>
  #include <asm/kvm_hyp.h>
+#include <asm/kvm_mmu.h>

... what's kvm_mmu.h needed for?
The __hyp_this_cpu_ptr() you add comes from kvm_asm.h.

Quoted 9 lines

/me tries it.

Heh, hyp_symbol_addr(). kvm_asm.h should include this, but can't because the kvm_ksym_ref() dependency is the other-way round. This is just going to bite us somewhere else later!
If we want to fix it now, moving hyp_symbol_addr() to kvm_asm.h would fix it. It's generating adrp/add so the 'asm' label is fair, and it really should live with its EL1 counterpart kvm_ksym_ref().

Yes moving hyp_symbol_addr() fixes the dependency error.

Quoted 43 lines
Quoted 20 lines
arch/arm64/kvm/hyp/sysreg-sr.c +7 -1
@@ -294,7 +295,7 @@ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
  	if (!has_vhe())
  		return;
 
-	deactivate_traps_vhe_put();
+	deactivate_traps_vhe_put(vcpu);
 
  	__sysreg_save_el1_state(guest_ctxt);
  	__sysreg_save_user_state(guest_ctxt);
@@ -316,3 +317,21 @@ void __hyp_text __kvm_enable_ssbs(void)
  	"msr	sctlr_el2, %0"
  	: "=&r" (tmp) : "L" (SCTLR_ELx_DSSBS));
  }
+
+/**
+ * __kvm_populate_host_regs - Stores host register values
+ *
+ * This function acts as a function handler parameter for kvm_call_hyp and
+ * may be called from EL1 exception level to fetch the register value.

arch/arm64/kvm/hyp/sysreg-sr.c void __hyp_text __kvm_enable_ssbs(void) · line 334

Quoted 10 lines
arch/arm64/kvm/hyp/sysreg-sr.c +8 -0
+ */
+void __hyp_text __kvm_populate_host_regs(void)
+{
+	struct kvm_cpu_context *host_ctxt;
 
 
+	if (has_vhe())
+		host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
+	else
+		host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);

You can use __hyp_this_cpu_ptr() here, even on VHE.

For VHE the guts are the same and its simpler to use the same version in both cases.

__hyp_this_cpu_ptr(sym) == hyp_symbol_addr(sym) + tpidr_el2;

hyp_symbol_addr() here is just to guarantee the address is generated based on where we're executing from, not loaded from a literal pool which would give us the link-time address. (or whenever kaslr applied the relocations). This matters for non-VHE because the compiler can't know the code has an EL2 address as well as its link-time address.

Quoted 6 lines

This doesn't matter for VHE, as there is no additional different address.

(the other trickery is on non-VHE the tpidr_el2 value isn't actually the same as the hosts.. but on VHE it is)

Thanks for the details.

Quoted 23 lines

virt/kvm/arm/arm.c static void cpu_hyp_reinit(void) · line 1331

Quoted 13 lines
arch/arm64/kvm/hyp/sysreg-sr.c +2 -0
+	host_ctxt->hcr_el2 = read_sysreg(hcr_el2);
+}
virt/kvm/arm/arm.c +1 -0
index 9e350fd3..8e18f7f 100644
@@ -1328,6 +1328,7 @@ static void cpu_hyp_reinit(void)
  		cpu_init_hyp_mode(NULL);
 
  	kvm_arm_init_debug();
+	__cpu_copy_hyp_conf();

Your commit message says:
| The saving of the register is done once during cpu hypervisor initialization state

But cpu_hyp_reinit() is called each time secondary CPUs come online. Its also called as part of the cpu-idle mechanism via hyp_init_cpu_pm_notifier(). cpu-idle can ask the firmware to power-off the CPU until an interrupt becomes pending for it. KVM's EL2 state disappears when this happens, these calls take care of setting it back up again. On Juno, this can happen tens of times a second, and this adds an extra call to EL2.

Quoted 3 lines

init_subsystems() would be the alternative place for this, but it wouldn't catch CPUs that came online after booting. I think you need something in cpu_hyp_reinit() or __cpu_copy_hyp_conf() to ensure it only happens once per CPU.

ok i will check on it.

Quoted 3 lines

I think you can test whether the HCR_EL2 value is zero, assuming zero means uninitialised. A VHE system would always set E2H, and a non-VHE system has to set RW.

It is not zero and is set to initial values.

Thanks,
Amit D

Quoted 11 lines

virt/kvm/arm/arm.c static void cpu_hyp_reinit(void)

Quoted 5 lines
virt/kvm/arm/arm.c +0 -0
  	if (vgic_present)
  		kvm_vgic_init_cpu_hardware();
 

Thanks,

James

#10 lore raw

PATCH v6 2/6 arm64/kvm: preserve host MDCR_EL2 value

In reply to Amit Daniel Kachhap

Save host MDCR_EL2 value during kvm HYP initialisation and restore after every switch from host to guest. There should not be any change in functionality due to this.

The value of mdcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

Cc Marc Zyngier <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Christoffer Dall <[email protected]>
---
 arch/arm/include/asm/kvm_host.h   |  1 -
 arch/arm64/include/asm/kvm_host.h |  6 ++----
 arch/arm64/kvm/debug.c            | 28 ++++++----------------------
 arch/arm64/kvm/hyp/switch.c       | 17 ++++-------------
 arch/arm64/kvm/hyp/sysreg-sr.c    |  6 ++++++
 virt/kvm/arm/arm.c                |  1 -
 6 files changed, 18 insertions(+), 41 deletions(-)
arch/arm/include/asm/kvm_host.h +0 -1
index 05706b4..704667e 100644
@@ -294,7 +294,6 @@ static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
 static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
 
-static inline void kvm_arm_init_debug(void) {}
 static inline void kvm_arm_setup_debug(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arm_clear_debug(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arm_reset_debug_ptr(struct kvm_vcpu *vcpu) {}
arch/arm64/include/asm/kvm_host.h +2 -4
index 1b2e05b..2f1bb86 100644
@@ -205,6 +205,8 @@ struct kvm_cpu_context {
 
 	/* HYP host/guest configuration */
 	u64 hcr_el2;
+	u32 mdcr_el2;
+
 	struct kvm_vcpu *__hyp_running_vcpu;
 };
 
@@ -213,9 +215,6 @@ typedef struct kvm_cpu_context kvm_cpu_context_t;
 struct kvm_vcpu_arch {
 	struct kvm_cpu_context ctxt;
 
-	/* HYP configuration */
-	u32 mdcr_el2;
-
 	/* Exception Information */
 	struct kvm_vcpu_fault_info fault;
 
@@ -446,7 +445,6 @@ static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
 static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
 
-void kvm_arm_init_debug(void);
 void kvm_arm_setup_debug(struct kvm_vcpu *vcpu);
 void kvm_arm_clear_debug(struct kvm_vcpu *vcpu);
 void kvm_arm_reset_debug_ptr(struct kvm_vcpu *vcpu);
arch/arm64/kvm/debug.c +6 -22
index f39801e..99dc0a4 100644
@@ -32,8 +32,6 @@
 				DBG_MDSCR_KDE | \
 				DBG_MDSCR_MDE)
 
-static DEFINE_PER_CPU(u32, mdcr_el2);
-
 /**
  * save/restore_guest_debug_regs
  *
@@ -65,21 +63,6 @@ static void restore_guest_debug_regs(struct kvm_vcpu *vcpu)
 }
 
 /**
- * kvm_arm_init_debug - grab what we need for debug
- *
- * Currently the sole task of this function is to retrieve the initial
- * value of mdcr_el2 so we can preserve MDCR_EL2.HPMN which has
- * presumably been set-up by some knowledgeable bootcode.
- *
- * It is called once per-cpu during CPU hyp initialisation.
- */
-
-void kvm_arm_init_debug(void)
-{
-	__this_cpu_write(mdcr_el2, kvm_call_hyp(__kvm_get_mdcr_el2));
-}
-
-/**
  * kvm_arm_reset_debug_ptr - reset the debug ptr to point to the vcpu state
  */
 
@@ -111,6 +94,7 @@ void kvm_arm_reset_debug_ptr(struct kvm_vcpu *vcpu)
 
 void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
 {
+	kvm_cpu_context_t *host_cxt = this_cpu_ptr(&kvm_host_cpu_state);
 	bool trap_debug = !(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY);
 	unsigned long mdscr;
 
@@ -120,8 +104,8 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
 	 * This also clears MDCR_EL2_E2PB_MASK to disable guest access
 	 * to the profiling buffer.
 	 */
-	vcpu->arch.mdcr_el2 = __this_cpu_read(mdcr_el2) & MDCR_EL2_HPMN_MASK;
-	vcpu->arch.mdcr_el2 |= (MDCR_EL2_TPM |
+	vcpu->arch.ctxt.mdcr_el2 = host_cxt->mdcr_el2 & MDCR_EL2_HPMN_MASK;
+	vcpu->arch.ctxt.mdcr_el2 |= (MDCR_EL2_TPM |
 				MDCR_EL2_TPMS |
 				MDCR_EL2_TPMCR |
 				MDCR_EL2_TDRA |
@@ -130,7 +114,7 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
 	/* Is Guest debugging in effect? */
 	if (vcpu->guest_debug) {
 		/* Route all software debug exceptions to EL2 */
-		vcpu->arch.mdcr_el2 |= MDCR_EL2_TDE;
+		vcpu->arch.ctxt.mdcr_el2 |= MDCR_EL2_TDE;
 
 		/* Save guest debug state */
 		save_guest_debug_regs(vcpu);
@@ -202,13 +186,13 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
 
 	/* Trap debug register access */
 	if (trap_debug)
-		vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
+		vcpu->arch.ctxt.mdcr_el2 |= MDCR_EL2_TDA;
 
 	/* If KDE or MDE are set, perform a full save/restore cycle. */
 	if (vcpu_read_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
 		vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
 
-	trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.mdcr_el2);
+	trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.ctxt.mdcr_el2);
 	trace_kvm_arm_set_dreg32("MDSCR_EL1", vcpu_read_sys_reg(vcpu, MDSCR_EL1));
 }
 
arch/arm64/kvm/hyp/switch.c +4 -13
index 006bd33..03b36f1 100644
@@ -82,7 +82,7 @@ static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
 	 */
 	write_sysreg(0, pmselr_el0);
 	write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
-	write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+	write_sysreg(vcpu->arch.ctxt.mdcr_el2, mdcr_el2);
 }
 
 static void __hyp_text __deactivate_traps_common(void)
@@ -157,14 +157,9 @@ static void deactivate_traps_vhe(struct kvm_cpu_context *host_ctxt)
 
 static void __hyp_text __deactivate_traps_nvhe(struct kvm_cpu_context *host_ctxt)
 {
-	u64 mdcr_el2 = read_sysreg(mdcr_el2);
-
 	__deactivate_traps_common();
 
-	mdcr_el2 &= MDCR_EL2_HPMN_MASK;
-	mdcr_el2 |= MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT;
-
-	write_sysreg(mdcr_el2, mdcr_el2);
+	write_sysreg(host_ctxt->mdcr_el2, mdcr_el2);
 	write_sysreg(host_ctxt->hcr_el2, hcr_el2);
 	write_sysreg(CPTR_EL2_DEFAULT, cptr_el2);
 }
@@ -196,13 +191,9 @@ void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
 
 void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu)
 {
-	u64 mdcr_el2 = read_sysreg(mdcr_el2);
-
-	mdcr_el2 &= MDCR_EL2_HPMN_MASK |
-		    MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT |
-		    MDCR_EL2_TPMS;
+	struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
 
-	write_sysreg(mdcr_el2, mdcr_el2);
+	write_sysreg(host_ctxt->mdcr_el2, mdcr_el2);
 
 	__deactivate_traps_common();
 }
arch/arm64/kvm/hyp/sysreg-sr.c +6 -0
index 68ddc0f..42ec50f 100644
@@ -334,4 +334,10 @@ void __hyp_text __kvm_populate_host_regs(void)
 		host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);
 
 	host_ctxt->hcr_el2 = read_sysreg(hcr_el2);
+	/*
+	 * Retrieve the initial value of mdcr_el2 so we can preserve
+	 * MDCR_EL2.HPMN which has presumably been set-up by some
+	 * knowledgeable bootcode.
+	 */
+	host_ctxt->mdcr_el2 = read_sysreg(mdcr_el2);
 }
virt/kvm/arm/arm.c +0 -1
index 8e18f7f..2032a66 100644
@@ -1327,7 +1327,6 @@ static void cpu_hyp_reinit(void)
 	else
 		cpu_init_hyp_mode(NULL);
 
-	kvm_arm_init_debug();
 	__cpu_copy_hyp_conf();
 
 	if (vgic_present)
Signature · 1 line
2.7.4
reviewer Acked-by on v6 2/6
#11 lore raw

Re: [PATCH v6 2/6] arm64/kvm: preserve host MDCR_EL2 value

Quoted 12 lines · Amit Daniel Kachhap

Save host MDCR_EL2 value during kvm HYP initialisation and restore after every switch from host to guest. There should not be any change in functionality due to this.

The value of mdcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

Cc Marc Zyngier <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Christoffer Dall <[email protected]>
Quoted 8 lines
---
 arch/arm/include/asm/kvm_host.h   |  1 -
 arch/arm64/include/asm/kvm_host.h |  6 ++----
 arch/arm64/kvm/debug.c            | 28 ++++++----------------------
 arch/arm64/kvm/hyp/switch.c       | 17 ++++-------------
 arch/arm64/kvm/hyp/sysreg-sr.c    |  6 ++++++
 virt/kvm/arm/arm.c                |  1 -
 6 files changed, 18 insertions(+), 41 deletions(-)

This looks like a nice cleanup! FWIW:

Thanks,
Mark.

Quoted 198 lines
arch/arm/include/asm/kvm_host.h +0 -1
index 05706b4..704667e 100644
@@ -294,7 +294,6 @@ static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
 static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
 
-static inline void kvm_arm_init_debug(void) {}
 static inline void kvm_arm_setup_debug(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arm_clear_debug(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arm_reset_debug_ptr(struct kvm_vcpu *vcpu) {}
arch/arm64/include/asm/kvm_host.h +2 -4
index 1b2e05b..2f1bb86 100644
@@ -205,6 +205,8 @@ struct kvm_cpu_context {
 
 	/* HYP host/guest configuration */
 	u64 hcr_el2;
+	u32 mdcr_el2;
+
 	struct kvm_vcpu *__hyp_running_vcpu;
 };
 
@@ -213,9 +215,6 @@ typedef struct kvm_cpu_context kvm_cpu_context_t;
 struct kvm_vcpu_arch {
 	struct kvm_cpu_context ctxt;
 
-	/* HYP configuration */
-	u32 mdcr_el2;
-
 	/* Exception Information */
 	struct kvm_vcpu_fault_info fault;
 
@@ -446,7 +445,6 @@ static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
 static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
 
-void kvm_arm_init_debug(void);
 void kvm_arm_setup_debug(struct kvm_vcpu *vcpu);
 void kvm_arm_clear_debug(struct kvm_vcpu *vcpu);
 void kvm_arm_reset_debug_ptr(struct kvm_vcpu *vcpu);
arch/arm64/kvm/debug.c +6 -22
index f39801e..99dc0a4 100644
@@ -32,8 +32,6 @@
 				DBG_MDSCR_KDE | \
 				DBG_MDSCR_MDE)
 
-static DEFINE_PER_CPU(u32, mdcr_el2);
-
 /**
  * save/restore_guest_debug_regs
  *
@@ -65,21 +63,6 @@ static void restore_guest_debug_regs(struct kvm_vcpu *vcpu)
 }
 
 /**
- * kvm_arm_init_debug - grab what we need for debug
- *
- * Currently the sole task of this function is to retrieve the initial
- * value of mdcr_el2 so we can preserve MDCR_EL2.HPMN which has
- * presumably been set-up by some knowledgeable bootcode.
- *
- * It is called once per-cpu during CPU hyp initialisation.
- */
-
-void kvm_arm_init_debug(void)
-{
-	__this_cpu_write(mdcr_el2, kvm_call_hyp(__kvm_get_mdcr_el2));
-}
-
-/**
  * kvm_arm_reset_debug_ptr - reset the debug ptr to point to the vcpu state
  */
 
@@ -111,6 +94,7 @@ void kvm_arm_reset_debug_ptr(struct kvm_vcpu *vcpu)
 
 void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
 {
+	kvm_cpu_context_t *host_cxt = this_cpu_ptr(&kvm_host_cpu_state);
 	bool trap_debug = !(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY);
 	unsigned long mdscr;
 
@@ -120,8 +104,8 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
 	 * This also clears MDCR_EL2_E2PB_MASK to disable guest access
 	 * to the profiling buffer.
 	 */
-	vcpu->arch.mdcr_el2 = __this_cpu_read(mdcr_el2) & MDCR_EL2_HPMN_MASK;
-	vcpu->arch.mdcr_el2 |= (MDCR_EL2_TPM |
+	vcpu->arch.ctxt.mdcr_el2 = host_cxt->mdcr_el2 & MDCR_EL2_HPMN_MASK;
+	vcpu->arch.ctxt.mdcr_el2 |= (MDCR_EL2_TPM |
 				MDCR_EL2_TPMS |
 				MDCR_EL2_TPMCR |
 				MDCR_EL2_TDRA |
@@ -130,7 +114,7 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
 	/* Is Guest debugging in effect? */
 	if (vcpu->guest_debug) {
 		/* Route all software debug exceptions to EL2 */
-		vcpu->arch.mdcr_el2 |= MDCR_EL2_TDE;
+		vcpu->arch.ctxt.mdcr_el2 |= MDCR_EL2_TDE;
 
 		/* Save guest debug state */
 		save_guest_debug_regs(vcpu);
@@ -202,13 +186,13 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
 
 	/* Trap debug register access */
 	if (trap_debug)
-		vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
+		vcpu->arch.ctxt.mdcr_el2 |= MDCR_EL2_TDA;
 
 	/* If KDE or MDE are set, perform a full save/restore cycle. */
 	if (vcpu_read_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
 		vcpu->arch.flags |= KVM_ARM64_DEBUG_DIRTY;
 
-	trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.mdcr_el2);
+	trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.ctxt.mdcr_el2);
 	trace_kvm_arm_set_dreg32("MDSCR_EL1", vcpu_read_sys_reg(vcpu, MDSCR_EL1));
 }
 
arch/arm64/kvm/hyp/switch.c +4 -13
index 006bd33..03b36f1 100644
@@ -82,7 +82,7 @@ static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
 	 */
 	write_sysreg(0, pmselr_el0);
 	write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
-	write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+	write_sysreg(vcpu->arch.ctxt.mdcr_el2, mdcr_el2);
 }
 
 static void __hyp_text __deactivate_traps_common(void)
@@ -157,14 +157,9 @@ static void deactivate_traps_vhe(struct kvm_cpu_context *host_ctxt)
 
 static void __hyp_text __deactivate_traps_nvhe(struct kvm_cpu_context *host_ctxt)
 {
-	u64 mdcr_el2 = read_sysreg(mdcr_el2);
-
 	__deactivate_traps_common();
 
-	mdcr_el2 &= MDCR_EL2_HPMN_MASK;
-	mdcr_el2 |= MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT;
-
-	write_sysreg(mdcr_el2, mdcr_el2);
+	write_sysreg(host_ctxt->mdcr_el2, mdcr_el2);
 	write_sysreg(host_ctxt->hcr_el2, hcr_el2);
 	write_sysreg(CPTR_EL2_DEFAULT, cptr_el2);
 }
@@ -196,13 +191,9 @@ void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
 
 void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu)
 {
-	u64 mdcr_el2 = read_sysreg(mdcr_el2);
-
-	mdcr_el2 &= MDCR_EL2_HPMN_MASK |
-		    MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT |
-		    MDCR_EL2_TPMS;
+	struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
 
-	write_sysreg(mdcr_el2, mdcr_el2);
+	write_sysreg(host_ctxt->mdcr_el2, mdcr_el2);
 
 	__deactivate_traps_common();
 }
arch/arm64/kvm/hyp/sysreg-sr.c +6 -0
index 68ddc0f..42ec50f 100644
@@ -334,4 +334,10 @@ void __hyp_text __kvm_populate_host_regs(void)
 		host_ctxt = __hyp_this_cpu_ptr(kvm_host_cpu_state);
 
 	host_ctxt->hcr_el2 = read_sysreg(hcr_el2);
+	/*
+	 * Retrieve the initial value of mdcr_el2 so we can preserve
+	 * MDCR_EL2.HPMN which has presumably been set-up by some
+	 * knowledgeable bootcode.
+	 */
+	host_ctxt->mdcr_el2 = read_sysreg(mdcr_el2);
 }
virt/kvm/arm/arm.c +0 -1
index 8e18f7f..2032a66 100644
@@ -1327,7 +1327,6 @@ static void cpu_hyp_reinit(void)
 	else
 		cpu_init_hyp_mode(NULL);
 
-	kvm_arm_init_debug();
 	__cpu_copy_hyp_conf();
 
 	if (vgic_present)
Signature · 1 line
2.7.4
#12 lore raw

Re: [PATCH v6 2/6] arm64/kvm: preserve host MDCR_EL2 value

In reply to Amit Daniel Kachhap
Quoted 6 lines · Amit Daniel Kachhap

Save host MDCR_EL2 value during kvm HYP initialisation and restore after every switch from host to guest. There should not be any change in functionality due to this.

The value of mdcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

Is MDCR_EL2 somehow relevant to pointer auth?

It's not entirely clear why this patch is here.

If this is a cleanup to align the handling of this register with how HCR_EL2 is handled, it would be good to explain that in the commit message.

Quoted 31 lines
Cc Marc Zyngier <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Christoffer Dall <[email protected]>
---
 arch/arm/include/asm/kvm_host.h   |  1 -
 arch/arm64/include/asm/kvm_host.h |  6 ++----
 arch/arm64/kvm/debug.c            | 28 ++++++----------------------
 arch/arm64/kvm/hyp/switch.c       | 17 ++++-------------
 arch/arm64/kvm/hyp/sysreg-sr.c    |  6 ++++++
 virt/kvm/arm/arm.c                |  1 -
 6 files changed, 18 insertions(+), 41 deletions(-)
arch/arm/include/asm/kvm_host.h +0 -1
index 05706b4..704667e 100644
@@ -294,7 +294,6 @@ static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
 static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
 
-static inline void kvm_arm_init_debug(void) {}
 static inline void kvm_arm_setup_debug(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arm_clear_debug(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arm_reset_debug_ptr(struct kvm_vcpu *vcpu) {}
arch/arm64/include/asm/kvm_host.h +0 -0
index 1b2e05b..2f1bb86 100644

arch/arm64/include/asm/kvm_host.h struct kvm_cpu_context { · line 209

Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +2 -0
@@ -205,6 +205,8 @@ struct kvm_cpu_context {
 
 	/* HYP host/guest configuration */
 	u64 hcr_el2;
+	u32 mdcr_el2;
+

ARMv8-A says MDCR_EL2 is a 64-bit register.

Bits [63:20] are currently RES0, so this is probably not a big deal. But it would be better to make this 64-bit to prevent future accidents. It may be better to make that change in a separate patch.

This is probably non-urgent, since this is clearly not causing problems for anyone today.

[...]

Cheers
---Dave

#13 lore raw

Re: [PATCH v6 2/6] arm64/kvm: preserve host MDCR_EL2 value

Hi,

Quoted 8 lines · Dave Martin
Quoted 6 lines · Amit Daniel Kachhap

Save host MDCR_EL2 value during kvm HYP initialisation and restore after every switch from host to guest. There should not be any change in functionality due to this.

The value of mdcr_el2 is now stored in struct kvm_cpu_context as both host and guest can now use this field in a common way.

Quoted 7 lines

Is MDCR_EL2 somehow relevant to pointer auth?

It's not entirely clear why this patch is here.

If this is a cleanup to align the handling of this register with how HCR_EL2 is handled, it would be good to explain that in the commit message.

Agreed I will more information in commit message.

Quoted 40 lines
Quoted 32 lines
Cc Marc Zyngier <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Christoffer Dall <[email protected]>
---
  arch/arm/include/asm/kvm_host.h   |  1 -
  arch/arm64/include/asm/kvm_host.h |  6 ++----
  arch/arm64/kvm/debug.c            | 28 ++++++----------------------
  arch/arm64/kvm/hyp/switch.c       | 17 ++++-------------
  arch/arm64/kvm/hyp/sysreg-sr.c    |  6 ++++++
  virt/kvm/arm/arm.c                |  1 -
  6 files changed, 18 insertions(+), 41 deletions(-)
arch/arm/include/asm/kvm_host.h +0 -1
index 05706b4..704667e 100644
@@ -294,7 +294,6 @@ static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}
  static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
  static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
 
-static inline void kvm_arm_init_debug(void) {}
  static inline void kvm_arm_setup_debug(struct kvm_vcpu *vcpu) {}
  static inline void kvm_arm_clear_debug(struct kvm_vcpu *vcpu) {}
  static inline void kvm_arm_reset_debug_ptr(struct kvm_vcpu *vcpu) {}
arch/arm64/include/asm/kvm_host.h +0 -0
index 1b2e05b..2f1bb86 100644

arch/arm64/include/asm/kvm_host.h struct kvm_cpu_context { · line 209

Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +2 -0
@@ -205,6 +205,8 @@ struct kvm_cpu_context {
 
  	/* HYP host/guest configuration */
  	u64 hcr_el2;
+	u32 mdcr_el2;
+

ARMv8-A says MDCR_EL2 is a 64-bit register.

Quoted 3 lines

Bits [63:20] are currently RES0, so this is probably not a big deal. But it would be better to make this 64-bit to prevent future accidents. It may be better to make that change in a separate patch.

Yes this a potential issue. I will fix it in a separate patch.

Thanks,
Amit D

Quoted 9 lines

This is probably non-urgent, since this is clearly not causing problems for anyone today.

[...]

Cheers
---Dave

#14 lore raw

PATCH v6 3/6 arm64/kvm: context-switch ptrauth registers

In reply to Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When pointer authentication is supported, a guest may wish to use it. This patch adds the necessary KVM infrastructure for this to work, with a semi-lazy context switch of the pointer auth state.

Pointer authentication feature is only enabled when VHE is built in the kernel and present into CPU implementation so only VHE code paths are modified.

When we schedule a vcpu, we disable guest usage of pointer authentication instructions and accesses to the keys. While these are disabled, we avoid context-switching the keys. When we trap the guest trying to use pointer authentication functionality, we change to eagerly context-switching the keys, and enable the feature. The next time the vcpu is scheduled out/in, we start again. However the host key registers are saved in vcpu load stage as they remain constant for each vcpu schedule.

Pointer authentication consists of address authentication and generic authentication, and CPUs in a system might have varied support for either. Where support for either feature is not uniform, it is hidden from guests via ID register emulation, as a result of the cpufeature framework in the host.

Unfortunately, address authentication and generic authentication cannot be trapped separately, as the architecture provides a single EL2 trap covering both. If we wish to expose one without the other, we cannot prevent a (badly-written) guest from intermittently using a feature which is not uniformly supported (when scheduled on a physical CPU which supports the relevant feature). Hence, this patch expects both type of authentication to be present in a cpu.

[Only VHE, key switch from from assembly, kvm_supports_ptrauth checks, save host key in vcpu_load]

---
 arch/arm/include/asm/kvm_host.h   |   1 +
 arch/arm64/include/asm/kvm_host.h |  23 +++++++++
 arch/arm64/include/asm/kvm_hyp.h  |   7 +++
 arch/arm64/kernel/traps.c         |   1 +
 arch/arm64/kvm/handle_exit.c      |  21 +++++---
 arch/arm64/kvm/hyp/Makefile       |   1 +
 arch/arm64/kvm/hyp/entry.S        |  17 +++++++
 arch/arm64/kvm/hyp/ptrauth-sr.c   | 101 ++++++++++++++++++++++++++++++++++++++
 arch/arm64/kvm/sys_regs.c         |  37 +++++++++++++-
 virt/kvm/arm/arm.c                |   2 +
 10 files changed, 201 insertions(+), 10 deletions(-)
 create mode 100644 arch/arm64/kvm/hyp/ptrauth-sr.c
arch/arm/include/asm/kvm_host.h +1 -0
index 704667e..b200c14 100644
@@ -345,6 +345,7 @@ static inline int kvm_arm_have_ssbd(void)
 
 static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
 static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
+static inline void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu) {}
 
 #define __KVM_HAVE_ARCH_VM_ALLOC
 struct kvm *kvm_arch_alloc_vm(void);
arch/arm64/include/asm/kvm_host.h +23 -0
index 2f1bb86..1bacf78 100644
@@ -146,6 +146,18 @@ enum vcpu_sysreg {
 	PMSWINC_EL0,	/* Software Increment Register */
 	PMUSERENR_EL0,	/* User Enable Register */
 
+	/* Pointer Authentication Registers */
+	APIAKEYLO_EL1,
+	APIAKEYHI_EL1,
+	APIBKEYLO_EL1,
+	APIBKEYHI_EL1,
+	APDAKEYLO_EL1,
+	APDAKEYHI_EL1,
+	APDBKEYLO_EL1,
+	APDBKEYHI_EL1,
+	APGAKEYLO_EL1,
+	APGAKEYHI_EL1,
+
 	/* 32bit specific registers. Keep them at the end of the range */
 	DACR32_EL2,	/* Domain Access Control Register */
 	IFSR32_EL2,	/* Instruction Fault Status Register */
@@ -439,6 +451,17 @@ static inline bool kvm_arch_requires_vhe(void)
 	return false;
 }
 
+static inline bool kvm_supports_ptrauth(void)
+{
+	return has_vhe() && system_supports_address_auth() &&
+				system_supports_generic_auth();
+}
+
+void kvm_arm_vcpu_ptrauth_enable(struct kvm_vcpu *vcpu);
+void kvm_arm_vcpu_ptrauth_disable(struct kvm_vcpu *vcpu);
+void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu);
+void kvm_arm_vcpu_ptrauth_trap(struct kvm_vcpu *vcpu);
+
 static inline void kvm_arch_hardware_unsetup(void) {}
 static inline void kvm_arch_sync_events(struct kvm *kvm) {}
 static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}
arch/arm64/include/asm/kvm_hyp.h +7 -0
index 6e65cad..09e061a 100644
@@ -153,6 +153,13 @@ bool __fpsimd_enabled(void);
 void activate_traps_vhe_load(struct kvm_vcpu *vcpu);
 void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu);
 
+void __ptrauth_switch_to_guest(struct kvm_vcpu *vcpu,
+			       struct kvm_cpu_context *host_ctxt,
+			       struct kvm_cpu_context *guest_ctxt);
+void __ptrauth_switch_to_host(struct kvm_vcpu *vcpu,
+			      struct kvm_cpu_context *guest_ctxt,
+			      struct kvm_cpu_context *host_ctxt);
+
 u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
 void __noreturn __hyp_do_panic(unsigned long, ...);
 
arch/arm64/kernel/traps.c +1 -0
index 4e2fb87..5cac605 100644
@@ -749,6 +749,7 @@ static const char *esr_class_str[] = {
 	[ESR_ELx_EC_CP14_LS]		= "CP14 LDC/STC",
 	[ESR_ELx_EC_FP_ASIMD]		= "ASIMD",
 	[ESR_ELx_EC_CP10_ID]		= "CP10 MRC/VMRS",
+	[ESR_ELx_EC_PAC]		= "Pointer authentication trap",
 	[ESR_ELx_EC_CP14_64]		= "CP14 MCRR/MRRC",
 	[ESR_ELx_EC_ILL]		= "PSTATE.IL",
 	[ESR_ELx_EC_SVC32]		= "SVC (AArch32)",
arch/arm64/kvm/handle_exit.c +13 -8
index 0b79834..7622ab3 100644
@@ -174,19 +174,24 @@ static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
 }
 
 /*
+ * Handle the guest trying to use a ptrauth instruction, or trying to access a
+ * ptrauth register.
+ */
+void kvm_arm_vcpu_ptrauth_trap(struct kvm_vcpu *vcpu)
+{
+	if (kvm_supports_ptrauth())
+		kvm_arm_vcpu_ptrauth_enable(vcpu);
+	else
+		kvm_inject_undefined(vcpu);
+}
+
+/*
  * Guest usage of a ptrauth instruction (which the guest EL1 did not turn into
  * a NOP).
  */
 static int kvm_handle_ptrauth(struct kvm_vcpu *vcpu, struct kvm_run *run)
 {
-	/*
-	 * We don't currently support ptrauth in a guest, and we mask the ID
-	 * registers to prevent well-behaved guests from trying to make use of
-	 * it.
-	 *
-	 * Inject an UNDEF, as if the feature really isn't present.
-	 */
-	kvm_inject_undefined(vcpu);
+	kvm_arm_vcpu_ptrauth_trap(vcpu);
 	return 1;
 }
 
arch/arm64/kvm/hyp/Makefile +1 -0
index 82d1904..17cec99 100644
@@ -19,6 +19,7 @@ obj-$(CONFIG_KVM_ARM_HOST) += switch.o
 obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
 obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
 obj-$(CONFIG_KVM_ARM_HOST) += hyp-entry.o
+obj-$(CONFIG_KVM_ARM_HOST) += ptrauth-sr.o
 
 # KVM code is run at a different exception code with a different map, so
 # compiler instrumentation that inserts callbacks or checks into the code may
arch/arm64/kvm/hyp/entry.S +17 -0
index 675fdc1..b78cc15 100644
@@ -64,6 +64,12 @@ ENTRY(__guest_enter)
 
 	add	x18, x0, #VCPU_CONTEXT
 
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_guest(vcpu, host, guest).
+	mov	x2, x18
+	bl	__ptrauth_switch_to_guest
+#endif
+
 	// Restore guest regs x0-x17
 	ldp	x0, x1,   [x18, #CPU_XREG_OFFSET(0)]
 	ldp	x2, x3,   [x18, #CPU_XREG_OFFSET(2)]
@@ -118,6 +124,17 @@ ENTRY(__guest_exit)
 
 	get_host_ctxt	x2, x3
 
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_host(vcpu, guest, host).
+	// Save x0, x2 which are used later in callee saved registers.
+	mov	x19, x0
+	mov	x20, x2
+	sub	x0, x1, #VCPU_CONTEXT
+	ldr	x29, [x2, #CPU_XREG_OFFSET(29)]
+	bl	__ptrauth_switch_to_host
+	mov	x0, x19
+	mov	x2, x20
+#endif
 	// Now restore the host regs
 	restore_callee_saved_regs x2
 
arch/arm64/kvm/hyp/ptrauth-sr.c new file +101 -0

101 lines not on this page · raw message

arch/arm64/kvm/sys_regs.c +35 -2
index a6c9381..12529df 100644
@@ -986,6 +986,32 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
 	{ SYS_DESC(SYS_PMEVTYPERn_EL0(n)),					\
 	  access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
 
+
+void kvm_arm_vcpu_ptrauth_enable(struct kvm_vcpu *vcpu)
+{
+	vcpu->arch.ctxt.hcr_el2 |= (HCR_API | HCR_APK);
+}
+
+void kvm_arm_vcpu_ptrauth_disable(struct kvm_vcpu *vcpu)
+{
+	vcpu->arch.ctxt.hcr_el2 &= ~(HCR_API | HCR_APK);
+}
+
+static bool trap_ptrauth(struct kvm_vcpu *vcpu,
+			 struct sys_reg_params *p,
+			 const struct sys_reg_desc *rd)
+{
+	kvm_arm_vcpu_ptrauth_trap(vcpu);
+	return false;
+}
+
+#define __PTRAUTH_KEY(k)						\
+	{ SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k }
+
+#define PTRAUTH_KEY(k)							\
+	__PTRAUTH_KEY(k ## KEYLO_EL1),					\
+	__PTRAUTH_KEY(k ## KEYHI_EL1)
+
 static bool access_cntp_tval(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
@@ -1045,9 +1071,10 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
 					 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPI_SHIFT);
-		if (val & ptrauth_mask)
+		if (!kvm_supports_ptrauth()) {
 			kvm_debug("ptrauth unsupported for guests, suppressing\n");
-		val &= ~ptrauth_mask;
+			val &= ~ptrauth_mask;
+		}
 	} else if (id == SYS_ID_AA64MMFR1_EL1) {
 		if (val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))
 			kvm_debug("LORegions unsupported for guests, suppressing\n");
@@ -1316,6 +1343,12 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	{ SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 },
 	{ SYS_DESC(SYS_TCR_EL1), access_vm_reg, reset_val, TCR_EL1, 0 },
 
+	PTRAUTH_KEY(APIA),
+	PTRAUTH_KEY(APIB),
+	PTRAUTH_KEY(APDA),
+	PTRAUTH_KEY(APDB),
+	PTRAUTH_KEY(APGA),
+
 	{ SYS_DESC(SYS_AFSR0_EL1), access_vm_reg, reset_unknown, AFSR0_EL1 },
 	{ SYS_DESC(SYS_AFSR1_EL1), access_vm_reg, reset_unknown, AFSR1_EL1 },
 	{ SYS_DESC(SYS_ESR_EL1), access_vm_reg, reset_unknown, ESR_EL1 },
virt/kvm/arm/arm.c +2 -0
index 2032a66..d7e003f 100644
@@ -388,6 +388,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		vcpu_clear_wfe_traps(vcpu);
 	else
 		vcpu_set_wfe_traps(vcpu);
+
+	kvm_arm_vcpu_ptrauth_reset(vcpu);
 }
 
 void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
Signature · 1 line
2.7.4
reviewer
#15 lore raw

Re: [PATCH v6 3/6] arm64/kvm: context-switch ptrauth registers

Quoted 9 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When pointer authentication is supported, a guest may wish to use it. This patch adds the necessary KVM infrastructure for this to work, with a semi-lazy context switch of the pointer auth state.

Pointer authentication feature is only enabled when VHE is built in the kernel and present into CPU implementation so only VHE code paths are modified.

Nit: s/into/in the/

Quoted 24 lines

When we schedule a vcpu, we disable guest usage of pointer authentication instructions and accesses to the keys. While these are disabled, we avoid context-switching the keys. When we trap the guest trying to use pointer authentication functionality, we change to eagerly context-switching the keys, and enable the feature. The next time the vcpu is scheduled out/in, we start again. However the host key registers are saved in vcpu load stage as they remain constant for each vcpu schedule.

Pointer authentication consists of address authentication and generic authentication, and CPUs in a system might have varied support for either. Where support for either feature is not uniform, it is hidden from guests via ID register emulation, as a result of the cpufeature framework in the host.

Unfortunately, address authentication and generic authentication cannot be trapped separately, as the architecture provides a single EL2 trap covering both. If we wish to expose one without the other, we cannot prevent a (badly-written) guest from intermittently using a feature which is not uniformly supported (when scheduled on a physical CPU which supports the relevant feature). Hence, this patch expects both type of authentication to be present in a cpu.

Quoted 3 lines

[Only VHE, key switch from from assembly, kvm_supports_ptrauth checks, save host key in vcpu_load]

Hmm, why do we need to do the key switch in assembly, given it's not used in-kernel right now?

Is that in preparation for in-kernel pointer auth usage? If so, please call that out in the commit message.

[...]

arch/arm64/kernel/traps.c static const char *esr_class_str[] = { · line 752

Quoted 9 lines
arch/arm64/kernel/traps.c +1 -0
index 4e2fb87..5cac605 100644
@@ -749,6 +749,7 @@ static const char *esr_class_str[] = {
 	[ESR_ELx_EC_CP14_LS]		= "CP14 LDC/STC",
 	[ESR_ELx_EC_FP_ASIMD]		= "ASIMD",
 	[ESR_ELx_EC_CP10_ID]		= "CP10 MRC/VMRS",
+	[ESR_ELx_EC_PAC]		= "Pointer authentication trap",

For consistency with the other strings, can we please make this "PAC"?

[...]

arch/arm64/kvm/hyp/Makefile obj-$(CONFIG_KVM_ARM_HOST) += switch.o · line 22

Quoted 9 lines
arch/arm64/kvm/hyp/Makefile +1 -0
index 82d1904..17cec99 100644
@@ -19,6 +19,7 @@ obj-$(CONFIG_KVM_ARM_HOST) += switch.o
 obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
 obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
 obj-$(CONFIG_KVM_ARM_HOST) += hyp-entry.o
+obj-$(CONFIG_KVM_ARM_HOST) += ptrauth-sr.o

Huh, so we're actually doing the switch in C code...

Quoted 7 lines
arch/arm64/kvm/hyp/Makefile +0 -0
 # KVM code is run at a different exception code with a different map, so
 # compiler instrumentation that inserts callbacks or checks into the code may
arch/arm64/kvm/hyp/entry.S +0 -0
index 675fdc1..b78cc15 100644

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_enter) · line 71

Quoted 8 lines
arch/arm64/kvm/hyp/entry.S +5 -0
@@ -64,6 +64,12 @@ ENTRY(__guest_enter)
 
 	add	x18, x0, #VCPU_CONTEXT
 
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_guest(vcpu, host, guest).
+	mov	x2, x18
+	bl	__ptrauth_switch_to_guest
+#endif

... and conditionally *calling* that switch code from assembly ...

Quoted 9 lines
arch/arm64/kvm/hyp/entry.S +2 -0
+
 	// Restore guest regs x0-x17
 	ldp	x0, x1,   [x18, #CPU_XREG_OFFSET(0)]
 	ldp	x2, x3,   [x18, #CPU_XREG_OFFSET(2)]
@@ -118,6 +124,17 @@ ENTRY(__guest_exit)
 
 	get_host_ctxt	x2, x3
 
+#ifdef	CONFIG_ARM64_PTR_AUTH

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_exit) · line 137

Quoted 10 lines
arch/arm64/kvm/hyp/entry.S +10 -0
+	// Prepare parameter for __ptrauth_switch_to_host(vcpu, guest, host).
+	// Save x0, x2 which are used later in callee saved registers.
+	mov	x19, x0
+	mov	x20, x2
+	sub	x0, x1, #VCPU_CONTEXT
+	ldr	x29, [x2, #CPU_XREG_OFFSET(29)]
+	bl	__ptrauth_switch_to_host
+	mov	x0, x19
+	mov	x2, x20
+#endif

... which adds a load of boilerplate for no immediate gain.

Do we really need to do this in assembly today?

Thanks,
Mark.

#16 lore raw

Re: [PATCH v6 3/6] arm64/kvm: context-switch ptrauth registers

Quoted 46 lines · Mark Rutland
Quoted 9 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When pointer authentication is supported, a guest may wish to use it. This patch adds the necessary KVM infrastructure for this to work, with a semi-lazy context switch of the pointer auth state.

Pointer authentication feature is only enabled when VHE is built in the kernel and present into CPU implementation so only VHE code paths are modified.

Nit: s/into/in the/

Quoted 24 lines

When we schedule a vcpu, we disable guest usage of pointer authentication instructions and accesses to the keys. While these are disabled, we avoid context-switching the keys. When we trap the guest trying to use pointer authentication functionality, we change to eagerly context-switching the keys, and enable the feature. The next time the vcpu is scheduled out/in, we start again. However the host key registers are saved in vcpu load stage as they remain constant for each vcpu schedule.

Pointer authentication consists of address authentication and generic authentication, and CPUs in a system might have varied support for either. Where support for either feature is not uniform, it is hidden from guests via ID register emulation, as a result of the cpufeature framework in the host.

Unfortunately, address authentication and generic authentication cannot be trapped separately, as the architecture provides a single EL2 trap covering both. If we wish to expose one without the other, we cannot prevent a (badly-written) guest from intermittently using a feature which is not uniformly supported (when scheduled on a physical CPU which supports the relevant feature). Hence, this patch expects both type of authentication to be present in a cpu.

Quoted 3 lines

[Only VHE, key switch from from assembly, kvm_supports_ptrauth checks, save host key in vcpu_load]

Hmm, why do we need to do the key switch in assembly, given it's not used in-kernel right now?

Is that in preparation for in-kernel pointer auth usage? If so, please call that out in the commit message.

[...]

Quoted 42 lines

Huh, so we're actually doing the switch in C code...

Quoted 7 lines

# KVM code is run at a different exception code with a different map, so

 # compiler instrumentation that inserts callbacks or checks into the code may
arch/arm64/kvm/hyp/entry.S +0 -0
index 675fdc1..b78cc15 100644

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_enter) · line 71

Quoted 8 lines
arch/arm64/kvm/hyp/entry.S +5 -0
@@ -64,6 +64,12 @@ ENTRY(__guest_enter)
 
 	add	x18, x0, #VCPU_CONTEXT
 
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_guest(vcpu, host, guest).
+	mov	x2, x18
+	bl	__ptrauth_switch_to_guest
+#endif

... and conditionally *calling* that switch code from assembly ...

Quoted 9 lines
arch/arm64/kvm/hyp/entry.S +2 -0
+
 	// Restore guest regs x0-x17
 	ldp	x0, x1,   [x18, #CPU_XREG_OFFSET(0)]
 	ldp	x2, x3,   [x18, #CPU_XREG_OFFSET(2)]
@@ -118,6 +124,17 @@ ENTRY(__guest_exit)
 
 	get_host_ctxt	x2, x3
 
+#ifdef	CONFIG_ARM64_PTR_AUTH

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_exit) · line 137

Quoted 10 lines
arch/arm64/kvm/hyp/entry.S +10 -0
+	// Prepare parameter for __ptrauth_switch_to_host(vcpu, guest, host).
+	// Save x0, x2 which are used later in callee saved registers.
+	mov	x19, x0
+	mov	x20, x2
+	sub	x0, x1, #VCPU_CONTEXT
+	ldr	x29, [x2, #CPU_XREG_OFFSET(29)]
+	bl	__ptrauth_switch_to_host
+	mov	x0, x19
+	mov	x2, x20
+#endif

... which adds a load of boilerplate for no immediate gain.

Quoted 1 line

Do we really need to do this in assembly today?

If we will need to move this to assembly when we add in-kernel ptrauth support, it may be best to have it in assembly from the start, to reduce unnecessary churn.

But having a mix of C and assembly is likely to make things more
complicated: we should go with one or the other IMHO.

Cheers
---Dave

#17 lore raw

Re: [PATCH v6 3/6] arm64/kvm: context-switch ptrauth registers

Quoted 92 lines · Dave Martin
Quoted 46 lines · Mark Rutland
Quoted 9 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When pointer authentication is supported, a guest may wish to use it. This patch adds the necessary KVM infrastructure for this to work, with a semi-lazy context switch of the pointer auth state.

Pointer authentication feature is only enabled when VHE is built in the kernel and present into CPU implementation so only VHE code paths are modified.

Nit: s/into/in the/

Quoted 24 lines

When we schedule a vcpu, we disable guest usage of pointer authentication instructions and accesses to the keys. While these are disabled, we avoid context-switching the keys. When we trap the guest trying to use pointer authentication functionality, we change to eagerly context-switching the keys, and enable the feature. The next time the vcpu is scheduled out/in, we start again. However the host key registers are saved in vcpu load stage as they remain constant for each vcpu schedule.

Pointer authentication consists of address authentication and generic authentication, and CPUs in a system might have varied support for either. Where support for either feature is not uniform, it is hidden from guests via ID register emulation, as a result of the cpufeature framework in the host.

Unfortunately, address authentication and generic authentication cannot be trapped separately, as the architecture provides a single EL2 trap covering both. If we wish to expose one without the other, we cannot prevent a (badly-written) guest from intermittently using a feature which is not uniformly supported (when scheduled on a physical CPU which supports the relevant feature). Hence, this patch expects both type of authentication to be present in a cpu.

Quoted 3 lines

[Only VHE, key switch from from assembly, kvm_supports_ptrauth checks, save host key in vcpu_load]

Hmm, why do we need to do the key switch in assembly, given it's not used in-kernel right now?

Is that in preparation for in-kernel pointer auth usage? If so, please call that out in the commit message.

[...]

Quoted 42 lines

Huh, so we're actually doing the switch in C code...

Quoted 7 lines

# KVM code is run at a different exception code with a different map, so

  # compiler instrumentation that inserts callbacks or checks into the code may
arch/arm64/kvm/hyp/entry.S +0 -0
index 675fdc1..b78cc15 100644

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_enter) · line 71

Quoted 8 lines
arch/arm64/kvm/hyp/entry.S +5 -0
@@ -64,6 +64,12 @@ ENTRY(__guest_enter)
 
  	add	x18, x0, #VCPU_CONTEXT
 
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_guest(vcpu, host, guest).
+	mov	x2, x18
+	bl	__ptrauth_switch_to_guest
+#endif

... and conditionally *calling* that switch code from assembly ...

Quoted 9 lines
arch/arm64/kvm/hyp/entry.S +2 -0
+
  	// Restore guest regs x0-x17
  	ldp	x0, x1,   [x18, #CPU_XREG_OFFSET(0)]
  	ldp	x2, x3,   [x18, #CPU_XREG_OFFSET(2)]
@@ -118,6 +124,17 @@ ENTRY(__guest_exit)
 
  	get_host_ctxt	x2, x3
 
+#ifdef	CONFIG_ARM64_PTR_AUTH

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_exit) · line 137

Quoted 10 lines
arch/arm64/kvm/hyp/entry.S +10 -0
+	// Prepare parameter for __ptrauth_switch_to_host(vcpu, guest, host).
+	// Save x0, x2 which are used later in callee saved registers.
+	mov	x19, x0
+	mov	x20, x2
+	sub	x0, x1, #VCPU_CONTEXT
+	ldr	x29, [x2, #CPU_XREG_OFFSET(29)]
+	bl	__ptrauth_switch_to_host
+	mov	x0, x19
+	mov	x2, x20
+#endif

... which adds a load of boilerplate for no immediate gain.

Quoted 8 lines
Quoted 1 line

Do we really need to do this in assembly today?

If we will need to move this to assembly when we add in-kernel ptrauth support, it may be best to have it in assembly from the start, to reduce unnecessary churn.

But having a mix of C and assembly is likely to make things more
complicated: we should go with one or the other IMHO.

ok, I will check on this.

Thanks,
Amit D

Quoted 4 lines

Cheers
---Dave

#18 lore raw

Re: [PATCH v6 3/6] arm64/kvm: context-switch ptrauth registers

In reply to Mark Rutland

Hi Mark,

Quoted 7 lines · Mark Rutland
Quoted 6 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When pointer authentication is supported, a guest may wish to use it. This patch adds the necessary KVM infrastructure for this to work, with a semi-lazy context switch of the pointer auth state.

Quoted 5 lines
Quoted 3 lines

Pointer authentication feature is only enabled when VHE is built in the kernel and present into CPU implementation so only VHE code paths are modified.

Nit: s/into/in the/

ok.

Quoted 29 lines
Quoted 25 lines

When we schedule a vcpu, we disable guest usage of pointer authentication instructions and accesses to the keys. While these are disabled, we avoid context-switching the keys. When we trap the guest trying to use pointer authentication functionality, we change to eagerly context-switching the keys, and enable the feature. The next time the vcpu is scheduled out/in, we start again. However the host key registers are saved in vcpu load stage as they remain constant for each vcpu schedule.

Pointer authentication consists of address authentication and generic authentication, and CPUs in a system might have varied support for either. Where support for either feature is not uniform, it is hidden from guests via ID register emulation, as a result of the cpufeature framework in the host.

Unfortunately, address authentication and generic authentication cannot be trapped separately, as the architecture provides a single EL2 trap covering both. If we wish to expose one without the other, we cannot prevent a (badly-written) guest from intermittently using a feature which is not uniformly supported (when scheduled on a physical CPU which supports the relevant feature). Hence, this patch expects both type of authentication to be present in a cpu.

Quoted 3 lines

[Only VHE, key switch from from assembly, kvm_supports_ptrauth checks, save host key in vcpu_load]

Quoted 5 lines

Hmm, why do we need to do the key switch in assembly, given it's not used in-kernel right now?

Is that in preparation for in-kernel pointer auth usage? If so, please call that out in the commit message.

ok sure.

Quoted 14 lines

[...]

arch/arm64/kernel/traps.c static const char *esr_class_str[] = { · line 752

Quoted 9 lines
arch/arm64/kernel/traps.c +1 -0
index 4e2fb87..5cac605 100644
@@ -749,6 +749,7 @@ static const char *esr_class_str[] = {
  	[ESR_ELx_EC_CP14_LS]		= "CP14 LDC/STC",
  	[ESR_ELx_EC_FP_ASIMD]		= "ASIMD",
  	[ESR_ELx_EC_CP10_ID]		= "CP10 MRC/VMRS",
+	[ESR_ELx_EC_PAC]		= "Pointer authentication trap",

For consistency with the other strings, can we please make this "PAC"?

ok. It makes sense.

Quoted 41 lines

[...]

arch/arm64/kvm/hyp/Makefile obj-$(CONFIG_KVM_ARM_HOST) += switch.o · line 22

Quoted 9 lines
arch/arm64/kvm/hyp/Makefile +1 -0
index 82d1904..17cec99 100644
@@ -19,6 +19,7 @@ obj-$(CONFIG_KVM_ARM_HOST) += switch.o
  obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
  obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
  obj-$(CONFIG_KVM_ARM_HOST) += hyp-entry.o
+obj-$(CONFIG_KVM_ARM_HOST) += ptrauth-sr.o

Huh, so we're actually doing the switch in C code...

Quoted 7 lines
arch/arm64/kvm/hyp/Makefile +0 -0
  # KVM code is run at a different exception code with a different map, so
  # compiler instrumentation that inserts callbacks or checks into the code may
arch/arm64/kvm/hyp/entry.S +0 -0
index 675fdc1..b78cc15 100644

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_enter) · line 71

Quoted 8 lines
arch/arm64/kvm/hyp/entry.S +5 -0
@@ -64,6 +64,12 @@ ENTRY(__guest_enter)
 
  	add	x18, x0, #VCPU_CONTEXT
 
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_guest(vcpu, host, guest).
+	mov	x2, x18
+	bl	__ptrauth_switch_to_guest
+#endif

... and conditionally *calling* that switch code from assembly ...

Quoted 8 lines
arch/arm64/kvm/hyp/entry.S +1 -0
+
  	// Restore guest regs x0-x17
  	ldp	x0, x1,   [x18, #CPU_XREG_OFFSET(0)]
  	ldp	x2, x3,   [x18, #CPU_XREG_OFFSET(2)]
@@ -118,6 +124,17 @@ ENTRY(__guest_exit)
 
  	get_host_ctxt	x2, x3
 
Quoted 13 lines

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_exit) · line 137

Quoted 11 lines
arch/arm64/kvm/hyp/entry.S +11 -0
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_host(vcpu, guest, host).
+	// Save x0, x2 which are used later in callee saved registers.
+	mov	x19, x0
+	mov	x20, x2
+	sub	x0, x1, #VCPU_CONTEXT
+	ldr	x29, [x2, #CPU_XREG_OFFSET(29)]
+	bl	__ptrauth_switch_to_host
+	mov	x0, x19
+	mov	x2, x20
+#endif

... which adds a load of boilerplate for no immediate gain.

Here some parameter optimizations may be possible as guest and host ctxt can be derived from vcpu itself as James suggested in other review comments. I thought about doing all save/restore in assembly but for optimization now host keys are saved in vcpu_load stage in C so reused those C codes here also.

Again all these codes are beneficial with in-kernel ptrauth and hence in case of strong objection may revert to old way.

Quoted 2 lines

Do we really need to do this in assembly today?

During the last patchset review [1], James provided a lot of supporting arguments to have these switch routines called from assembly due to function outlining between kvm_vcpu_run_vhe() and __kvm_vcpu_run_nvhe().

[1]: https://lkml.org/lkml/2019/1/31/662

Thanks,
Amit D

Quoted 4 lines

Thanks,
Mark.

#19 lore raw

Re: [PATCH v6 3/6] arm64/kvm: context-switch ptrauth registers

In reply to Amit Daniel Kachhap
Quoted 54 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When pointer authentication is supported, a guest may wish to use it. This patch adds the necessary KVM infrastructure for this to work, with a semi-lazy context switch of the pointer auth state.

Pointer authentication feature is only enabled when VHE is built in the kernel and present into CPU implementation so only VHE code paths are modified.

When we schedule a vcpu, we disable guest usage of pointer authentication instructions and accesses to the keys. While these are disabled, we avoid context-switching the keys. When we trap the guest trying to use pointer authentication functionality, we change to eagerly context-switching the keys, and enable the feature. The next time the vcpu is scheduled out/in, we start again. However the host key registers are saved in vcpu load stage as they remain constant for each vcpu schedule.

Pointer authentication consists of address authentication and generic authentication, and CPUs in a system might have varied support for either. Where support for either feature is not uniform, it is hidden from guests via ID register emulation, as a result of the cpufeature framework in the host.

Unfortunately, address authentication and generic authentication cannot be trapped separately, as the architecture provides a single EL2 trap covering both. If we wish to expose one without the other, we cannot prevent a (badly-written) guest from intermittently using a feature which is not uniformly supported (when scheduled on a physical CPU which supports the relevant feature). Hence, this patch expects both type of authentication to be present in a cpu.

[Only VHE, key switch from from assembly, kvm_supports_ptrauth checks, save host key in vcpu_load]

---
 arch/arm/include/asm/kvm_host.h   |   1 +
 arch/arm64/include/asm/kvm_host.h |  23 +++++++++
 arch/arm64/include/asm/kvm_hyp.h  |   7 +++
 arch/arm64/kernel/traps.c         |   1 +
 arch/arm64/kvm/handle_exit.c      |  21 +++++---
 arch/arm64/kvm/hyp/Makefile       |   1 +
 arch/arm64/kvm/hyp/entry.S        |  17 +++++++
 arch/arm64/kvm/hyp/ptrauth-sr.c   | 101 ++++++++++++++++++++++++++++++++++++++
 arch/arm64/kvm/sys_regs.c         |  37 +++++++++++++-
 virt/kvm/arm/arm.c                |   2 +
 10 files changed, 201 insertions(+), 10 deletions(-)
 create mode 100644 arch/arm64/kvm/hyp/ptrauth-sr.c

[...]

Quoted 26 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +20 -0
new file mode 100644
index 0000000..528ee6e
@@ -0,0 +1,101 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * arch/arm64/kvm/hyp/ptrauth-sr.c: Guest/host ptrauth save/restore
+ *
+ * Copyright 2018 Arm Limited
+ * Author: Mark Rutland <[email protected]>
+ *         Amit Daniel Kachhap <[email protected]>
+ */
+#include <linux/compiler.h>
+#include <linux/kvm_host.h>
+
+#include <asm/cpucaps.h>
+#include <asm/cpufeature.h>
+#include <asm/kvm_asm.h>
+#include <asm/kvm_hyp.h>
+#include <asm/pointer_auth.h>
+
+static __always_inline bool __ptrauth_is_enabled(struct kvm_vcpu *vcpu)
+{
+	return IS_ENABLED(CONFIG_ARM64_PTR_AUTH) &&

arch/arm64/kvm/hyp/ptrauth-sr.c line 30

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +10 -0
+			vcpu->arch.ctxt.hcr_el2 & (HCR_API | HCR_APK);
+}
+
+#define __ptrauth_save_key(regs, key)						\
+({										\
+	regs[key ## KEYLO_EL1] = read_sysreg_s(SYS_ ## key ## KEYLO_EL1);	\
+	regs[key ## KEYHI_EL1] = read_sysreg_s(SYS_ ## key ## KEYHI_EL1);	\
+})
+
+static __always_inline void __ptrauth_save_state(struct kvm_cpu_context *ctxt)

Why __always_inline?

Quoted 5 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +5 -0
+{
+	__ptrauth_save_key(ctxt->sys_regs, APIA);
+	__ptrauth_save_key(ctxt->sys_regs, APIB);
+	__ptrauth_save_key(ctxt->sys_regs, APDA);
+	__ptrauth_save_key(ctxt->sys_regs, APDB);

arch/arm64/kvm/hyp/ptrauth-sr.c line 45

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +10 -0
+	__ptrauth_save_key(ctxt->sys_regs, APGA);
+}
+
+#define __ptrauth_restore_key(regs, key) 					\
+({										\
+	write_sysreg_s(regs[key ## KEYLO_EL1], SYS_ ## key ## KEYLO_EL1);	\
+	write_sysreg_s(regs[key ## KEYHI_EL1], SYS_ ## key ## KEYHI_EL1);	\
+})
+
+static __always_inline void __ptrauth_restore_state(struct kvm_cpu_context *ctxt)

Same here. I would hope these just need to be marked with the correct function attribute to disable ptrauth by the compiler. I don't see why it makes a difference whether it's inline or not.

If the compiler semantics are not sufficiently clear, make it a macro.

(Bikeshedding here, so it you feel this has already been discussed to death I'm happy for this to stay as-is.)

arch/arm64/kvm/hyp/ptrauth-sr.c line 57

Quoted 12 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +12 -0
+{
+	__ptrauth_restore_key(ctxt->sys_regs, APIA);
+	__ptrauth_restore_key(ctxt->sys_regs, APIB);
+	__ptrauth_restore_key(ctxt->sys_regs, APDA);
+	__ptrauth_restore_key(ctxt->sys_regs, APDB);
+	__ptrauth_restore_key(ctxt->sys_regs, APGA);
+}
+
+/**
+ * This function changes the key so assign Pointer Authentication safe
+ * GCC attribute if protected by it.
+ */

(I'd have preferred to keep __noptrauth here and define it do nothing for now. But I'll defer to others on that, since this has already been discussed...)

Quoted 25 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +25 -0
+void __ptrauth_switch_to_guest(struct kvm_vcpu *vcpu,
+				  struct kvm_cpu_context *host_ctxt,
+				  struct kvm_cpu_context *guest_ctxt)
+{
+	if (!__ptrauth_is_enabled(vcpu))
+		return;
+
+	__ptrauth_restore_state(guest_ctxt);
+}
+
+/**
+ * This function changes the key so assign Pointer Authentication safe
+ * GCC attribute if protected by it.
+ */
+void __ptrauth_switch_to_host(struct kvm_vcpu *vcpu,
+				 struct kvm_cpu_context *guest_ctxt,
+				 struct kvm_cpu_context *host_ctxt)
+{
+	if (!__ptrauth_is_enabled(vcpu))
+		return;
+
+	__ptrauth_save_state(guest_ctxt);
+	__ptrauth_restore_state(host_ctxt);
+}
+

arch/arm64/kvm/hyp/ptrauth-sr.c line 92

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +10 -0
+/**
+ * kvm_arm_vcpu_ptrauth_reset - resets ptrauth for vcpu schedule
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * This function may be used to disable ptrauth and use it in a lazy context
+ * via traps. However host key registers are saved here as they dont change
+ * during host/guest switch.
+ */
+void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu)

I feel this is not a good name. It sounds too much like it resets the registers as part of vcpu reset, whereas really it's doing something completely different.

(Do you reset the regs anywhere btw? I may have missed it...)

Quoted 14 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +9 -0
+{
+	struct kvm_cpu_context *host_ctxt;
+
+	if (kvm_supports_ptrauth()) {
+		kvm_arm_vcpu_ptrauth_disable(vcpu);
+		host_ctxt = vcpu->arch.host_cpu_context;
+		__ptrauth_save_state(host_ctxt);
+	}
+}
arch/arm64/kvm/sys_regs.c +0 -0
index a6c9381..12529df 100644

arch/arm64/kvm/sys_regs.c static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, · line 992

Quoted 7 lines
arch/arm64/kvm/sys_regs.c +4 -0
@@ -986,6 +986,32 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
 	{ SYS_DESC(SYS_PMEVTYPERn_EL0(n)),					\
 	  access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
 
+
+void kvm_arm_vcpu_ptrauth_enable(struct kvm_vcpu *vcpu)
+{
+	vcpu->arch.ctxt.hcr_el2 |= (HCR_API | HCR_APK);

Pedantic nit: surplus ().

(Although opinions differ, and keeping them looks more symmetric with kvm_arm_vcpu_ptrauth_disable() -- either way, the code can stay as-is if you prefer.)

arch/arm64/kvm/sys_regs.c static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, · line 1005

Quoted 13 lines
arch/arm64/kvm/sys_regs.c +13 -0
+}
+
+void kvm_arm_vcpu_ptrauth_disable(struct kvm_vcpu *vcpu)
+{
+	vcpu->arch.ctxt.hcr_el2 &= ~(HCR_API | HCR_APK);
+}
+
+static bool trap_ptrauth(struct kvm_vcpu *vcpu,
+			 struct sys_reg_params *p,
+			 const struct sys_reg_desc *rd)
+{
+	kvm_arm_vcpu_ptrauth_trap(vcpu);
+	return false;

Can we ever get here? Won't PAC traps always be handled via handle_exit()?

Or can we also take sysreg access traps when the guest tries to access the ptrauth key registers?

(I'm now wondering how this works for SVE.)

Quoted 13 lines
arch/arm64/kvm/sys_regs.c +9 -0
+}
+
+#define __PTRAUTH_KEY(k)						\
+	{ SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k }
+
+#define PTRAUTH_KEY(k)							\
+	__PTRAUTH_KEY(k ## KEYLO_EL1),					\
+	__PTRAUTH_KEY(k ## KEYHI_EL1)
+
 static bool access_cntp_tval(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)

arch/arm64/kvm/sys_regs.c static u64 read_id_reg(struct sys_reg_desc const *r, bool raz) · line 1074

Quoted 5 lines
arch/arm64/kvm/sys_regs.c +1 -1
@@ -1045,9 +1071,10 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
 					 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPI_SHIFT);
-		if (val & ptrauth_mask)
+		if (!kvm_supports_ptrauth()) {

Don't we now always print this when ptrauth is not supported?

Previously we only printed a message in the interesting case, i.e., where the host supports ptrauch but we cannot offer it to the guest.

Quoted 40 lines
arch/arm64/kvm/sys_regs.c +8 -1
 			kvm_debug("ptrauth unsupported for guests, suppressing\n");
-		val &= ~ptrauth_mask;
+			val &= ~ptrauth_mask;
+		}
 	} else if (id == SYS_ID_AA64MMFR1_EL1) {
 		if (val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))
 			kvm_debug("LORegions unsupported for guests, suppressing\n");
@@ -1316,6 +1343,12 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 	{ SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 },
 	{ SYS_DESC(SYS_TCR_EL1), access_vm_reg, reset_val, TCR_EL1, 0 },
 
+	PTRAUTH_KEY(APIA),
+	PTRAUTH_KEY(APIB),
+	PTRAUTH_KEY(APDA),
+	PTRAUTH_KEY(APDB),
+	PTRAUTH_KEY(APGA),
+
 	{ SYS_DESC(SYS_AFSR0_EL1), access_vm_reg, reset_unknown, AFSR0_EL1 },
 	{ SYS_DESC(SYS_AFSR1_EL1), access_vm_reg, reset_unknown, AFSR1_EL1 },
 	{ SYS_DESC(SYS_ESR_EL1), access_vm_reg, reset_unknown, ESR_EL1 },
virt/kvm/arm/arm.c +2 -0
index 2032a66..d7e003f 100644
@@ -388,6 +388,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		vcpu_clear_wfe_traps(vcpu);
 	else
 		vcpu_set_wfe_traps(vcpu);
+
+	kvm_arm_vcpu_ptrauth_reset(vcpu);
 }
 
 void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
Signature · 6 lines
2.7.4

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
#20 lore raw

Re: [PATCH v6 3/6] arm64/kvm: context-switch ptrauth registers

Hi,

Quoted 117 lines · Dave Martin
Quoted 54 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When pointer authentication is supported, a guest may wish to use it. This patch adds the necessary KVM infrastructure for this to work, with a semi-lazy context switch of the pointer auth state.

Pointer authentication feature is only enabled when VHE is built in the kernel and present into CPU implementation so only VHE code paths are modified.

When we schedule a vcpu, we disable guest usage of pointer authentication instructions and accesses to the keys. While these are disabled, we avoid context-switching the keys. When we trap the guest trying to use pointer authentication functionality, we change to eagerly context-switching the keys, and enable the feature. The next time the vcpu is scheduled out/in, we start again. However the host key registers are saved in vcpu load stage as they remain constant for each vcpu schedule.

Pointer authentication consists of address authentication and generic authentication, and CPUs in a system might have varied support for either. Where support for either feature is not uniform, it is hidden from guests via ID register emulation, as a result of the cpufeature framework in the host.

Unfortunately, address authentication and generic authentication cannot be trapped separately, as the architecture provides a single EL2 trap covering both. If we wish to expose one without the other, we cannot prevent a (badly-written) guest from intermittently using a feature which is not uniformly supported (when scheduled on a physical CPU which supports the relevant feature). Hence, this patch expects both type of authentication to be present in a cpu.

[Only VHE, key switch from from assembly, kvm_supports_ptrauth checks, save host key in vcpu_load]

---
  arch/arm/include/asm/kvm_host.h   |   1 +
  arch/arm64/include/asm/kvm_host.h |  23 +++++++++
  arch/arm64/include/asm/kvm_hyp.h  |   7 +++
  arch/arm64/kernel/traps.c         |   1 +
  arch/arm64/kvm/handle_exit.c      |  21 +++++---
  arch/arm64/kvm/hyp/Makefile       |   1 +
  arch/arm64/kvm/hyp/entry.S        |  17 +++++++
  arch/arm64/kvm/hyp/ptrauth-sr.c   | 101 ++++++++++++++++++++++++++++++++++++++
  arch/arm64/kvm/sys_regs.c         |  37 +++++++++++++-
  virt/kvm/arm/arm.c                |   2 +
  10 files changed, 201 insertions(+), 10 deletions(-)

create mode 100644 arch/arm64/kvm/hyp/ptrauth-sr.c

[...]

Quoted 26 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +20 -0
new file mode 100644
index 0000000..528ee6e
@@ -0,0 +1,101 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * arch/arm64/kvm/hyp/ptrauth-sr.c: Guest/host ptrauth save/restore
+ *
+ * Copyright 2018 Arm Limited
+ * Author: Mark Rutland <[email protected]>
+ *         Amit Daniel Kachhap <[email protected]>
+ */
+#include <linux/compiler.h>
+#include <linux/kvm_host.h>
+
+#include <asm/cpucaps.h>
+#include <asm/cpufeature.h>
+#include <asm/kvm_asm.h>
+#include <asm/kvm_hyp.h>
+#include <asm/pointer_auth.h>
+
+static __always_inline bool __ptrauth_is_enabled(struct kvm_vcpu *vcpu)
+{
+	return IS_ENABLED(CONFIG_ARM64_PTR_AUTH) &&

arch/arm64/kvm/hyp/ptrauth-sr.c line 30

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +10 -0
+			vcpu->arch.ctxt.hcr_el2 & (HCR_API | HCR_APK);
+}
+
+#define __ptrauth_save_key(regs, key)						\
+({										\
+	regs[key ## KEYLO_EL1] = read_sysreg_s(SYS_ ## key ## KEYLO_EL1);	\
+	regs[key ## KEYHI_EL1] = read_sysreg_s(SYS_ ## key ## KEYHI_EL1);	\
+})
+
+static __always_inline void __ptrauth_save_state(struct kvm_cpu_context *ctxt)

Why __always_inline?

Quoted 5 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +5 -0
+{
+	__ptrauth_save_key(ctxt->sys_regs, APIA);
+	__ptrauth_save_key(ctxt->sys_regs, APIB);
+	__ptrauth_save_key(ctxt->sys_regs, APDA);
+	__ptrauth_save_key(ctxt->sys_regs, APDB);

arch/arm64/kvm/hyp/ptrauth-sr.c line 45

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +10 -0
+	__ptrauth_save_key(ctxt->sys_regs, APGA);
+}
+
+#define __ptrauth_restore_key(regs, key) 					\
+({										\
+	write_sysreg_s(regs[key ## KEYLO_EL1], SYS_ ## key ## KEYLO_EL1);	\
+	write_sysreg_s(regs[key ## KEYHI_EL1], SYS_ ## key ## KEYHI_EL1);	\
+})
+
+static __always_inline void __ptrauth_restore_state(struct kvm_cpu_context *ctxt)

Same here. I would hope these just need to be marked with the correct function attribute to disable ptrauth by the compiler. I don't see why it makes a difference whether it's inline or not.

Quoted 1 line

If the compiler semantics are not sufficiently clear, make it a macro.

ok.

Quoted 4 lines

(Bikeshedding here, so it you feel this has already been discussed to death I'm happy for this to stay as-is.)

Quoted 16 lines

arch/arm64/kvm/hyp/ptrauth-sr.c line 57

Quoted 12 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +12 -0
+{
+	__ptrauth_restore_key(ctxt->sys_regs, APIA);
+	__ptrauth_restore_key(ctxt->sys_regs, APIB);
+	__ptrauth_restore_key(ctxt->sys_regs, APDA);
+	__ptrauth_restore_key(ctxt->sys_regs, APDB);
+	__ptrauth_restore_key(ctxt->sys_regs, APGA);
+}
+
+/**
+ * This function changes the key so assign Pointer Authentication safe
+ * GCC attribute if protected by it.
+ */

(I'd have preferred to keep __noptrauth here and define it do nothing for now. But I'll defer to others on that, since this has already been discussed...)

ok __noptrauth annotation will make it clear. I will add it for all C error prone functions in the next iteration.

Quoted 41 lines
Quoted 26 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +25 -0
+void __ptrauth_switch_to_guest(struct kvm_vcpu *vcpu,
+				  struct kvm_cpu_context *host_ctxt,
+				  struct kvm_cpu_context *guest_ctxt)
+{
+	if (!__ptrauth_is_enabled(vcpu))
+		return;
+
+	__ptrauth_restore_state(guest_ctxt);
+}
+
+/**
+ * This function changes the key so assign Pointer Authentication safe
+ * GCC attribute if protected by it.
+ */
+void __ptrauth_switch_to_host(struct kvm_vcpu *vcpu,
+				 struct kvm_cpu_context *guest_ctxt,
+				 struct kvm_cpu_context *host_ctxt)
+{
+	if (!__ptrauth_is_enabled(vcpu))
+		return;
+
+	__ptrauth_save_state(guest_ctxt);
+	__ptrauth_restore_state(host_ctxt);
+}
+

arch/arm64/kvm/hyp/ptrauth-sr.c line 92

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +10 -0
+/**
+ * kvm_arm_vcpu_ptrauth_reset - resets ptrauth for vcpu schedule
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * This function may be used to disable ptrauth and use it in a lazy context
+ * via traps. However host key registers are saved here as they dont change
+ * during host/guest switch.
+ */
+void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu)

I feel this is not a good name. It sounds too much like it resets the registers as part of vcpu reset, whereas really it's doing something completely different.

Quoted 1 line

(Do you reset the regs anywhere btw? I may have missed it...)

No there is not reset of registers. May be name like kvm_arm_vcpu_ptrauth_setup_lazy would be better.

Quoted 25 lines
Quoted 15 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +9 -0
+{
+	struct kvm_cpu_context *host_ctxt;
+
+	if (kvm_supports_ptrauth()) {
+		kvm_arm_vcpu_ptrauth_disable(vcpu);
+		host_ctxt = vcpu->arch.host_cpu_context;
+		__ptrauth_save_state(host_ctxt);
+	}
+}
arch/arm64/kvm/sys_regs.c +0 -0
index a6c9381..12529df 100644

arch/arm64/kvm/sys_regs.c static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, · line 992

Quoted 7 lines
arch/arm64/kvm/sys_regs.c +4 -0
@@ -986,6 +986,32 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
  	{ SYS_DESC(SYS_PMEVTYPERn_EL0(n)),					\
  	  access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
 
+
+void kvm_arm_vcpu_ptrauth_enable(struct kvm_vcpu *vcpu)
+{
+	vcpu->arch.ctxt.hcr_el2 |= (HCR_API | HCR_APK);

Pedantic nit: surplus ().

Quoted 3 lines

(Although opinions differ, and keeping them looks more symmetric with kvm_arm_vcpu_ptrauth_disable() -- either way, the code can stay as-is if you prefer.)

ok.

Quoted 18 lines
Quoted 4 lines
arch/arm64/kvm/sys_regs.c +3 -0
+}
+
+void kvm_arm_vcpu_ptrauth_disable(struct kvm_vcpu *vcpu)

arch/arm64/kvm/sys_regs.c static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, · line 1005

Quoted 10 lines
arch/arm64/kvm/sys_regs.c +10 -0
+{
+	vcpu->arch.ctxt.hcr_el2 &= ~(HCR_API | HCR_APK);
+}
+
+static bool trap_ptrauth(struct kvm_vcpu *vcpu,
+			 struct sys_reg_params *p,
+			 const struct sys_reg_desc *rd)
+{
+	kvm_arm_vcpu_ptrauth_trap(vcpu);
+	return false;

Can we ever get here? Won't PAC traps always be handled via handle_exit()?

Quoted 2 lines

Or can we also take sysreg access traps when the guest tries to access the ptrauth key registers?

When Guest kernel forks thread then the key registers are accessed to fill them with ptrauth keys and hcr_el2(APK bit) is not appropriately set at that moment. This causes trap to EL2 and the above function is invoked.

Quoted 2 lines

(I'm now wondering how this works for SVE.)

Not sure. Need to check.

Quoted 22 lines
Quoted 14 lines
arch/arm64/kvm/sys_regs.c +9 -0
+}
+
+#define __PTRAUTH_KEY(k)						\
+	{ SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k }
+
+#define PTRAUTH_KEY(k)							\
+	__PTRAUTH_KEY(k ## KEYLO_EL1),					\
+	__PTRAUTH_KEY(k ## KEYHI_EL1)
+
  static bool access_cntp_tval(struct kvm_vcpu *vcpu,
  		struct sys_reg_params *p,
  		const struct sys_reg_desc *r)

arch/arm64/kvm/sys_regs.c static u64 read_id_reg(struct sys_reg_desc const *r, bool raz) · line 1074

Quoted 5 lines
arch/arm64/kvm/sys_regs.c +1 -1
@@ -1045,9 +1071,10 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
  					 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
  					 (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
  					 (0xfUL << ID_AA64ISAR1_GPI_SHIFT);
-		if (val & ptrauth_mask)
+		if (!kvm_supports_ptrauth()) {

Don't we now always print this when ptrauth is not supported?

Quoted 2 lines

Previously we only printed a message in the interesting case, i.e., where the host supports ptrauch but we cannot offer it to the guest.

Yes agreed. I will add proper checks here to skip prints for non ptrauth hosts.

Thanks,
Amit D

Quoted 41 lines
Quoted 41 lines
arch/arm64/kvm/sys_regs.c +8 -1
  			kvm_debug("ptrauth unsupported for guests, suppressing\n");
-		val &= ~ptrauth_mask;
+			val &= ~ptrauth_mask;
+		}
  	} else if (id == SYS_ID_AA64MMFR1_EL1) {
  		if (val & (0xfUL << ID_AA64MMFR1_LOR_SHIFT))
  			kvm_debug("LORegions unsupported for guests, suppressing\n");
@@ -1316,6 +1343,12 @@ static const struct sys_reg_desc sys_reg_descs[] = {
  	{ SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 },
  	{ SYS_DESC(SYS_TCR_EL1), access_vm_reg, reset_val, TCR_EL1, 0 },
 
+	PTRAUTH_KEY(APIA),
+	PTRAUTH_KEY(APIB),
+	PTRAUTH_KEY(APDA),
+	PTRAUTH_KEY(APDB),
+	PTRAUTH_KEY(APGA),
+
  	{ SYS_DESC(SYS_AFSR0_EL1), access_vm_reg, reset_unknown, AFSR0_EL1 },
  	{ SYS_DESC(SYS_AFSR1_EL1), access_vm_reg, reset_unknown, AFSR1_EL1 },
  	{ SYS_DESC(SYS_ESR_EL1), access_vm_reg, reset_unknown, ESR_EL1 },
virt/kvm/arm/arm.c +2 -0
index 2032a66..d7e003f 100644
@@ -388,6 +388,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
  		vcpu_clear_wfe_traps(vcpu);
  	else
  		vcpu_set_wfe_traps(vcpu);
+
+	kvm_arm_vcpu_ptrauth_reset(vcpu);
  }
 
  void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
Signature · 6 lines
2.7.4

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
#21 lore raw

Re: [PATCH v6 3/6] arm64/kvm: context-switch ptrauth registers

In reply to Amit Daniel Kachhap

Hi Amit,

Quoted 17 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When pointer authentication is supported, a guest may wish to use it. This patch adds the necessary KVM infrastructure for this to work, with a semi-lazy context switch of the pointer auth state.

Pointer authentication feature is only enabled when VHE is built in the kernel and present into CPU implementation so only VHE code paths are modified.

When we schedule a vcpu, we disable guest usage of pointer authentication instructions and accesses to the keys. While these are disabled, we avoid context-switching the keys. When we trap the guest trying to use pointer authentication functionality, we change to eagerly context-switching the keys, and enable the feature. The next time the vcpu is scheduled out/in, we start again.

Quoted 3 lines

However the host key registers
are saved in vcpu load stage as they remain constant for each vcpu schedule.

(I think we can get away with doing this later ... with the hope of doing it never!)

Quoted 20 lines

Pointer authentication consists of address authentication and generic authentication, and CPUs in a system might have varied support for either. Where support for either feature is not uniform, it is hidden from guests via ID register emulation, as a result of the cpufeature framework in the host.

Unfortunately, address authentication and generic authentication cannot be trapped separately, as the architecture provides a single EL2 trap covering both. If we wish to expose one without the other, we cannot prevent a (badly-written) guest from intermittently using a feature which is not uniformly supported (when scheduled on a physical CPU which supports the relevant feature). Hence, this patch expects both type of authentication to be present in a cpu.

arch/arm64/include/asm/kvm_host.h +0 -0
index 2f1bb86..1bacf78 100644

arch/arm64/include/asm/kvm_host.h enum vcpu_sysreg { · line 151

Quoted 6 lines
arch/arm64/include/asm/kvm_host.h +5 -0
@@ -146,6 +146,18 @@ enum vcpu_sysreg {
 
+static inline bool kvm_supports_ptrauth(void)
+{
+	return has_vhe() && system_supports_address_auth() &&
+				system_supports_generic_auth();
+}

Do we intend to support the implementation defined algorithm? I'd assumed not.

system_supports_address_auth() is defined as:

| static inline bool system_supports_address_auth(void)
| {
| 	return IS_ENABLED(CONFIG_ARM64_PTR_AUTH) &&
| 		(cpus_have_const_cap(ARM64_HAS_ADDRESS_AUTH_ARCH) ||
| 		cpus_have_const_cap(ARM64_HAS_ADDRESS_AUTH_IMP_DEF));
| }

So we could return true from kvm_supports_ptrauth() even if we only support the imp-def algorithm.

I think we should hide the imp-def ptrauth support as KVM hides all other imp-def features. This lets us avoid trying to migrate values that have been signed with the imp-def algorithm.

I'm worried that it could include some value that we can't migrate (e.g. the SoC serial number). Does the ARM-ARM say this can't happen?

All I can find is D5.1.5 "Pointer authentication in AArch64 state" of DDI0487D.a which has this clause for the imp-def algorithm:
| For a set of arguments passed to the function, must give the same result for all PEs | that a thread of execution could migrate between.

... with KVM we've extended the scope of migration significantly.

Could we check the cpus_have_const_cap() values for the two architected algorithms directly?

Quoted 5 lines
arch/arm64/include/asm/kvm_hyp.h +0 -0
index 6e65cad..09e061a 100644

arch/arm64/include/asm/kvm_hyp.h bool __fpsimd_enabled(void); · line 161

Quoted 9 lines
arch/arm64/include/asm/kvm_hyp.h +6 -0
@@ -153,6 +153,13 @@ bool __fpsimd_enabled(void);
 void activate_traps_vhe_load(struct kvm_vcpu *vcpu);
 void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu);
 
+void __ptrauth_switch_to_guest(struct kvm_vcpu *vcpu,
+			       struct kvm_cpu_context *host_ctxt,
+			       struct kvm_cpu_context *guest_ctxt);
+void __ptrauth_switch_to_host(struct kvm_vcpu *vcpu,
+			      struct kvm_cpu_context *guest_ctxt,
+			      struct kvm_cpu_context *host_ctxt);

Why do you need the vcpu and the guest_ctxt?
Would it be possible for these to just take the vcpu, and to pull the host context from the per-cpu variable?
This would avoid any future bugs where the ctxt's are the wrong way round, taking two is unusual in KVM, but necessary here.

As you're calling these from asm you want the compiler to do as much of the type mangling as possible.

arch/arm64/kernel/traps.c static const char *esr_class_str[] = { · line 755

Quoted 12 lines
arch/arm64/kernel/traps.c +1 -0
index 4e2fb87..5cac605 100644
@@ -749,6 +749,7 @@ static const char *esr_class_str[] = {
 	[ESR_ELx_EC_CP14_LS]		= "CP14 LDC/STC",
 	[ESR_ELx_EC_FP_ASIMD]		= "ASIMD",
 	[ESR_ELx_EC_CP10_ID]		= "CP10 MRC/VMRS",
+	[ESR_ELx_EC_PAC]		= "Pointer authentication trap",
 	[ESR_ELx_EC_CP14_64]		= "CP14 MCRR/MRRC",
 	[ESR_ELx_EC_ILL]		= "PSTATE.IL",
 	[ESR_ELx_EC_SVC32]		= "SVC (AArch32)",

Is this needed? Or was it just missing from the parts already merged. (should it be a separate patch for the arch code)

It looks like KVM only prints it from kvm_handle_unknown_ec(), which would never happen as arm_exit_handlers[] has an entry for ESR_ELx_EC_PAC.

Is it true that the host could never take this trap either?, as it can only be taken when HCR_EL2.TGE is clear.
(breadcrumbs from the ESR_ELx definition to "Trap to EL2 of EL0 accesses to Pointer authentication instructions")

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_enter) · line 71

Quoted 13 lines
arch/arm64/kvm/hyp/entry.S +5 -0
index 675fdc1..b78cc15 100644
@@ -64,6 +64,12 @@ ENTRY(__guest_enter)
 
 	add	x18, x0, #VCPU_CONTEXT
 
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_guest(vcpu, host, guest).
+	mov	x2, x18
+	bl	__ptrauth_switch_to_guest
+#endif

This calls back into C code with x18 clobbered... is that allowed?
x18 has this weird platform-register/temporary-register behaviour, that depends on the compiler. The PCS[0] has a note that 'hand-coded assembler should avoid it entirely'!

Can we assume that compiler generated code is using it from something, and depends on that value, which means we need to preserve or save/restore it when calling into C.

The upshot? Could you use one of the callee saved registers instead of x18, then move it after your C call.

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_exit) · line 134

Quoted 12 lines
arch/arm64/kvm/hyp/entry.S +7 -0
@@ -118,6 +124,17 @@ ENTRY(__guest_exit)
 
 	get_host_ctxt	x2, x3
 
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_host(vcpu, guest, host).
+	// Save x0, x2 which are used later in callee saved registers.
+	mov	x19, x0
+	mov	x20, x2
+	sub	x0, x1, #VCPU_CONTEXT
 
+	ldr	x29, [x2, #CPU_XREG_OFFSET(29)]

Is this to make the stack-trace look plausible?

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_exit) · line 137

Quoted 4 lines
arch/arm64/kvm/hyp/entry.S +4 -0
+	bl	__ptrauth_switch_to_host
+	mov	x0, x19
+	mov	x2, x20
+#endif

(ditching the host_ctxt would let you move this above get_host_ctxt and the need to preserve its result)

arch/arm64/kvm/hyp/ptrauth-sr.c

Quoted 8 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +1 -0
new file mode 100644
index 0000000..528ee6e
@@ -0,0 +1,101 @@
 
+static __always_inline bool __ptrauth_is_enabled(struct kvm_vcpu *vcpu)

This __always_inline still looks weird! You said it might be needed to make it function pointer safe. If it is, could you add a comment explaining why.

(alternatives would be making it an #ifdef, disabling ptrauth for the whole file, or annotating this function too)

Quoted 18 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +18 -0
+#define __ptrauth_save_key(regs, key)						\
+({										\
+	regs[key ## KEYLO_EL1] = read_sysreg_s(SYS_ ## key ## KEYLO_EL1);	\
+	regs[key ## KEYHI_EL1] = read_sysreg_s(SYS_ ## key ## KEYHI_EL1);	\
+})
+
+static __always_inline void __ptrauth_save_state(struct kvm_cpu_context *ctxt)
+{
+	__ptrauth_save_key(ctxt->sys_regs, APIA);
+	__ptrauth_save_key(ctxt->sys_regs, APIB);
+	__ptrauth_save_key(ctxt->sys_regs, APDA);
+	__ptrauth_save_key(ctxt->sys_regs, APDB);
+	__ptrauth_save_key(ctxt->sys_regs, APGA);
+}
+
+#define __ptrauth_restore_key(regs, key) 					\
+({										\
+	write_sysreg_s(regs[key ## KEYLO_EL1], SYS_ ## key ## KEYLO_EL1);	\

arch/arm64/kvm/hyp/ptrauth-sr.c line 51

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +10 -0
+	write_sysreg_s(regs[key ## KEYHI_EL1], SYS_ ## key ## KEYHI_EL1);	\
+})
+
+static __always_inline void __ptrauth_restore_state(struct kvm_cpu_context *ctxt)
+{
+	__ptrauth_restore_key(ctxt->sys_regs, APIA);
+	__ptrauth_restore_key(ctxt->sys_regs, APIB);
+	__ptrauth_restore_key(ctxt->sys_regs, APDA);
+	__ptrauth_restore_key(ctxt->sys_regs, APDB);
+	__ptrauth_restore_key(ctxt->sys_regs, APGA);

Are writes to these registers self synchronising? I'd assume not, as they come as a pair.

I think this means we need an isb() here to ensure that when restoring the host registers the next host authentication attempt uses the key we wrote here? We don't need it for the guest, so we could put it at the end of __ptrauth_switch_to_host().

Quoted 4 lines

+/**
+ * This function changes the key so assign Pointer Authentication safe
+ * GCC attribute if protected by it.
+ */

... this comment is the reminder for 'once we have host kernel ptrauth support'? could we add something to say that kernel support is when the attribute would be needed. Otherwise it reads like we're waiting for GCC support.

(I assume LLVM has a similar attribute ... is it exactly the same?)

arch/arm64/kvm/hyp/ptrauth-sr.c line 81

Quoted 13 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +10 -0
+void __ptrauth_switch_to_guest(struct kvm_vcpu *vcpu,
+				  struct kvm_cpu_context *host_ctxt,
+				  struct kvm_cpu_context *guest_ctxt)
+{
 
+}
 
+void __ptrauth_switch_to_host(struct kvm_vcpu *vcpu,
+				 struct kvm_cpu_context *guest_ctxt,
+				 struct kvm_cpu_context *host_ctxt)
+{
 
+}

Could you add NOKPROBE_SYMBOL(symbol_name) for these. This adds them to the kprobe blacklist as they aren't part of the __hyp_text. (and don't need to be as its VHE only). Without this, you can patch a software-breakpoint in here, which KVM won't handle as its already switched VBAR for entry to the guest.

Details in 7d82602909ed ("KVM: arm64: Forbid kprobing of the VHE world-switch code")

(... this turned up in a kernel version later than you based on ...)

Quoted 8 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +8 -0
+/**
+ * kvm_arm_vcpu_ptrauth_reset - resets ptrauth for vcpu schedule
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * This function may be used to disable ptrauth and use it in a lazy context
+ * via traps. However host key registers are saved here as they dont change
+ * during host/guest switch.

arch/arm64/kvm/hyp/ptrauth-sr.c line 99

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +9 -0
+ */
+void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu)
+{
+	struct kvm_cpu_context *host_ctxt;
+
+	if (kvm_supports_ptrauth()) {
+		kvm_arm_vcpu_ptrauth_disable(vcpu);
+		host_ctxt = vcpu->arch.host_cpu_context;
 
+		__ptrauth_save_state(host_ctxt);

Could you equally do this host-save in kvm_arm_vcpu_ptrauth_trap() before kvm_arm_vcpu_ptrauth_enable()? This would avoid saving the keys if the guest never gets the opportunity to change them. At the moment we do it on every vcpu_load().

As kvm_arm_vcpu_ptrauth_reset() isn't used as part of the world-switch, could we keep it outside the 'hyp' directory? The Makefile for that directory expects to be building the hyp text, so it disables KASAN, KCOV and friends. kvm_arm_vcpu_ptrauth_reset() is safe for all of these, and its good for it to be covered by this debug infrastructure. Could you move it to guest.c?

arch/arm64/kvm/sys_regs.c> @@ -1045,9 +1071,10 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)

Quoted 13 lines
arch/arm64/kvm/sys_regs.c> @@ -1045,9 +1071,10 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz) renamed from arch/arm64/kvm/sys_regs.c +3 -2
index a6c9381..12529df 100644
 					 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPI_SHIFT);
-		if (val & ptrauth_mask)
+		if (!kvm_supports_ptrauth()) {
 			kvm_debug("ptrauth unsupported for guests, suppressing\n");
-		val &= ~ptrauth_mask;
+			val &= ~ptrauth_mask;
+		}

This means that debug message gets printed even on systems that don't support ptrauth in hardware. (val&ptrauth_mask) used to cut them out, now kvm_supports_ptrauth() fails if the static keys are false, and we end up printing this message.
Now that KVM supports pointer-auth, I don't think the debug message is useful, can we remove it? (it would now mean 'you didn't ask for ptrauth to be turned on')

Could we always mask out the imp-def bits?

This patch needs to be merged together with 4 & 5 so the user-abi is as it should be. This means the check_present/restrictions thing needs sorting so they're ready together.

Thanks,

James

[0] http://infocenter.arm.com/help/topic/com.arm.doc.ihi0055b/IHI0055B_aapcs64.pdf

#22 lore raw

Re: [PATCH v6 3/6] arm64/kvm: context-switch ptrauth registers

Hi James,

Quoted 65 lines · James Morse

Hi Amit,

Quoted 17 lines · Amit Daniel Kachhap

From: Mark Rutland <[email protected]>

When pointer authentication is supported, a guest may wish to use it. This patch adds the necessary KVM infrastructure for this to work, with a semi-lazy context switch of the pointer auth state.

Pointer authentication feature is only enabled when VHE is built in the kernel and present into CPU implementation so only VHE code paths are modified.

When we schedule a vcpu, we disable guest usage of pointer authentication instructions and accesses to the keys. While these are disabled, we avoid context-switching the keys. When we trap the guest trying to use pointer authentication functionality, we change to eagerly context-switching the keys, and enable the feature. The next time the vcpu is scheduled out/in, we start again.

Quoted 3 lines

However the host key registers
are saved in vcpu load stage as they remain constant for each vcpu schedule.

(I think we can get away with doing this later ... with the hope of doing it never!)

Quoted 20 lines

Pointer authentication consists of address authentication and generic authentication, and CPUs in a system might have varied support for either. Where support for either feature is not uniform, it is hidden from guests via ID register emulation, as a result of the cpufeature framework in the host.

Unfortunately, address authentication and generic authentication cannot be trapped separately, as the architecture provides a single EL2 trap covering both. If we wish to expose one without the other, we cannot prevent a (badly-written) guest from intermittently using a feature which is not uniformly supported (when scheduled on a physical CPU which supports the relevant feature). Hence, this patch expects both type of authentication to be present in a cpu.

arch/arm64/include/asm/kvm_host.h +0 -0
index 2f1bb86..1bacf78 100644

arch/arm64/include/asm/kvm_host.h enum vcpu_sysreg { · line 151

Quoted 6 lines
arch/arm64/include/asm/kvm_host.h +5 -0
@@ -146,6 +146,18 @@ enum vcpu_sysreg {
 
+static inline bool kvm_supports_ptrauth(void)
+{
+	return has_vhe() && system_supports_address_auth() &&
+				system_supports_generic_auth();
+}

Do we intend to support the implementation defined algorithm? I'd assumed not.

system_supports_address_auth() is defined as:

| static inline bool system_supports_address_auth(void)
| {
| 	return IS_ENABLED(CONFIG_ARM64_PTR_AUTH) &&
| 		(cpus_have_const_cap(ARM64_HAS_ADDRESS_AUTH_ARCH) ||
| 		cpus_have_const_cap(ARM64_HAS_ADDRESS_AUTH_IMP_DEF));
| }
Quoted 6 lines

So we could return true from kvm_supports_ptrauth() even if we only support the imp-def algorithm.

I think we should hide the imp-def ptrauth support as KVM hides all other imp-def features. This lets us avoid trying to migrate values that have been signed with the imp-def algorithm.

I suppose imp-def algorithm should not make any difference in migration case even if the 2 system uses different imp-def algorithm. As the LR PAC field generation happens at runtime so only things matters is key value and SP which is taken care. Also the model on which I am testing uses imp-def algorithm. Or am I missing something ?

Quoted 37 lines

I'm worried that it could include some value that we can't migrate (e.g. the SoC serial number). Does the ARM-ARM say this can't happen?

All I can find is D5.1.5 "Pointer authentication in AArch64 state" of DDI0487D.a which has this clause for the imp-def algorithm:
| For a set of arguments passed to the function, must give the same result for all PEs | that a thread of execution could migrate between.

... with KVM we've extended the scope of migration significantly.

Could we check the cpus_have_const_cap() values for the two architected algorithms directly?

Quoted 5 lines
arch/arm64/include/asm/kvm_hyp.h +0 -0
index 6e65cad..09e061a 100644

arch/arm64/include/asm/kvm_hyp.h bool __fpsimd_enabled(void); · line 161

Quoted 9 lines
arch/arm64/include/asm/kvm_hyp.h +6 -0
@@ -153,6 +153,13 @@ bool __fpsimd_enabled(void);
  void activate_traps_vhe_load(struct kvm_vcpu *vcpu);
  void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu);
 
+void __ptrauth_switch_to_guest(struct kvm_vcpu *vcpu,
+			       struct kvm_cpu_context *host_ctxt,
+			       struct kvm_cpu_context *guest_ctxt);
+void __ptrauth_switch_to_host(struct kvm_vcpu *vcpu,
+			      struct kvm_cpu_context *guest_ctxt,
+			      struct kvm_cpu_context *host_ctxt);

Why do you need the vcpu and the guest_ctxt?
Would it be possible for these to just take the vcpu, and to pull the host context from the per-cpu variable?
This would avoid any future bugs where the ctxt's are the wrong way round, taking two is unusual in KVM, but necessary here.

Quoted 2 lines

As you're calling these from asm you want the compiler to do as much of the type mangling as possible.

Yes it is possible. I will implement in my upcoming version.

Quoted 17 lines
Quoted 7 lines
arch/arm64/kernel/traps.c +0 -0
index 4e2fb87..5cac605 100644

arch/arm64/kernel/traps.c static const char *esr_class_str[] = { · line 755

Quoted 7 lines
arch/arm64/kernel/traps.c +1 -0
@@ -749,6 +749,7 @@ static const char *esr_class_str[] = {
  	[ESR_ELx_EC_CP14_LS]		= "CP14 LDC/STC",
  	[ESR_ELx_EC_FP_ASIMD]		= "ASIMD",
  	[ESR_ELx_EC_CP10_ID]		= "CP10 MRC/VMRS",
+	[ESR_ELx_EC_PAC]		= "Pointer authentication trap",
  	[ESR_ELx_EC_CP14_64]		= "CP14 MCRR/MRRC",
  	[ESR_ELx_EC_ILL]		= "PSTATE.IL",
  	[ESR_ELx_EC_SVC32]		= "SVC (AArch32)",

Is this needed? Or was it just missing from the parts already merged. (should it be a separate patch for the arch code)

Yes you are right and looks like this is a missed patch of commit aa6eece8ec5095e479. I suppose this can be posted as a separate patch.

Quoted 3 lines

It looks like KVM only prints it from kvm_handle_unknown_ec(), which would never happen as arm_exit_handlers[] has an entry for ESR_ELx_EC_PAC.

yes.

Quoted 6 lines

Is it true that the host could never take this trap either?, as it can only be taken when HCR_EL2.TGE is clear.
(breadcrumbs from the ESR_ELx definition to "Trap to EL2 of EL0 accesses to Pointer authentication instructions")

Yes most of the ptrauth instructions are treated as NOP but some instructions like PACGA and XPAC [0] are always enabled and may trap if CONFIG_ARM64_PTR_AUTH is disabled. In VHE mode this is not trapping as HCR_EL2.TGE is set. But in NVHE mode this is causing hang instead of trap if tested with HCR_EL2.API=0. Further checking on it.

[0]: DDI0487D_a_arm (page: D5-2390)

Quoted 8 lines
Quoted 8 lines
arch/arm64/kvm/hyp/entry.S +0 -0
index 675fdc1..b78cc15 100644
@@ -64,6 +64,12 @@ ENTRY(__guest_enter)
 
  	add	x18, x0, #VCPU_CONTEXT
 
Quoted 9 lines

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_enter) · line 71

Quoted 5 lines
arch/arm64/kvm/hyp/entry.S +5 -0
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_guest(vcpu, host, guest).
+	mov	x2, x18
+	bl	__ptrauth_switch_to_guest
+#endif

This calls back into C code with x18 clobbered... is that allowed?
x18 has this weird platform-register/temporary-register behaviour, that depends on the compiler. The PCS[0] has a note that 'hand-coded assembler should avoid it entirely'!

Yes agree with you that x18 may get clobbered.

Quoted 7 lines

Can we assume that compiler generated code is using it from something, and depends on that value, which means we need to preserve or save/restore it when calling into C.

The upshot? Could you use one of the callee saved registers instead of x18, then move it after your C call.

Yes using a callee register is an option.

Quoted 18 lines
Quoted 4 lines
arch/arm64/kvm/hyp/entry.S +0 -0
@@ -118,6 +124,17 @@ ENTRY(__guest_exit)
 

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_exit) · line 134

Quoted 10 lines
arch/arm64/kvm/hyp/entry.S +7 -0
  	get_host_ctxt	x2, x3
 
+#ifdef	CONFIG_ARM64_PTR_AUTH
+	// Prepare parameter for __ptrauth_switch_to_host(vcpu, guest, host).
+	// Save x0, x2 which are used later in callee saved registers.
+	mov	x19, x0
+	mov	x20, x2
+	sub	x0, x1, #VCPU_CONTEXT
 
+	ldr	x29, [x2, #CPU_XREG_OFFSET(29)]

Is this to make the stack-trace look plausible?

Quoted 7 lines

arch/arm64/kvm/hyp/entry.S ENTRY(__guest_exit) · line 137

Quoted 4 lines
arch/arm64/kvm/hyp/entry.S +4 -0
+	bl	__ptrauth_switch_to_host
+	mov	x0, x19
+	mov	x2, x20
+#endif

(ditching the host_ctxt would let you move this above get_host_ctxt and the need to preserve its result)

ok.

Quoted 9 lines
Quoted 8 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +0 -0
new file mode 100644
index 0000000..528ee6e
@@ -0,0 +1,101 @@
Quoted 7 lines

arch/arm64/kvm/hyp/ptrauth-sr.c line 18

Quoted 1 line
arch/arm64/kvm/hyp/ptrauth-sr.c new file +1 -0
+static __always_inline bool __ptrauth_is_enabled(struct kvm_vcpu *vcpu)

This __always_inline still looks weird! You said it might be needed to make it function pointer safe. If it is, could you add a comment explaining why.

(alternatives would be making it an #ifdef, disabling ptrauth for the whole file, or annotating this function too)

ok __noptrauth annotation may be better as some functions are already using it in this file.

Quoted 33 lines
Quoted 20 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +18 -0
+#define __ptrauth_save_key(regs, key)						\
+({										\
+	regs[key ## KEYLO_EL1] = read_sysreg_s(SYS_ ## key ## KEYLO_EL1);	\
+	regs[key ## KEYHI_EL1] = read_sysreg_s(SYS_ ## key ## KEYHI_EL1);	\
+})
+
+static __always_inline void __ptrauth_save_state(struct kvm_cpu_context *ctxt)
+{
+	__ptrauth_save_key(ctxt->sys_regs, APIA);
+	__ptrauth_save_key(ctxt->sys_regs, APIB);
+	__ptrauth_save_key(ctxt->sys_regs, APDA);
+	__ptrauth_save_key(ctxt->sys_regs, APDB);
+	__ptrauth_save_key(ctxt->sys_regs, APGA);
+}
+
+#define __ptrauth_restore_key(regs, key) 					\
+({										\
+	write_sysreg_s(regs[key ## KEYLO_EL1], SYS_ ## key ## KEYLO_EL1);	\

arch/arm64/kvm/hyp/ptrauth-sr.c line 51

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +10 -0
+	write_sysreg_s(regs[key ## KEYHI_EL1], SYS_ ## key ## KEYHI_EL1);	\
+})
+
+static __always_inline void __ptrauth_restore_state(struct kvm_cpu_context *ctxt)
+{
+	__ptrauth_restore_key(ctxt->sys_regs, APIA);
+	__ptrauth_restore_key(ctxt->sys_regs, APIB);
+	__ptrauth_restore_key(ctxt->sys_regs, APDA);
+	__ptrauth_restore_key(ctxt->sys_regs, APDB);
+	__ptrauth_restore_key(ctxt->sys_regs, APGA);

Are writes to these registers self synchronising? I'd assume not, as they come as a pair.

Quoted 3 lines

I think this means we need an isb() here to ensure that when restoring the host registers the next host authentication attempt uses the key we wrote here? We don't need it for the guest, so we could put it at the end of __ptrauth_switch_to_host().

yes isb() is required.

Quoted 10 lines

arch/arm64/kvm/hyp/ptrauth-sr.c line 57

Quoted 6 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +4 -0
+/**
+ * This function changes the key so assign Pointer Authentication safe
+ * GCC attribute if protected by it.
+ */

... this comment is the reminder for 'once we have host kernel ptrauth support'? could we add something to say that kernel support is when the attribute would be needed. Otherwise it reads like we're waiting for GCC support.

ok.

Quoted 16 lines

(I assume LLVM has a similar attribute ... is it exactly the same?)

Quoted 11 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +9 -0
+void __ptrauth_switch_to_guest(struct kvm_vcpu *vcpu,
+				  struct kvm_cpu_context *host_ctxt,
+				  struct kvm_cpu_context *guest_ctxt)
+{
 
+}
 
+void __ptrauth_switch_to_host(struct kvm_vcpu *vcpu,
+				 struct kvm_cpu_context *guest_ctxt,
+				 struct kvm_cpu_context *host_ctxt)
+{
Quoted 7 lines
Quoted 1 line

+}

Could you add NOKPROBE_SYMBOL(symbol_name) for these. This adds them to the kprobe blacklist as they aren't part of the __hyp_text. (and don't need to be as its VHE only). Without this, you can patch a software-breakpoint in here, which KVM won't handle as its already switched VBAR for entry to the guest.

ok.

Quoted 23 lines

Details in 7d82602909ed ("KVM: arm64: Forbid kprobing of the VHE world-switch code")

(... this turned up in a kernel version later than you based on ...)

Quoted 16 lines
arch/arm64/kvm/hyp/ptrauth-sr.c new file +16 -0
+/**
+ * kvm_arm_vcpu_ptrauth_reset - resets ptrauth for vcpu schedule
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * This function may be used to disable ptrauth and use it in a lazy context
+ * via traps. However host key registers are saved here as they dont change
+ * during host/guest switch.
+ */
+void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu)
+{
+	struct kvm_cpu_context *host_ctxt;
+
+	if (kvm_supports_ptrauth()) {
+		kvm_arm_vcpu_ptrauth_disable(vcpu);
+		host_ctxt = vcpu->arch.host_cpu_context;
Quoted 5 lines

arch/arm64/kvm/hyp/ptrauth-sr.c line 99

Quoted 1 line
arch/arm64/kvm/hyp/ptrauth-sr.c new file +1 -0
+		__ptrauth_save_state(host_ctxt);

Could you equally do this host-save in kvm_arm_vcpu_ptrauth_trap() before kvm_arm_vcpu_ptrauth_enable()? This would avoid saving the keys if the guest never gets the opportunity to change them. At the moment we do it on every vcpu_load().

ok nice suggestion. It works fine in kvm_arm_vcpu_ptrauth_trap().

Quoted 7 lines

As kvm_arm_vcpu_ptrauth_reset() isn't used as part of the world-switch, could we keep it outside the 'hyp' directory? The Makefile for that directory expects to be building the hyp text, so it disables KASAN, KCOV and friends. kvm_arm_vcpu_ptrauth_reset() is safe for all of these, and its good for it to be covered by this debug infrastructure. Could you move it to guest.c?

ok.

Quoted 13 lines
Quoted 13 lines
arch/arm64/kvm/sys_regs.c> @@ -1045,9 +1071,10 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz) renamed from arch/arm64/kvm/sys_regs.c +1 -2
index a6c9381..12529df 100644
  					 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
  					 (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
  					 (0xfUL << ID_AA64ISAR1_GPI_SHIFT);
-		if (val & ptrauth_mask)
+		if (!kvm_supports_ptrauth()) {
  			kvm_debug("ptrauth unsupported for guests, suppressing\n");
-		val &= ~ptrauth_mask;
Quoted 8 lines

arch/arm64/kvm/sys_regs.c> @@ -1045,9 +1071,10 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)

Quoted 2 lines
arch/arm64/kvm/sys_regs.c> @@ -1045,9 +1071,10 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz) renamed from arch/arm64/kvm/sys_regs.c +2 -0
+			val &= ~ptrauth_mask;
+		}

This means that debug message gets printed even on systems that don't support ptrauth in hardware. (val&ptrauth_mask) used to cut them out, now kvm_supports_ptrauth() fails if the static keys are false, and we end up printing this message.
Now that KVM supports pointer-auth, I don't think the debug message is useful, can we remove it? (it would now mean 'you didn't ask for ptrauth to be turned on')

ok.

Quoted 3 lines

Could we always mask out the imp-def bits?

I guess no as explained before.

Quoted 4 lines

This patch needs to be merged together with 4 & 5 so the user-abi is as it should be. This means the check_present/restrictions thing needs sorting so they're ready together.

ok.

Thanks,
Amit D>

Quoted 8 lines

Thanks,

James

[0] http://infocenter.arm.com/help/topic/com.arm.doc.ihi0055b/IHI0055B_aapcs64.pdf

#23 lore raw

PATCH v6 4/6 arm64/kvm: add a userspace option to enable pointer authentication

In reply to Amit Daniel Kachhap

This feature will allow the KVM guest to allow the handling of pointer authentication instructions or to treat them as undefined if not set. It uses the existing vcpu API KVM_ARM_VCPU_INIT to supply this parameter instead of creating a new API.

A new register is not created to pass this parameter via SET/GET_ONE_REG interface as just a flag (KVM_ARM_VCPU_PTRAUTH) supplied is enough to enable this feature.

Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>
---
 Documentation/arm64/pointer-authentication.txt |  9 +++++----
 Documentation/virtual/kvm/api.txt              |  4 ++++
 arch/arm64/include/asm/kvm_host.h              |  3 ++-
 arch/arm64/include/uapi/asm/kvm.h              |  1 +
 arch/arm64/kvm/handle_exit.c                   |  2 +-
 arch/arm64/kvm/hyp/ptrauth-sr.c                | 16 +++++++++++++++-
 arch/arm64/kvm/reset.c                         |  3 +++
 arch/arm64/kvm/sys_regs.c                      | 26 +++++++++++++-------------
 include/uapi/linux/kvm.h                       |  1 +
 9 files changed, 45 insertions(+), 20 deletions(-)
Documentation/arm64/pointer-authentication.txt +5 -4
index a25cd21..0529a7d 100644
@@ -82,7 +82,8 @@ pointers).
 Virtualization
 --------------
 
-Pointer authentication is not currently supported in KVM guests. KVM
-will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
-the feature will result in an UNDEFINED exception being injected into
-the guest.
+Pointer authentication is enabled in KVM guest when virtual machine is
+created by passing a flag (KVM_ARM_VCPU_PTRAUTH) requesting this feature
+to be enabled. Without this flag, pointer authentication is not enabled
+in KVM guests and attempted use of the feature will result in an UNDEFINED
+exception being injected into the guest.
Documentation/virtual/kvm/api.txt +4 -0
index 356156f..1e646fb 100644
@@ -2642,6 +2642,10 @@ Possible features:
 	  Depends on KVM_CAP_ARM_PSCI_0_2.
 	- KVM_ARM_VCPU_PMU_V3: Emulate PMUv3 for the CPU.
 	  Depends on KVM_CAP_ARM_PMU_V3.
+	- KVM_ARM_VCPU_PTRAUTH: Emulate Pointer authentication for the CPU.
+	  Depends on KVM_CAP_ARM_PTRAUTH and only on arm64 architecture. If
+	  set, then the KVM guest allows the execution of pointer authentication
+	  instructions. Otherwise, KVM treats these instructions as undefined.
 
 
 4.83 KVM_ARM_PREFERRED_TARGET
arch/arm64/include/asm/kvm_host.h +2 -1
index 1bacf78..2768a53 100644
@@ -43,7 +43,7 @@
 
 #define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
 
-#define KVM_VCPU_MAX_FEATURES 4
+#define KVM_VCPU_MAX_FEATURES 5
 
 #define KVM_REQ_SLEEP \
 	KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
@@ -451,6 +451,7 @@ static inline bool kvm_arch_requires_vhe(void)
 	return false;
 }
 
+bool kvm_arm_vcpu_ptrauth_allowed(const struct kvm_vcpu *vcpu);
 static inline bool kvm_supports_ptrauth(void)
 {
 	return has_vhe() && system_supports_address_auth() &&
arch/arm64/include/uapi/asm/kvm.h +1 -0
index 97c3478..5f82ca1 100644
@@ -102,6 +102,7 @@ struct kvm_regs {
 #define KVM_ARM_VCPU_EL1_32BIT		1 /* CPU running a 32bit VM */
 #define KVM_ARM_VCPU_PSCI_0_2		2 /* CPU uses PSCI v0.2 */
 #define KVM_ARM_VCPU_PMU_V3		3 /* Support guest PMUv3 */
+#define KVM_ARM_VCPU_PTRAUTH		4 /* VCPU uses address authentication */
 
 struct kvm_vcpu_init {
 	__u32 target;
arch/arm64/kvm/handle_exit.c +1 -1
index 7622ab3..d9f583b 100644
@@ -179,7 +179,7 @@ static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
  */
 void kvm_arm_vcpu_ptrauth_trap(struct kvm_vcpu *vcpu)
 {
-	if (kvm_supports_ptrauth())
+	if (kvm_arm_vcpu_ptrauth_allowed(vcpu))
 		kvm_arm_vcpu_ptrauth_enable(vcpu);
 	else
 		kvm_inject_undefined(vcpu);
arch/arm64/kvm/hyp/ptrauth-sr.c +15 -1
index 528ee6e..6846a23 100644
@@ -93,9 +93,23 @@ void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu)
 {
 	struct kvm_cpu_context *host_ctxt;
 
-	if (kvm_supports_ptrauth()) {
+	if (kvm_arm_vcpu_ptrauth_allowed(vcpu)) {
 		kvm_arm_vcpu_ptrauth_disable(vcpu);
 		host_ctxt = vcpu->arch.host_cpu_context;
 		__ptrauth_save_state(host_ctxt);
 	}
 }
+
+/**
+ * kvm_arm_vcpu_ptrauth_allowed - checks if ptrauth feature is allowed by user
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * This function will be used to check userspace option to have ptrauth or not
+ * in the guest kernel.
+ */
+bool kvm_arm_vcpu_ptrauth_allowed(const struct kvm_vcpu *vcpu)
+{
+	return kvm_supports_ptrauth() &&
+		test_bit(KVM_ARM_VCPU_PTRAUTH, vcpu->arch.features);
+}
arch/arm64/kvm/reset.c +3 -0
index b72a3dd..987e0c3c 100644
@@ -91,6 +91,9 @@ int kvm_arch_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_ARM_VM_IPA_SIZE:
 		r = kvm_ipa_limit;
 		break;
+	case KVM_CAP_ARM_PTRAUTH:
+		r = kvm_supports_ptrauth();
+		break;
 	default:
 		r = 0;
 	}
arch/arm64/kvm/sys_regs.c +13 -13
index 12529df..f7bcc60 100644
@@ -1055,7 +1055,7 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
 }
 
 /* Read a sanitised cpufeature ID register by sys_reg_desc */
-static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
+static u64 read_id_reg(struct kvm_vcpu *vcpu, struct sys_reg_desc const *r, bool raz)
 {
 	u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
 			 (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
@@ -1071,7 +1071,7 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
 					 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPI_SHIFT);
-		if (!kvm_supports_ptrauth()) {
+		if (!kvm_arm_vcpu_ptrauth_allowed(vcpu)) {
 			kvm_debug("ptrauth unsupported for guests, suppressing\n");
 			val &= ~ptrauth_mask;
 		}
@@ -1095,7 +1095,7 @@ static bool __access_id_reg(struct kvm_vcpu *vcpu,
 	if (p->is_write)
 		return write_to_read_only(vcpu, p, r);
 
-	p->regval = read_id_reg(r, raz);
+	p->regval = read_id_reg(vcpu, r, raz);
 	return true;
 }
 
@@ -1124,17 +1124,17 @@ static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
  * are stored, and for set_id_reg() we don't allow the effective value
  * to be changed.
  */
-static int __get_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
-			bool raz)
+static int __get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+			void __user *uaddr, bool raz)
 {
 	const u64 id = sys_reg_to_index(rd);
-	const u64 val = read_id_reg(rd, raz);
+	const u64 val = read_id_reg(vcpu, rd, raz);
 
 	return reg_to_user(uaddr, &val, id);
 }
 
-static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
-			bool raz)
+static int __set_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+			void __user *uaddr, bool raz)
 {
 	const u64 id = sys_reg_to_index(rd);
 	int err;
@@ -1145,7 +1145,7 @@ static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
 		return err;
 
 	/* This is what we mean by invariant: you can't change it. */
-	if (val != read_id_reg(rd, raz))
+	if (val != read_id_reg(vcpu, rd, raz))
 		return -EINVAL;
 
 	return 0;
@@ -1154,25 +1154,25 @@ static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
 static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
 		      const struct kvm_one_reg *reg, void __user *uaddr)
 {
-	return __get_id_reg(rd, uaddr, false);
+	return __get_id_reg(vcpu, rd, uaddr, false);
 }
 
 static int set_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
 		      const struct kvm_one_reg *reg, void __user *uaddr)
 {
-	return __set_id_reg(rd, uaddr, false);
+	return __set_id_reg(vcpu, rd, uaddr, false);
 }
 
 static int get_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
 			  const struct kvm_one_reg *reg, void __user *uaddr)
 {
-	return __get_id_reg(rd, uaddr, true);
+	return __get_id_reg(vcpu, rd, uaddr, true);
 }
 
 static int set_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
 			  const struct kvm_one_reg *reg, void __user *uaddr)
 {
-	return __set_id_reg(rd, uaddr, true);
+	return __set_id_reg(vcpu, rd, uaddr, true);
 }
 
 /* sys_reg_desc initialiser for known cpufeature ID registers */
include/uapi/linux/kvm.h +1 -0
index 6d4ea4b..a553477 100644
@@ -988,6 +988,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_ARM_VM_IPA_SIZE 165
 #define KVM_CAP_MANUAL_DIRTY_LOG_PROTECT 166
 #define KVM_CAP_HYPERV_CPUID 167
+#define KVM_CAP_ARM_PTRAUTH 168
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
Signature · 1 line
2.7.4
reviewer
#24 lore raw

Re: [PATCH v6 4/6] arm64/kvm: add a userspace option to enable pointer authentication

Quoted 48 lines · Amit Daniel Kachhap

This feature will allow the KVM guest to allow the handling of pointer authentication instructions or to treat them as undefined if not set. It uses the existing vcpu API KVM_ARM_VCPU_INIT to supply this parameter instead of creating a new API.

A new register is not created to pass this parameter via SET/GET_ONE_REG interface as just a flag (KVM_ARM_VCPU_PTRAUTH) supplied is enough to enable this feature.

Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>
---
 Documentation/arm64/pointer-authentication.txt |  9 +++++----
 Documentation/virtual/kvm/api.txt              |  4 ++++
 arch/arm64/include/asm/kvm_host.h              |  3 ++-
 arch/arm64/include/uapi/asm/kvm.h              |  1 +
 arch/arm64/kvm/handle_exit.c                   |  2 +-
 arch/arm64/kvm/hyp/ptrauth-sr.c                | 16 +++++++++++++++-
 arch/arm64/kvm/reset.c                         |  3 +++
 arch/arm64/kvm/sys_regs.c                      | 26 +++++++++++++-------------
 include/uapi/linux/kvm.h                       |  1 +
 9 files changed, 45 insertions(+), 20 deletions(-)
Documentation/arm64/pointer-authentication.txt +5 -4
index a25cd21..0529a7d 100644
@@ -82,7 +82,8 @@ pointers).
 Virtualization
 --------------
 
-Pointer authentication is not currently supported in KVM guests. KVM
-will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
-the feature will result in an UNDEFINED exception being injected into
-the guest.
+Pointer authentication is enabled in KVM guest when virtual machine is
+created by passing a flag (KVM_ARM_VCPU_PTRAUTH) requesting this feature
+to be enabled. Without this flag, pointer authentication is not enabled
+in KVM guests and attempted use of the feature will result in an UNDEFINED
+exception being injected into the guest.
Documentation/virtual/kvm/api.txt +0 -0
index 356156f..1e646fb 100644

Documentation/virtual/kvm/api.txt Possible features: · line 2648

Quoted 7 lines
Documentation/virtual/kvm/api.txt +4 -0
@@ -2642,6 +2642,10 @@ Possible features:
 	  Depends on KVM_CAP_ARM_PSCI_0_2.
 	- KVM_ARM_VCPU_PMU_V3: Emulate PMUv3 for the CPU.
 	  Depends on KVM_CAP_ARM_PMU_V3.
+	- KVM_ARM_VCPU_PTRAUTH: Emulate Pointer authentication for the CPU.
+	  Depends on KVM_CAP_ARM_PTRAUTH and only on arm64 architecture. If
+	  set, then the KVM guest allows the execution of pointer authentication
+	  instructions. Otherwise, KVM treats these instructions as undefined.

I think that we should have separate flags for address auth and generic auth, to match the ID register split.

For now, we can have KVM only support the case where both are set, but it gives us freedom to support either in isolation if we have to in future, without an ABI break.

Thanks,
Mark.

#25 lore raw

Re: [PATCH v6 4/6] arm64/kvm: add a userspace option to enable pointer authentication

Hi,

Quoted 60 lines · Mark Rutland
Quoted 48 lines · Amit Daniel Kachhap

This feature will allow the KVM guest to allow the handling of pointer authentication instructions or to treat them as undefined if not set. It uses the existing vcpu API KVM_ARM_VCPU_INIT to supply this parameter instead of creating a new API.

A new register is not created to pass this parameter via SET/GET_ONE_REG interface as just a flag (KVM_ARM_VCPU_PTRAUTH) supplied is enough to enable this feature.

Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>
---
  Documentation/arm64/pointer-authentication.txt |  9 +++++----
  Documentation/virtual/kvm/api.txt              |  4 ++++
  arch/arm64/include/asm/kvm_host.h              |  3 ++-
  arch/arm64/include/uapi/asm/kvm.h              |  1 +
  arch/arm64/kvm/handle_exit.c                   |  2 +-
  arch/arm64/kvm/hyp/ptrauth-sr.c                | 16 +++++++++++++++-
  arch/arm64/kvm/reset.c                         |  3 +++
  arch/arm64/kvm/sys_regs.c                      | 26 +++++++++++++-------------
  include/uapi/linux/kvm.h                       |  1 +
  9 files changed, 45 insertions(+), 20 deletions(-)
Documentation/arm64/pointer-authentication.txt +5 -4
index a25cd21..0529a7d 100644
@@ -82,7 +82,8 @@ pointers).
  Virtualization
  --------------
 
-Pointer authentication is not currently supported in KVM guests. KVM
-will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
-the feature will result in an UNDEFINED exception being injected into
-the guest.
+Pointer authentication is enabled in KVM guest when virtual machine is
+created by passing a flag (KVM_ARM_VCPU_PTRAUTH) requesting this feature
+to be enabled. Without this flag, pointer authentication is not enabled
+in KVM guests and attempted use of the feature will result in an UNDEFINED
+exception being injected into the guest.
Documentation/virtual/kvm/api.txt +0 -0
index 356156f..1e646fb 100644

Documentation/virtual/kvm/api.txt Possible features: · line 2648

Quoted 7 lines
Documentation/virtual/kvm/api.txt +4 -0
@@ -2642,6 +2642,10 @@ Possible features:
  	  Depends on KVM_CAP_ARM_PSCI_0_2.
  	- KVM_ARM_VCPU_PMU_V3: Emulate PMUv3 for the CPU.
  	  Depends on KVM_CAP_ARM_PMU_V3.
+	- KVM_ARM_VCPU_PTRAUTH: Emulate Pointer authentication for the CPU.
+	  Depends on KVM_CAP_ARM_PTRAUTH and only on arm64 architecture. If
+	  set, then the KVM guest allows the execution of pointer authentication
+	  instructions. Otherwise, KVM treats these instructions as undefined.

I think that we should have separate flags for address auth and generic auth, to match the ID register split.

Quoted 3 lines

For now, we can have KVM only support the case where both are set, but it gives us freedom to support either in isolation if we have to in future, without an ABI break.

yes agree with you about having two address and generic ptrauth flags. Will add in next iteration.

Thanks,
Amit D

Quoted 4 lines

Thanks,
Mark.

#26 lore raw

Re: [PATCH v6 4/6] arm64/kvm: add a userspace option to enable pointer authentication

In reply to Amit Daniel Kachhap
Quoted 26 lines · Amit Daniel Kachhap

This feature will allow the KVM guest to allow the handling of pointer authentication instructions or to treat them as undefined if not set. It uses the existing vcpu API KVM_ARM_VCPU_INIT to supply this parameter instead of creating a new API.

A new register is not created to pass this parameter via SET/GET_ONE_REG interface as just a flag (KVM_ARM_VCPU_PTRAUTH) supplied is enough to enable this feature.

Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>
---
 Documentation/arm64/pointer-authentication.txt |  9 +++++----
 Documentation/virtual/kvm/api.txt              |  4 ++++
 arch/arm64/include/asm/kvm_host.h              |  3 ++-
 arch/arm64/include/uapi/asm/kvm.h              |  1 +
 arch/arm64/kvm/handle_exit.c                   |  2 +-
 arch/arm64/kvm/hyp/ptrauth-sr.c                | 16 +++++++++++++++-
 arch/arm64/kvm/reset.c                         |  3 +++
 arch/arm64/kvm/sys_regs.c                      | 26 +++++++++++++-------------
 include/uapi/linux/kvm.h                       |  1 +
 9 files changed, 45 insertions(+), 20 deletions(-)

[...]

Quoted 21 lines
arch/arm64/include/asm/kvm_host.h +2 -1
index 1bacf78..2768a53 100644
@@ -43,7 +43,7 @@
 
 #define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
 
-#define KVM_VCPU_MAX_FEATURES 4
+#define KVM_VCPU_MAX_FEATURES 5
 
 #define KVM_REQ_SLEEP \
 	KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
@@ -451,6 +451,7 @@ static inline bool kvm_arch_requires_vhe(void)
 	return false;
 }
 
+bool kvm_arm_vcpu_ptrauth_allowed(const struct kvm_vcpu *vcpu);
 static inline bool kvm_supports_ptrauth(void)
 {
 	return has_vhe() && system_supports_address_auth() &&

[...]

Quoted 5 lines
arch/arm64/kvm/hyp/ptrauth-sr.c +0 -0
index 528ee6e..6846a23 100644
@@ -93,9 +93,23 @@ void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu)

[...]

arch/arm64/kvm/hyp/ptrauth-sr.c void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu) · line 115

Quoted 13 lines
arch/arm64/kvm/hyp/ptrauth-sr.c +13 -0
+/**
+ * kvm_arm_vcpu_ptrauth_allowed - checks if ptrauth feature is allowed by user
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * This function will be used to check userspace option to have ptrauth or not
+ * in the guest kernel.
+ */
+bool kvm_arm_vcpu_ptrauth_allowed(const struct kvm_vcpu *vcpu)
+{
+	return kvm_supports_ptrauth() &&
+		test_bit(KVM_ARM_VCPU_PTRAUTH, vcpu->arch.features);
+}

Nit: for SVE is called the equivalent helper vcpu_has_sve(vcpu).

Neither naming is more correct, but it would make sense to be consistent. We will likely accumulate more of these vcpu feature predicates over time.

Given that this is trivial and will be used all over the place, it probably makes sense to define it in kvm_host.h rather than having it out of line in a separate C file.

Quoted 37 lines
arch/arm64/kvm/reset.c +3 -0
index b72a3dd..987e0c3c 100644
@@ -91,6 +91,9 @@ int kvm_arch_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_ARM_VM_IPA_SIZE:
 		r = kvm_ipa_limit;
 		break;
+	case KVM_CAP_ARM_PTRAUTH:
+		r = kvm_supports_ptrauth();
+		break;
 	default:
 		r = 0;
 	}
arch/arm64/kvm/sys_regs.c +2 -2
index 12529df..f7bcc60 100644
@@ -1055,7 +1055,7 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
 }
 
 /* Read a sanitised cpufeature ID register by sys_reg_desc */
-static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
+static u64 read_id_reg(struct kvm_vcpu *vcpu, struct sys_reg_desc const *r, bool raz)
 {
 	u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
 			 (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
@@ -1071,7 +1071,7 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
 					 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
 					 (0xfUL << ID_AA64ISAR1_GPI_SHIFT);
-		if (!kvm_supports_ptrauth()) {
+		if (!kvm_arm_vcpu_ptrauth_allowed(vcpu)) {
 			kvm_debug("ptrauth unsupported for guests, suppressing\n");
 			val &= ~ptrauth_mask;
 		}

arch/arm64/kvm/sys_regs.c static bool __access_id_reg(struct kvm_vcpu *vcpu, · line 1100

Quoted 7 lines
arch/arm64/kvm/sys_regs.c +1 -1
@@ -1095,7 +1095,7 @@ static bool __access_id_reg(struct kvm_vcpu *vcpu,
 	if (p->is_write)
 		return write_to_read_only(vcpu, p, r);
 
-	p->regval = read_id_reg(r, raz);
+	p->regval = read_id_reg(vcpu, r, raz);
 	return true;
 }

The SVE KVM series makes various overlapping changes to propagate vcpuo into the relevant places, but hopefully the rebase is not too painful. Many of the changes are probably virtually identical between the two series.

See for example [1]. Maybe you could cherry-pick and drop the equivalent changes here (though if your series is picked up first, I will live with it ;)

[...]

Cheers
---Dave

[1] [PATCH v5 10/26] KVM: arm64: Propagate vcpu into read_id_reg()
https://lists.cs.columbia.edu/pipermail/kvmarm/2019-February/034687.html

#27 lore raw

Re: [PATCH v6 4/6] arm64/kvm: add a userspace option to enable pointer authentication

Hi,

Quoted 78 lines · Dave Martin
Quoted 26 lines · Amit Daniel Kachhap

This feature will allow the KVM guest to allow the handling of pointer authentication instructions or to treat them as undefined if not set. It uses the existing vcpu API KVM_ARM_VCPU_INIT to supply this parameter instead of creating a new API.

A new register is not created to pass this parameter via SET/GET_ONE_REG interface as just a flag (KVM_ARM_VCPU_PTRAUTH) supplied is enough to enable this feature.

Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>
---
  Documentation/arm64/pointer-authentication.txt |  9 +++++----
  Documentation/virtual/kvm/api.txt              |  4 ++++
  arch/arm64/include/asm/kvm_host.h              |  3 ++-
  arch/arm64/include/uapi/asm/kvm.h              |  1 +
  arch/arm64/kvm/handle_exit.c                   |  2 +-
  arch/arm64/kvm/hyp/ptrauth-sr.c                | 16 +++++++++++++++-
  arch/arm64/kvm/reset.c                         |  3 +++
  arch/arm64/kvm/sys_regs.c                      | 26 +++++++++++++-------------
  include/uapi/linux/kvm.h                       |  1 +
  9 files changed, 45 insertions(+), 20 deletions(-)

[...]

Quoted 21 lines
arch/arm64/include/asm/kvm_host.h +2 -1
index 1bacf78..2768a53 100644
@@ -43,7 +43,7 @@
 
  #define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
 
-#define KVM_VCPU_MAX_FEATURES 4
+#define KVM_VCPU_MAX_FEATURES 5
 
  #define KVM_REQ_SLEEP \
  	KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
@@ -451,6 +451,7 @@ static inline bool kvm_arch_requires_vhe(void)
  	return false;
  }
 
+bool kvm_arm_vcpu_ptrauth_allowed(const struct kvm_vcpu *vcpu);
  static inline bool kvm_supports_ptrauth(void)
  {
  	return has_vhe() && system_supports_address_auth() &&

[...]

Quoted 5 lines
arch/arm64/kvm/hyp/ptrauth-sr.c +0 -0
index 528ee6e..6846a23 100644
@@ -93,9 +93,23 @@ void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu)

[...]

arch/arm64/kvm/hyp/ptrauth-sr.c void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu) · line 115

Quoted 13 lines
arch/arm64/kvm/hyp/ptrauth-sr.c +13 -0
+/**
+ * kvm_arm_vcpu_ptrauth_allowed - checks if ptrauth feature is allowed by user
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * This function will be used to check userspace option to have ptrauth or not
+ * in the guest kernel.
+ */
+bool kvm_arm_vcpu_ptrauth_allowed(const struct kvm_vcpu *vcpu)
+{
+	return kvm_supports_ptrauth() &&
+		test_bit(KVM_ARM_VCPU_PTRAUTH, vcpu->arch.features);
+}

Nit: for SVE is called the equivalent helper vcpu_has_sve(vcpu).

Quoted 7 lines

Neither naming is more correct, but it would make sense to be consistent. We will likely accumulate more of these vcpu feature predicates over time.

Given that this is trivial and will be used all over the place, it probably makes sense to define it in kvm_host.h rather than having it out of line in a separate C file.

Ok I checked the SVE implementation. vcpu_has_ptrauth macro make more sense.

Quoted 51 lines
Quoted 38 lines
arch/arm64/kvm/reset.c +3 -0
index b72a3dd..987e0c3c 100644
@@ -91,6 +91,9 @@ int kvm_arch_vm_ioctl_check_extension(struct kvm *kvm, long ext)
  	case KVM_CAP_ARM_VM_IPA_SIZE:
  		r = kvm_ipa_limit;
  		break;
+	case KVM_CAP_ARM_PTRAUTH:
+		r = kvm_supports_ptrauth();
+		break;
  	default:
  		r = 0;
  	}
arch/arm64/kvm/sys_regs.c +2 -2
index 12529df..f7bcc60 100644
@@ -1055,7 +1055,7 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
  }
 
  /* Read a sanitised cpufeature ID register by sys_reg_desc */
-static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
+static u64 read_id_reg(struct kvm_vcpu *vcpu, struct sys_reg_desc const *r, bool raz)
  {
  	u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
  			 (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
@@ -1071,7 +1071,7 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
  					 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
  					 (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
  					 (0xfUL << ID_AA64ISAR1_GPI_SHIFT);
-		if (!kvm_supports_ptrauth()) {
+		if (!kvm_arm_vcpu_ptrauth_allowed(vcpu)) {
  			kvm_debug("ptrauth unsupported for guests, suppressing\n");
  			val &= ~ptrauth_mask;
  		}

arch/arm64/kvm/sys_regs.c static bool __access_id_reg(struct kvm_vcpu *vcpu, · line 1100

Quoted 7 lines
arch/arm64/kvm/sys_regs.c +1 -1
@@ -1095,7 +1095,7 @@ static bool __access_id_reg(struct kvm_vcpu *vcpu,
  	if (p->is_write)
  		return write_to_read_only(vcpu, p, r);
 
-	p->regval = read_id_reg(r, raz);
+	p->regval = read_id_reg(vcpu, r, raz);
  	return true;
  }

The SVE KVM series makes various overlapping changes to propagate vcpuo into the relevant places, but hopefully the rebase is not too painful. Many of the changes are probably virtually identical between the two series.

Quoted 3 lines

See for example [1]. Maybe you could cherry-pick and drop the equivalent changes here (though if your series is picked up first, I will live with it ;)

Yes no issue. I will cherry-pick your specific patch and rebase mine on it.

Thanks,
Amit D

Quoted 10 lines

[...]

Cheers
---Dave

[1] [PATCH v5 10/26] KVM: arm64: Propagate vcpu into read_id_reg()
https://lists.cs.columbia.edu/pipermail/kvmarm/2019-February/034687.html

#28 lore raw

Re: [PATCH v6 4/6] arm64/kvm: add a userspace option to enable pointer authentication

Quoted 18 lines · Amit Daniel Kachhap

Hi,

Quoted 15 lines · Dave Martin
Quoted 14 lines · Amit Daniel Kachhap

This feature will allow the KVM guest to allow the handling of pointer authentication instructions or to treat them as undefined if not set. It uses the existing vcpu API KVM_ARM_VCPU_INIT to supply this parameter instead of creating a new API.

A new register is not created to pass this parameter via SET/GET_ONE_REG interface as just a flag (KVM_ARM_VCPU_PTRAUTH) supplied is enough to enable this feature.

Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>

[...]

Quoted 36 lines
Quoted 36 lines
Quoted 23 lines
arch/arm64/kvm/sys_regs.c +2 -2
index 12529df..f7bcc60 100644
@@ -1055,7 +1055,7 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
  }
 
  /* Read a sanitised cpufeature ID register by sys_reg_desc */
-static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
+static u64 read_id_reg(struct kvm_vcpu *vcpu, struct sys_reg_desc const *r, bool raz)
  {
  u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
   (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
@@ -1071,7 +1071,7 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
   (0xfUL << ID_AA64ISAR1_API_SHIFT) |
   (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
   (0xfUL << ID_AA64ISAR1_GPI_SHIFT);
-if (!kvm_supports_ptrauth()) {
+if (!kvm_arm_vcpu_ptrauth_allowed(vcpu)) {
  kvm_debug("ptrauth unsupported for guests, suppressing\n");
  val &= ~ptrauth_mask;
  }

arch/arm64/kvm/sys_regs.c static bool __access_id_reg(struct kvm_vcpu *vcpu, · line 1100

Quoted 7 lines
arch/arm64/kvm/sys_regs.c +1 -1
@@ -1095,7 +1095,7 @@ static bool __access_id_reg(struct kvm_vcpu *vcpu,
  if (p->is_write)
  return write_to_read_only(vcpu, p, r);
 
-p->regval = read_id_reg(r, raz);
+p->regval = read_id_reg(vcpu, r, raz);
  return true;
  }

The SVE KVM series makes various overlapping changes to propagate vcpuo into the relevant places, but hopefully the rebase is not too painful. Many of the changes are probably virtually identical between the two series.

Quoted 4 lines
Quoted 3 lines

See for example [1]. Maybe you could cherry-pick and drop the equivalent changes here (though if your series is picked up first, I will live with it ;)

Yes no issue. I will cherry-pick your specific patch and rebase mine on it.

OK, thanks.

Unfortunately it is likely to churn a bit due to review-- my v6 series will rename some stuff. Hopefully it will be stable from then on.

Cheers
---Dave
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

#29 lore raw

Re: [PATCH v6 4/6] arm64/kvm: add a userspace option to enable pointer authentication

In reply to Amit Daniel Kachhap

Hi Amit,

Quoted 8 lines · Amit Daniel Kachhap

This feature will allow the KVM guest to allow the handling of pointer authentication instructions or to treat them as undefined if not set. It uses the existing vcpu API KVM_ARM_VCPU_INIT to supply this parameter instead of creating a new API.

A new register is not created to pass this parameter via SET/GET_ONE_REG interface as just a flag (KVM_ARM_VCPU_PTRAUTH) supplied is enough to enable this feature.

and an attempt to restore the id register with the other version would fail.

Quoted 5 lines
Documentation/arm64/pointer-authentication.txt +0 -0
index a25cd21..0529a7d 100644

Documentation/arm64/pointer-authentication.txt pointers).

Quoted 10 lines
Documentation/arm64/pointer-authentication.txt +2 -4
@@ -82,7 +82,8 @@ pointers).
 Virtualization
 --------------
 
-Pointer authentication is not currently supported in KVM guests. KVM
-will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
-the feature will result in an UNDEFINED exception being injected into
-the guest.
 
+Pointer authentication is enabled in KVM guest when virtual machine is
+created by passing a flag (KVM_ARM_VCPU_PTRAUTH)

(This is still mixing VM and VCPU)

Quoted 1 line

+ requesting this feature to be enabled.

.. on each vcpu?

Documentation/arm64/pointer-authentication.txt pointers). · line 89

Quoted 3 lines
Documentation/arm64/pointer-authentication.txt +3 -0
+Without this flag, pointer authentication is not enabled
+in KVM guests and attempted use of the feature will result in an UNDEFINED
+exception being injected into the guest.

'guests' here suggests its a VM property. If you set it on some VCPU but not others KVM will generate undefs instead of enabling the feature. (which is the right thing to do)

I think it needs to be clear this is a per-vcpu property.

arch/arm64/include/uapi/asm/kvm.h struct kvm_regs { · line 106

Quoted 10 lines
arch/arm64/include/uapi/asm/kvm.h +1 -0
index 97c3478..5f82ca1 100644
@@ -102,6 +102,7 @@ struct kvm_regs {
 #define KVM_ARM_VCPU_EL1_32BIT		1 /* CPU running a 32bit VM */
 #define KVM_ARM_VCPU_PSCI_0_2		2 /* CPU uses PSCI v0.2 */
 #define KVM_ARM_VCPU_PMU_V3		3 /* Support guest PMUv3 */
 
+#define KVM_ARM_VCPU_PTRAUTH		4 /* VCPU uses address authentication */

Just address authentication? I agree with Mark we should have two bits to match what gets exposed to EL0. One would then be address, the other generic.

Quoted 9 lines
arch/arm64/kvm/hyp/ptrauth-sr.c +3 -0
index 528ee6e..6846a23 100644
@@ -93,9 +93,23 @@ void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu)
 
+/**
+ * kvm_arm_vcpu_ptrauth_allowed - checks if ptrauth feature is allowed by user
+ *

arch/arm64/kvm/hyp/ptrauth-sr.c void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu) · line 106

Quoted 10 lines
arch/arm64/kvm/hyp/ptrauth-sr.c +10 -0
+ * @vcpu: The VCPU pointer
+ *
+ * This function will be used to check userspace option to have ptrauth or not
+ * in the guest kernel.
+ */
+bool kvm_arm_vcpu_ptrauth_allowed(const struct kvm_vcpu *vcpu)
+{
+	return kvm_supports_ptrauth() &&
+		test_bit(KVM_ARM_VCPU_PTRAUTH, vcpu->arch.features);
+}

This isn't used from world-switch, could it be moved to guest.c?

arch/arm64/kvm/sys_regs.c static bool access_cntp_cval(struct kvm_vcpu *vcpu, · line 1058

Quoted 10 lines
arch/arm64/kvm/sys_regs.c +1 -1
index 12529df..f7bcc60 100644
@@ -1055,7 +1055,7 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
 }
 
 /* Read a sanitised cpufeature ID register by sys_reg_desc */
-static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
+static u64 read_id_reg(struct kvm_vcpu *vcpu, struct sys_reg_desc const *r, bool raz)

(It might be easier on the reviewer to move these mechanical changes to an earlier patch)

Looks good,

Thanks,

James

#30 lore raw

Re: [PATCH v6 4/6] arm64/kvm: add a userspace option to enable pointer authentication

Hi James,

Quoted 39 lines · James Morse

Hi Amit,

Quoted 8 lines · Amit Daniel Kachhap

This feature will allow the KVM guest to allow the handling of pointer authentication instructions or to treat them as undefined if not set. It uses the existing vcpu API KVM_ARM_VCPU_INIT to supply this parameter instead of creating a new API.

A new register is not created to pass this parameter via SET/GET_ONE_REG interface as just a flag (KVM_ARM_VCPU_PTRAUTH) supplied is enough to enable this feature.

and an attempt to restore the id register with the other version would fail.

Quoted 5 lines
Documentation/arm64/pointer-authentication.txt +0 -0
index a25cd21..0529a7d 100644

Documentation/arm64/pointer-authentication.txt pointers).

Quoted 10 lines
Documentation/arm64/pointer-authentication.txt +2 -4
@@ -82,7 +82,8 @@ pointers).
  Virtualization
  --------------
 
-Pointer authentication is not currently supported in KVM guests. KVM
-will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
-the feature will result in an UNDEFINED exception being injected into
-the guest.
 
+Pointer authentication is enabled in KVM guest when virtual machine is
+created by passing a flag (KVM_ARM_VCPU_PTRAUTH)

(This is still mixing VM and VCPU)

Quoted 1 line

+ requesting this feature to be enabled.

.. on each vcpu?

Quoted 8 lines

Documentation/arm64/pointer-authentication.txt pointers). · line 89

Quoted 3 lines
Documentation/arm64/pointer-authentication.txt +3 -0
+Without this flag, pointer authentication is not enabled
+in KVM guests and attempted use of the feature will result in an UNDEFINED
+exception being injected into the guest.

'guests' here suggests its a VM property. If you set it on some VCPU but not others KVM will generate undefs instead of enabling the feature. (which is the right thing to do)

I think it needs to be clear this is a per-vcpu property.

ok.

Quoted 11 lines
Quoted 10 lines
arch/arm64/include/uapi/asm/kvm.h +0 -0
index 97c3478..5f82ca1 100644
@@ -102,6 +102,7 @@ struct kvm_regs {
  #define KVM_ARM_VCPU_EL1_32BIT		1 /* CPU running a 32bit VM */
  #define KVM_ARM_VCPU_PSCI_0_2		2 /* CPU uses PSCI v0.2 */
  #define KVM_ARM_VCPU_PMU_V3		3 /* Support guest PMUv3 */
Quoted 4 lines

arch/arm64/include/uapi/asm/kvm.h struct kvm_regs { · line 105

Quoted 1 line
arch/arm64/include/uapi/asm/kvm.h +1 -0
+#define KVM_ARM_VCPU_PTRAUTH		4 /* VCPU uses address authentication */

Just address authentication? I agree with Mark we should have two bits to match what gets exposed to EL0. One would then be address, the other generic.

ok.

Quoted 8 lines
Quoted 7 lines
arch/arm64/kvm/hyp/ptrauth-sr.c +0 -0
index 528ee6e..6846a23 100644
@@ -93,9 +93,23 @@ void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu)
Quoted 15 lines

arch/arm64/kvm/hyp/ptrauth-sr.c void kvm_arm_vcpu_ptrauth_reset(struct kvm_vcpu *vcpu) · line 115

Quoted 13 lines
arch/arm64/kvm/hyp/ptrauth-sr.c +13 -0
+/**
+ * kvm_arm_vcpu_ptrauth_allowed - checks if ptrauth feature is allowed by user
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * This function will be used to check userspace option to have ptrauth or not
+ * in the guest kernel.
+ */
+bool kvm_arm_vcpu_ptrauth_allowed(const struct kvm_vcpu *vcpu)
+{
+	return kvm_supports_ptrauth() &&
+		test_bit(KVM_ARM_VCPU_PTRAUTH, vcpu->arch.features);
+}

This isn't used from world-switch, could it be moved to guest.c?

yes sure.

Quoted 9 lines
Quoted 9 lines
arch/arm64/kvm/sys_regs.c +0 -0
index 12529df..f7bcc60 100644
@@ -1055,7 +1055,7 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
  }
 
Quoted 5 lines

arch/arm64/kvm/sys_regs.c static bool access_cntp_cval(struct kvm_vcpu *vcpu, · line 1058

Quoted 3 lines
arch/arm64/kvm/sys_regs.c +1 -1
  /* Read a sanitised cpufeature ID register by sys_reg_desc */
-static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
+static u64 read_id_reg(struct kvm_vcpu *vcpu, struct sys_reg_desc const *r, bool raz)

(It might be easier on the reviewer to move these mechanical changes to an earlier patch)

Yes with including some of Dave SVE patches this wont be required.

Thanks,
Amit D

Quoted 8 lines

Looks good,

Thanks,

James

#31 lore raw

PATCH v6 5/6 arm64/kvm: control accessibility of ptrauth key registers

In reply to Amit Daniel Kachhap

According to userspace settings, ptrauth key registers are conditionally present in guest system register list based on user specified flag KVM_ARM_VCPU_PTRAUTH.

Reset routines still sets these registers to default values but they are left like that as they are conditionally accessible (set/get).

Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>

---
This patch needs patch [1] by Dave Martin and adds feature to manage accessibility in a scalable way.

[1]: https://lore.kernel.org/linux-arm-kernel/[email protected]/

 Documentation/arm64/pointer-authentication.txt | 4 ++++
 arch/arm64/kvm/sys_regs.c                      | 7 ++++++-
 2 files changed, 10 insertions(+), 1 deletion(-)
Documentation/arm64/pointer-authentication.txt +4 -0
index 0529a7d..996e435 100644
@@ -87,3 +87,7 @@ created by passing a flag (KVM_ARM_VCPU_PTRAUTH) requesting this feature
 to be enabled. Without this flag, pointer authentication is not enabled
 in KVM guests and attempted use of the feature will result in an UNDEFINED
 exception being injected into the guest.
+
+Additionally, when KVM_ARM_VCPU_PTRAUTH is not set then KVM will filter
+out the Pointer Authentication system key registers from KVM_GET/SET_REG_*
+ioctls.
arch/arm64/kvm/sys_regs.c +6 -1
index f7bcc60..c2f4974 100644
@@ -1005,8 +1005,13 @@ static bool trap_ptrauth(struct kvm_vcpu *vcpu,
 	return false;
 }
 
+static bool check_ptrauth(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
+{
+	return kvm_arm_vcpu_ptrauth_allowed(vcpu);
+}
+
 #define __PTRAUTH_KEY(k)						\
-	{ SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k }
+	{ SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k , .check_present = check_ptrauth}
 
 #define PTRAUTH_KEY(k)							\
 	__PTRAUTH_KEY(k ## KEYLO_EL1),					\
Signature · 1 line
2.7.4
#32 lore raw

Re: [PATCH v6 5/6] arm64/kvm: control accessibility of ptrauth key registers

Quoted 12 lines · Amit Daniel Kachhap

According to userspace settings, ptrauth key registers are conditionally present in guest system register list based on user specified flag KVM_ARM_VCPU_PTRAUTH.

Reset routines still sets these registers to default values but they are left like that as they are conditionally accessible (set/get).

Cc Mark Rutland <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Christoffer Dall <[email protected]>
Quoted 4 lines

---
This patch needs patch [1] by Dave Martin and adds feature to manage accessibility in a scalable way.

[1]: https://lore.kernel.org/linux-arm-kernel/[email protected]/

FYI, check_present() has changed a bit in the SVE v5 series [2].

The precise interface is still under discussion, so please take a look and feel free to comment.

You'll probably need to tweak some things so that the KVM_GET_REG_LIST output is consistent with the set of regs that do/don't yield -ENOENT in KVM_GET_ONE_REG/KVM_SET_ONE_REG.

See other patches in the series for examples of how I use the modified interface.

[...]

Quoted 6 lines
arch/arm64/kvm/sys_regs.c +0 -0
index f7bcc60..c2f4974 100644
@@ -1005,8 +1005,13 @@ static bool trap_ptrauth(struct kvm_vcpu *vcpu,
 	return false;

arch/arm64/kvm/sys_regs.c static bool trap_ptrauth(struct kvm_vcpu *vcpu, · line 1014

Quoted 10 lines
arch/arm64/kvm/sys_regs.c +6 -1
 }
 
+static bool check_ptrauth(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
+{
+	return kvm_arm_vcpu_ptrauth_allowed(vcpu);
+}
+
 #define __PTRAUTH_KEY(k)						\
-	{ SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k }
+	{ SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k , .check_present = check_ptrauth}

Cheers
---Dave

[2] [PATCH v5 12/26] KVM: arm64: Support runtime sysreg visibility filtering
https://lists.cs.columbia.edu/pipermail/kvmarm/2019-February/034671.html

#33 lore raw

Re: [PATCH v6 5/6] arm64/kvm: control accessibility of ptrauth key registers

In reply to Amit Daniel Kachhap

Hi Amit,

Quoted 6 lines · Amit Daniel Kachhap

According to userspace settings, ptrauth key registers are conditionally present in guest system register list based on user specified flag KVM_ARM_VCPU_PTRAUTH.

Reset routines still sets these registers to default values but they are left like that as they are conditionally accessible (set/get).

What problem is this patch solving?

I think it's that now we have ptrauth support, we have a glut of new registers visible to user-space. This breaks migration and save/resume if the target machine doesn't have pointer-auth configured, even if the guest wasn't using it.
Because we've got a VCPU bit, we can be smarter about this, and only expose the registers if user-space was able to enable ptrauth.

Quoted 4 lines

---
This patch needs patch [1] by Dave Martin and adds feature to manage accessibility in a scalable way.

[1]: https://lore.kernel.org/linux-arm-kernel/[email protected]/

This is v4. Shortly before you posted this there was a v5 (but the subject changed, easily missed). V5 has subsequently been reviewed. As we can't have both, could you rebase onto that v5 so that there is one way of doing this?

(in general if you could re-post the version you develop/tested with then it makes it clear what is going on)

Quoted 18 lines
Documentation/arm64/pointer-authentication.txt +4 -0
index 0529a7d..996e435 100644
@@ -87,3 +87,7 @@ created by passing a flag (KVM_ARM_VCPU_PTRAUTH) requesting this feature
 to be enabled. Without this flag, pointer authentication is not enabled
 in KVM guests and attempted use of the feature will result in an UNDEFINED
 exception being injected into the guest.
+
+Additionally, when KVM_ARM_VCPU_PTRAUTH is not set then KVM will filter
+out the Pointer Authentication system key registers from KVM_GET/SET_REG_*
+ioctls.
arch/arm64/kvm/sys_regs.c +0 -0
index f7bcc60..c2f4974 100644
@@ -1005,8 +1005,13 @@ static bool trap_ptrauth(struct kvm_vcpu *vcpu,
 	return false;

arch/arm64/kvm/sys_regs.c static bool trap_ptrauth(struct kvm_vcpu *vcpu, · line 1014

Quoted 10 lines
arch/arm64/kvm/sys_regs.c +6 -1
 }
 
+static bool check_ptrauth(const struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
+{
+	return kvm_arm_vcpu_ptrauth_allowed(vcpu);
+}
+
 #define __PTRAUTH_KEY(k)						\
-	{ SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k }
+	{ SYS_DESC(SYS_## k), trap_ptrauth, reset_unknown, k , .check_present = check_ptrauth}

Looks good. I'm pretty sure the changes due to Dave's v5 map neatly.

Thanks,

James

#34 lore raw

PATCH v6 6/6 arm/kvm: arm64: Add a vcpu feature for pointer authentication

In reply to Amit Daniel Kachhap

This is a runtime capabality for KVM tool to enable Armv8.3 Pointer Authentication in guest kernel. A command line option --ptrauth is required for this.

---
 arm/aarch32/include/kvm/kvm-cpu-arch.h    | 1 +
 arm/aarch64/include/asm/kvm.h             | 1 +
 arm/aarch64/include/kvm/kvm-config-arch.h | 4 +++-
 arm/aarch64/include/kvm/kvm-cpu-arch.h    | 1 +
 arm/include/arm-common/kvm-config-arch.h  | 1 +
 arm/kvm-cpu.c                             | 6 ++++++
 include/linux/kvm.h                       | 1 +
 7 files changed, 14 insertions(+), 1 deletion(-)
arm/aarch32/include/kvm/kvm-cpu-arch.h +1 -0
index d28ea67..520ea76 100644
@@ -13,4 +13,5 @@
 #define ARM_CPU_ID		0, 0, 0
 #define ARM_CPU_ID_MPIDR	5
 
+#define ARM_VCPU_PTRAUTH_FEATURE	0
 #endif /* KVM__KVM_CPU_ARCH_H */
arm/aarch64/include/asm/kvm.h +1 -0
index 97c3478..1068fd1 100644
@@ -102,6 +102,7 @@ struct kvm_regs {
 #define KVM_ARM_VCPU_EL1_32BIT		1 /* CPU running a 32bit VM */
 #define KVM_ARM_VCPU_PSCI_0_2		2 /* CPU uses PSCI v0.2 */
 #define KVM_ARM_VCPU_PMU_V3		3 /* Support guest PMUv3 */
+#define KVM_ARM_VCPU_PTRAUTH		4 /* CPU uses pointer authentication */
 
 struct kvm_vcpu_init {
 	__u32 target;
arm/aarch64/include/kvm/kvm-config-arch.h +3 -1
index 04be43d..2074684 100644
@@ -8,7 +8,9 @@
 			"Create PMUv3 device"),				\
 	OPT_U64('\0', "kaslr-seed", &(cfg)->kaslr_seed,			\
 			"Specify random seed for Kernel Address Space "	\
-			"Layout Randomization (KASLR)"),
+			"Layout Randomization (KASLR)"),		\
+	OPT_BOOLEAN('\0', "ptrauth", &(cfg)->has_ptrauth,		\
+			"Enable address authentication"),
 
 #include "arm-common/kvm-config-arch.h"
 
arm/aarch64/include/kvm/kvm-cpu-arch.h +1 -0
index a9d8563..496ece8 100644
@@ -17,4 +17,5 @@
 #define ARM_CPU_CTRL		3, 0, 1, 0
 #define ARM_CPU_CTRL_SCTLR_EL1	0
 
+#define ARM_VCPU_PTRAUTH_FEATURE	(1UL << KVM_ARM_VCPU_PTRAUTH)
 #endif /* KVM__KVM_CPU_ARCH_H */
arm/include/arm-common/kvm-config-arch.h +1 -0
index 5734c46..5badcbd 100644
@@ -10,6 +10,7 @@ struct kvm_config_arch {
 	bool		aarch32_guest;
 	bool		has_pmuv3;
 	u64		kaslr_seed;
+	bool		has_ptrauth;
 	enum irqchip_type irqchip;
 	u64		fw_addr;
 };
arm/kvm-cpu.c +6 -0
index 7780251..4ac80f8 100644
@@ -68,6 +68,12 @@ struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id)
 		vcpu_init.features[0] |= (1UL << KVM_ARM_VCPU_PSCI_0_2);
 	}
 
+	/* Set KVM_ARM_VCPU_PTRAUTH if available */
+	if (kvm__supports_extension(kvm, KVM_CAP_ARM_PTRAUTH)) {
+		if (kvm->cfg.arch.has_ptrauth)
+			vcpu_init.features[0] |= ARM_VCPU_PTRAUTH_FEATURE;
+	}
+
 	/*
 	 * If the preferred target ioctl is successful then
 	 * use preferred target else try each and every target type
include/linux/kvm.h +1 -0
index 6d4ea4b..a553477 100644
@@ -988,6 +988,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_ARM_VM_IPA_SIZE 165
 #define KVM_CAP_MANUAL_DIRTY_LOG_PROTECT 166
 #define KVM_CAP_HYPERV_CPUID 167
+#define KVM_CAP_ARM_PTRAUTH 168
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
Signature · 1 line
2.7.4
#35 lore raw

Re: [kvmtool PATCH v6 6/6] arm/kvm: arm64: Add a vcpu feature for pointer authentication

Quoted 42 lines · Amit Daniel Kachhap

This is a runtime capabality for KVM tool to enable Armv8.3 Pointer Authentication in guest kernel. A command line option --ptrauth is required for this.

---
 arm/aarch32/include/kvm/kvm-cpu-arch.h    | 1 +
 arm/aarch64/include/asm/kvm.h             | 1 +
 arm/aarch64/include/kvm/kvm-config-arch.h | 4 +++-
 arm/aarch64/include/kvm/kvm-cpu-arch.h    | 1 +
 arm/include/arm-common/kvm-config-arch.h  | 1 +
 arm/kvm-cpu.c                             | 6 ++++++
 include/linux/kvm.h                       | 1 +
 7 files changed, 14 insertions(+), 1 deletion(-)
arm/aarch32/include/kvm/kvm-cpu-arch.h +1 -0
index d28ea67..520ea76 100644
@@ -13,4 +13,5 @@
 #define ARM_CPU_ID		0, 0, 0
 #define ARM_CPU_ID_MPIDR	5
 
+#define ARM_VCPU_PTRAUTH_FEATURE	0
 #endif /* KVM__KVM_CPU_ARCH_H */
arm/aarch64/include/asm/kvm.h +1 -0
index 97c3478..1068fd1 100644
@@ -102,6 +102,7 @@ struct kvm_regs {
 #define KVM_ARM_VCPU_EL1_32BIT		1 /* CPU running a 32bit VM */
 #define KVM_ARM_VCPU_PSCI_0_2		2 /* CPU uses PSCI v0.2 */
 #define KVM_ARM_VCPU_PMU_V3		3 /* Support guest PMUv3 */
+#define KVM_ARM_VCPU_PTRAUTH		4 /* CPU uses pointer authentication */
 
 struct kvm_vcpu_init {
 	__u32 target;
arm/aarch64/include/kvm/kvm-config-arch.h +0 -0
index 04be43d..2074684 100644

arm/aarch64/include/kvm/kvm-config-arch.h line 13

Quoted 7 lines
arm/aarch64/include/kvm/kvm-config-arch.h +3 -1
@@ -8,7 +8,9 @@
 			"Create PMUv3 device"),				\
 	OPT_U64('\0', "kaslr-seed", &(cfg)->kaslr_seed,			\
 			"Specify random seed for Kernel Address Space "	\
-			"Layout Randomization (KASLR)"),
+			"Layout Randomization (KASLR)"),		\
+	OPT_BOOLEAN('\0', "ptrauth", &(cfg)->has_ptrauth,		\
+			"Enable address authentication"),

Nit: doesn't this enable address *and* generic authentication? The discussion on what capababilities and enables the ABI exposes probably needs to conclude before we can finalise this here.

However, I would recommend that we provide a single option here that turns both address authentication and generic authentication on, even if the ABI treats them independently. This is expected to be the common case by far.

We can always add more fine-grained options later if it turns out to be necessary.

Quoted 29 lines
arm/aarch64/include/kvm/kvm-config-arch.h +0 -0
 #include "arm-common/kvm-config-arch.h"
 
arm/aarch64/include/kvm/kvm-cpu-arch.h +1 -0
index a9d8563..496ece8 100644
@@ -17,4 +17,5 @@
 #define ARM_CPU_CTRL		3, 0, 1, 0
 #define ARM_CPU_CTRL_SCTLR_EL1	0
 
+#define ARM_VCPU_PTRAUTH_FEATURE	(1UL << KVM_ARM_VCPU_PTRAUTH)
 #endif /* KVM__KVM_CPU_ARCH_H */
arm/include/arm-common/kvm-config-arch.h +1 -0
index 5734c46..5badcbd 100644
@@ -10,6 +10,7 @@ struct kvm_config_arch {
 	bool		aarch32_guest;
 	bool		has_pmuv3;
 	u64		kaslr_seed;
+	bool		has_ptrauth;
 	enum irqchip_type irqchip;
 	u64		fw_addr;
 };
arm/kvm-cpu.c +0 -0
index 7780251..4ac80f8 100644

arm/kvm-cpu.c struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id) · line 76

Quoted 9 lines
arm/kvm-cpu.c +6 -0
@@ -68,6 +68,12 @@ struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id)
 		vcpu_init.features[0] |= (1UL << KVM_ARM_VCPU_PSCI_0_2);
 	}
 
+	/* Set KVM_ARM_VCPU_PTRAUTH if available */
+	if (kvm__supports_extension(kvm, KVM_CAP_ARM_PTRAUTH)) {
+		if (kvm->cfg.arch.has_ptrauth)
+			vcpu_init.features[0] |= ARM_VCPU_PTRAUTH_FEATURE;
+	}
+

I'm not too keen on requiring a dummy #define for AArch32 here. How do we handle other subarch-specific feature flags? Is there something we can reuse?

(For SVE I didn''t have a proper solution for this yet: my kvmtool patches are still a dirty hack...)

[...]

Cheers
---Dave

#36 lore raw

Re: [kvmtool PATCH v6 6/6] arm/kvm: arm64: Add a vcpu feature for pointer authentication

Hi,

Quoted 46 lines · Dave Martin
Quoted 45 lines · Amit Daniel Kachhap

This is a runtime capabality for KVM tool to enable Armv8.3 Pointer Authentication in guest kernel. A command line option --ptrauth is required for this.

---
  arm/aarch32/include/kvm/kvm-cpu-arch.h    | 1 +
  arm/aarch64/include/asm/kvm.h             | 1 +
  arm/aarch64/include/kvm/kvm-config-arch.h | 4 +++-
  arm/aarch64/include/kvm/kvm-cpu-arch.h    | 1 +
  arm/include/arm-common/kvm-config-arch.h  | 1 +
  arm/kvm-cpu.c                             | 6 ++++++
  include/linux/kvm.h                       | 1 +
  7 files changed, 14 insertions(+), 1 deletion(-)
arm/aarch32/include/kvm/kvm-cpu-arch.h +1 -0
index d28ea67..520ea76 100644
@@ -13,4 +13,5 @@
  #define ARM_CPU_ID		0, 0, 0
  #define ARM_CPU_ID_MPIDR	5
 
+#define ARM_VCPU_PTRAUTH_FEATURE	0
  #endif /* KVM__KVM_CPU_ARCH_H */
arm/aarch64/include/asm/kvm.h +1 -0
index 97c3478..1068fd1 100644
@@ -102,6 +102,7 @@ struct kvm_regs {
  #define KVM_ARM_VCPU_EL1_32BIT		1 /* CPU running a 32bit VM */
  #define KVM_ARM_VCPU_PSCI_0_2		2 /* CPU uses PSCI v0.2 */
  #define KVM_ARM_VCPU_PMU_V3		3 /* Support guest PMUv3 */
+#define KVM_ARM_VCPU_PTRAUTH		4 /* CPU uses pointer authentication */
 
  struct kvm_vcpu_init {
  	__u32 target;
arm/aarch64/include/kvm/kvm-config-arch.h +0 -0
index 04be43d..2074684 100644
@@ -8,7 +8,9 @@
  			"Create PMUv3 device"),				\
  	OPT_U64('\0', "kaslr-seed", &(cfg)->kaslr_seed,			\
  			"Specify random seed for Kernel Address Space "	\
Quoted 8 lines

arm/aarch64/include/kvm/kvm-config-arch.h line 13

Quoted 4 lines
arm/aarch64/include/kvm/kvm-config-arch.h +3 -1
-			"Layout Randomization (KASLR)"),
+			"Layout Randomization (KASLR)"),		\
+	OPT_BOOLEAN('\0', "ptrauth", &(cfg)->has_ptrauth,		\
+			"Enable address authentication"),

Nit: doesn't this enable address *and* generic authentication? The discussion on what capababilities and enables the ABI exposes probably needs to conclude before we can finalise this here.

ok.

Quoted 5 lines

However, I would recommend that we provide a single option here that turns both address authentication and generic authentication on, even if the ABI treats them independently. This is expected to be the common case by far.

ok

Quoted 3 lines

We can always add more fine-grained options later if it turns out to be necessary.

Mark suggested to provide 2 flags [1] for Address and Generic authentication so I was thinking of adding 2 features like,

+#define KVM_ARM_VCPU_PTRAUTH_ADDR 4 /* CPU uses pointer address authentication */
+#define KVM_ARM_VCPU_PTRAUTH_GENERIC 5 /* CPU uses pointer generic authentication */

And supply both of them concatenated in VCPU_INIT stage. Kernel KVM would expect both feature requested together.

[1]: https://www.spinics.net/lists/arm-kernel/msg709181.html

Quoted 33 lines
Quoted 33 lines
arm/aarch64/include/kvm/kvm-config-arch.h +0 -0
  #include "arm-common/kvm-config-arch.h"
 
arm/aarch64/include/kvm/kvm-cpu-arch.h +1 -0
index a9d8563..496ece8 100644
@@ -17,4 +17,5 @@
  #define ARM_CPU_CTRL		3, 0, 1, 0
  #define ARM_CPU_CTRL_SCTLR_EL1	0
 
+#define ARM_VCPU_PTRAUTH_FEATURE	(1UL << KVM_ARM_VCPU_PTRAUTH)
  #endif /* KVM__KVM_CPU_ARCH_H */
arm/include/arm-common/kvm-config-arch.h +1 -0
index 5734c46..5badcbd 100644
@@ -10,6 +10,7 @@ struct kvm_config_arch {
  	bool		aarch32_guest;
  	bool		has_pmuv3;
  	u64		kaslr_seed;
+	bool		has_ptrauth;
  	enum irqchip_type irqchip;
  	u64		fw_addr;
  };
arm/kvm-cpu.c +0 -0
index 7780251..4ac80f8 100644
@@ -68,6 +68,12 @@ struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id)
  		vcpu_init.features[0] |= (1UL << KVM_ARM_VCPU_PSCI_0_2);
  	}
 
Quoted 10 lines

arm/kvm-cpu.c struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id) · line 76

Quoted 6 lines
arm/kvm-cpu.c +6 -0
+	/* Set KVM_ARM_VCPU_PTRAUTH if available */
+	if (kvm__supports_extension(kvm, KVM_CAP_ARM_PTRAUTH)) {
+		if (kvm->cfg.arch.has_ptrauth)
+			vcpu_init.features[0] |= ARM_VCPU_PTRAUTH_FEATURE;
+	}
+

I'm not too keen on requiring a dummy #define for AArch32 here. How do we handle other subarch-specific feature flags? Is there something we can reuse?

I will check it.

Thanks,
Amit D

Quoted 9 lines

(For SVE I didn''t have a proper solution for this yet: my kvmtool patches are still a dirty hack...)

[...]

Cheers
---Dave

#37 lore raw

Re: [kvmtool PATCH v6 6/6] arm/kvm: arm64: Add a vcpu feature for pointer authentication

Quoted 4 lines · Amit Daniel Kachhap

Hi,

Quoted 1 line · Dave Martin

On Tue, Feb 19, 2019 at 02:54:31PM +0530, Amit Daniel Kachhap wrote:

[...]

Quoted 29 lines
Quoted 8 lines
Quoted 8 lines
arm/aarch64/include/kvm/kvm-config-arch.h +0 -0
index 04be43d..2074684 100644
@@ -8,7 +8,9 @@
  "Create PMUv3 device"),\
  OPT_U64('\0', "kaslr-seed", &(cfg)->kaslr_seed,\
  "Specify random seed for Kernel Address Space "\
Quoted 8 lines

arm/aarch64/include/kvm/kvm-config-arch.h line 13

Quoted 4 lines
arm/aarch64/include/kvm/kvm-config-arch.h +3 -1
-"Layout Randomization (KASLR)"),
+"Layout Randomization (KASLR)"),\
+OPT_BOOLEAN('\0', "ptrauth", &(cfg)->has_ptrauth,\
+"Enable address authentication"),

Nit: doesn't this enable address *and* generic authentication? The discussion on what capababilities and enables the ABI exposes probably needs to conclude before we can finalise this here.

ok.

Quoted 5 lines

However, I would recommend that we provide a single option here that turns both address authentication and generic authentication on, even if the ABI treats them independently. This is expected to be the common case by far.

ok

Quoted 3 lines

We can always add more fine-grained options later if it turns out to be necessary.

Mark suggested to provide 2 flags [1] for Address and Generic authentication so I was thinking of adding 2 features like,

Quoted 7 lines

+#define KVM_ARM_VCPU_PTRAUTH_ADDR4 /* CPU uses pointer address authentication */
+#define KVM_ARM_VCPU_PTRAUTH_GENERIC5 /* CPU uses pointer generic authentication */

And supply both of them concatenated in VCPU_INIT stage. Kernel KVM would expect both feature requested together.

Seems reasonable. Do you mean the kernel would treat it as an error if only one of these flags is passed to KVM_ARM_VCPU_INIT, or would KVM simply treat them as independent?

[...]

Quoted 18 lines
Quoted 8 lines
Quoted 8 lines
arm/kvm-cpu.c +0 -0
index 7780251..4ac80f8 100644
@@ -68,6 +68,12 @@ struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id)
  vcpu_init.features[0] |= (1UL << KVM_ARM_VCPU_PSCI_0_2);
  }
 
Quoted 10 lines

arm/kvm-cpu.c struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id) · line 76

Quoted 6 lines
arm/kvm-cpu.c +6 -0
+/* Set KVM_ARM_VCPU_PTRAUTH if available */
+if (kvm__supports_extension(kvm, KVM_CAP_ARM_PTRAUTH)) {
+if (kvm->cfg.arch.has_ptrauth)
+vcpu_init.features[0] |= ARM_VCPU_PTRAUTH_FEATURE;
+}
+

I'm not too keen on requiring a dummy #define for AArch32 here. How do we handle other subarch-specific feature flags? Is there something we can reuse?

Quoted 1 line

I will check it.

OK

Cheers
---Dave
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

#38 lore raw

Re: [kvmtool PATCH v6 6/6] arm/kvm: arm64: Add a vcpu feature for pointer authentication

Hi Dave,

Quoted 42 lines · Dave P Martin
Quoted 4 lines · Amit Daniel Kachhap

Hi,

Quoted 1 line · Dave Martin

On Tue, Feb 19, 2019 at 02:54:31PM +0530, Amit Daniel Kachhap wrote:

[...]

Quoted 34 lines
Quoted 8 lines
Quoted 8 lines
arm/aarch64/include/kvm/kvm-config-arch.h +0 -0
index 04be43d..2074684 100644
@@ -8,7 +8,9 @@
   			"Create PMUv3 device"),				\
   	OPT_U64('\0', "kaslr-seed", &(cfg)->kaslr_seed,			\
   			"Specify random seed for Kernel Address Space "	\
Quoted 8 lines

arm/aarch64/include/kvm/kvm-config-arch.h line 13

Quoted 4 lines
arm/aarch64/include/kvm/kvm-config-arch.h +3 -1
-			"Layout Randomization (KASLR)"),
+			"Layout Randomization (KASLR)"),		\
+	OPT_BOOLEAN('\0', "ptrauth", &(cfg)->has_ptrauth,		\
+			"Enable address authentication"),

Nit: doesn't this enable address *and* generic authentication? The discussion on what capababilities and enables the ABI exposes probably needs to conclude before we can finalise this here.

ok.

Quoted 5 lines

However, I would recommend that we provide a single option here that turns both address authentication and generic authentication on, even if the ABI treats them independently. This is expected to be the common case by far.

ok

Quoted 3 lines

We can always add more fine-grained options later if it turns out to be necessary.

Mark suggested to provide 2 flags [1] for Address and Generic authentication so I was thinking of adding 2 features like,

+#define KVM_ARM_VCPU_PTRAUTH_ADDR 4 /* CPU uses pointer address
authentication */
+#define KVM_ARM_VCPU_PTRAUTH_GENERIC 5 /* CPU uses pointer generic
authentication */

Quoted 6 lines
Quoted 2 lines

And supply both of them concatenated in VCPU_INIT stage. Kernel KVM would expect both feature requested together.

Seems reasonable. Do you mean the kernel would treat it as an error if only one of these flags is passed to KVM_ARM_VCPU_INIT, or would KVM simply treat them as independent?

If both flags are passed together then only start using ptrauth otherwise keep ptrauth disabled. This is just to finalize the user side abi as of now and KVM can be updated later.

Thanks,
Amit D

Quoted 28 lines

[...]

Quoted 18 lines
Quoted 8 lines
Quoted 8 lines
arm/kvm-cpu.c +0 -0
index 7780251..4ac80f8 100644
@@ -68,6 +68,12 @@ struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id)
   		vcpu_init.features[0] |= (1UL << KVM_ARM_VCPU_PSCI_0_2);
   	}
 
Quoted 10 lines

arm/kvm-cpu.c struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id) · line 76

Quoted 6 lines
arm/kvm-cpu.c +6 -0
+	/* Set KVM_ARM_VCPU_PTRAUTH if available */
+	if (kvm__supports_extension(kvm, KVM_CAP_ARM_PTRAUTH)) {
+		if (kvm->cfg.arch.has_ptrauth)
+			vcpu_init.features[0] |= ARM_VCPU_PTRAUTH_FEATURE;
+	}
+

I'm not too keen on requiring a dummy #define for AArch32 here. How do we handle other subarch-specific feature flags? Is there something we can reuse?

Quoted 1 line

I will check it.

OK

Cheers
---Dave

#39 lore raw

Re: [kvmtool PATCH v6 6/6] arm/kvm: arm64: Add a vcpu feature for pointer authentication

Quoted 52 lines · Amit Daniel Kachhap

Hi Dave,

Quoted 42 lines · Dave P Martin
Quoted 4 lines · Amit Daniel Kachhap

Hi,

Quoted 1 line · Dave Martin

On Tue, Feb 19, 2019 at 02:54:31PM +0530, Amit Daniel Kachhap wrote:

[...]

Quoted 34 lines
Quoted 8 lines
Quoted 8 lines
arm/aarch64/include/kvm/kvm-config-arch.h +0 -0
index 04be43d..2074684 100644
@@ -8,7 +8,9 @@
  			"Create PMUv3 device"),				\
  	OPT_U64('\0', "kaslr-seed", &(cfg)->kaslr_seed,			\
  			"Specify random seed for Kernel Address Space "	\
Quoted 8 lines

arm/aarch64/include/kvm/kvm-config-arch.h line 13

Quoted 4 lines
arm/aarch64/include/kvm/kvm-config-arch.h +3 -1
-			"Layout Randomization (KASLR)"),
+			"Layout Randomization (KASLR)"),		\
+	OPT_BOOLEAN('\0', "ptrauth", &(cfg)->has_ptrauth,		\
+			"Enable address authentication"),

Nit: doesn't this enable address *and* generic authentication? The discussion on what capababilities and enables the ABI exposes probably needs to conclude before we can finalise this here.

ok.

Quoted 5 lines

However, I would recommend that we provide a single option here that turns both address authentication and generic authentication on, even if the ABI treats them independently. This is expected to be the common case by far.

ok

Quoted 3 lines

We can always add more fine-grained options later if it turns out to be necessary.

Mark suggested to provide 2 flags [1] for Address and Generic authentication so I was thinking of adding 2 features like,

+#define KVM_ARM_VCPU_PTRAUTH_ADDR 4 /* CPU uses pointer address
authentication */
+#define KVM_ARM_VCPU_PTRAUTH_GENERIC 5 /* CPU uses pointer generic
authentication */

Quoted 6 lines
Quoted 2 lines

And supply both of them concatenated in VCPU_INIT stage. Kernel KVM would expect both feature requested together.

Seems reasonable. Do you mean the kernel would treat it as an error if only one of these flags is passed to KVM_ARM_VCPU_INIT, or would KVM simply treat them as independent?

Quoted 3 lines

If both flags are passed together then only start using ptrauth otherwise keep ptrauth disabled. This is just to finalize the user side abi as of now and KVM can be updated later.

If just flag is passed, I think KVM_ARM_VCPU_INIT should just fail. Otherwise we risk userspace becoming accidentally reliant on behaviour that may change in the future.

Cheers
---Dave

#40 lore raw
In reply to Amit Daniel Kachhap

Hi Amit,

Quoted 23 lines · Amit Daniel Kachhap

This patch series adds pointer authentication support for KVM guest and is based on top of Linux 5.0-rc6. The basic patches in this series was originally posted by Mark Rutland earlier[1,2] and contains some history of this work.

Extension Overview:
=============================================

The ARMv8.3 pointer authentication extension adds functionality to detect modification of pointer values, mitigating certain classes of attack such as stack smashing, and making return oriented programming attacks harder.

The extension introduces the concept of a pointer authentication code (PAC), which is stored in some upper bits of pointers. Each PAC is derived from the original pointer, another 64-bit value (e.g. the stack pointer), and a secret 128-bit key.

New instructions are added which can be used to:

* Insert a PAC into a pointer
* Strip a PAC from a pointer
* Authenticate and strip a PAC from a pointer

Quoted 7 lines

The detailed description of ARMv8.3 pointer authentication support in userspace/kernel and can be found in Kristina's generic pointer authentication patch series[3].

This patch series is based on just a single patch from Dave Martin [8] which add control checks for accessing sys registers.

Ooeer, If you miss this patch, (like I did) the series still applies to rc6, it just doesn't build. If you depend on extra patches like this, please re-post them as part of the series. (you need to add your Signed-off-by if picked the patch up from the list).

This lets people apply the series from the list (everyone has a script to to do this), without having to go and find the dependencies.

Quoted 1 line

[8]: https://lore.kernel.org/linux-arm-kernel/[email protected]/

This is v4 of Dave's patch. He changed the subject and posted a v5 here:
https://lore.kernel.org/linux-arm-kernel/[email protected]/

Re-posting the patch you tested with would avoid someone accidentally pickup v5, then trying to work out how its supposed to work with your series. (check_present() was replaced by a restrictions() bitmask).

As we can't have both, and v5 of that patch has been reviewed, could you rebase onto it? You'll need to pick up any tags and make any changes reviewers asked for. If you could note 'this v7 patch is Dave's v5 with $changes', then it makes it clear what is going on.

Thanks,

James