Skip to contentSkip to conversation

KVM: arm64: nv: Work around lack of pauth support in old toolchains

PATCH In review Marc Zyngier 4 participants 9 messages Apr 22, 2024 – Apr 24, 2024

Subsystems (2)
lore
Share & tools
lore

Marc Zyngier

author maintainer
#0 lore raw

PATCH 1/1 KVM: arm64: nv: Work around lack of pauth support in old toolchains

We still support GCC 8.x, and it appears that this toolchain does not understand "pauth" as a valid architectural extension. After all, it's only been 8 years since ARMv8.3 was released...

This results in the NV ERETAx code breaking the build, as it relies on this extention to make use of the PACGA instruction.

Work around it by hand-assembling the instruction using a mind-bending trick lifted from an old patch by Will. Magic.

Fixes e09faab353a6 ("KVM: arm64: nv: Add emulation for ERETAx instructions")
---
 arch/arm64/kvm/pauth.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)
arch/arm64/kvm/pauth.c +17 -2
index a3a5c404375b..8baf2a2cbdd3 100644
@@ -17,6 +17,22 @@
 #include <asm/kvm_emulate.h>
 #include <asm/pointer_auth.h>
 
+/*
+ * "// This is some of my finest work" (Will Deacon, 2019-02-12)
+ *
+ * The jury is still out on that one.
+ */
+#define REG(r)	"(0%x[" #r "] - ((0%x[" #r "] >> 4) * 6))"
+
+/* PACGA Xd, Xn, Xm */
+#define PACGA(d,n,m)						\
+	asm volatile(".inst 0x9AC03000   |"			\
+		     "(" REG(Rd) "<< 0)  |"			\
+		     "(" REG(Rn) "<< 5)  |"			\
+		     "(" REG(Rm) "<< 16)\n"			\
+		     : [Rd] "=r" ((d))				\
+		     : [Rn] "r" ((n)), [Rm] "r" ((m)))
+
 static u64 compute_pac(struct kvm_vcpu *vcpu, u64 ptr,
 		       struct ptrauth_key ikey)
 {
@@ -36,8 +52,7 @@ static u64 compute_pac(struct kvm_vcpu *vcpu, u64 ptr,
 	__ptrauth_key_install_nosync(APGA, ikey);
 	isb();
 
-	asm volatile(ARM64_ASM_PREAMBLE ".arch_extension pauth\n"
-		     "pacga %0, %1, %2" : "=r" (pac) : "r" (ptr), "r" (mod));
+	PACGA(pac, ptr, mod);
 	isb();
 
 	__ptrauth_key_install_nosync(APGA, gkey);
Signature · 1 line
2.39.2
Acked-by on v1 1/1
#1 lore raw
Quoted 3 lines · Marc Zyngier

We still support GCC 8.x, and it appears that this toolchain does not understand "pauth" as a valid architectural extension. After all, it's only been 8 years since ARMv8.3 was released...

Just to clarify: I'm fairly sure that all supported toolchains support ARMv8.3 and PACGA, the problem with ".arch_extension pauth\n" seems to be that it was retroactively made an optional feature for earlier architecture versions a few years after ARMv8.3, so most binutils versions we support understand pacga as an armv8.3 feature but reject the pauth name for the extension.

Quoted 9 lines

This results in the NV ERETAx code breaking the build, as it relies on this extention to make use of the PACGA instruction.

Work around it by hand-assembling the instruction using a mind-bending trick lifted from an old patch by Will. Magic.

Fixes e09faab353a6 ("KVM: arm64: nv: Add emulation for ERETAx instructions")

I usually try to enclose these in a version check like "#if CONFIG_AS_VERSION >= 22800" so we can be sure we clean them up whenever the minimum version changes, but I assume you'll remember this one.

#2 lore raw
Quoted 5 lines · Arnd Bergmann
Quoted 3 lines · Marc Zyngier

We still support GCC 8.x, and it appears that this toolchain does not understand "pauth" as a valid architectural extension. After all, it's only been 8 years since ARMv8.3 was released...

Quoted 7 lines

Just to clarify: I'm fairly sure that all supported toolchains support ARMv8.3 and PACGA, the problem with ".arch_extension pauth\n" seems to be that it was retroactively made an optional feature for earlier architecture versions a few years after ARMv8.3, so most binutils versions we support understand pacga as an armv8.3 feature but reject the pauth name for the extension.

Kind of agree with Arnd here.
Shall the fix just remove the ".arch_extension pauth"?

I've tried gcc 7 failed with the pauth name for the extension. After I remove the ".arch_extension pauth" and use "pacga" instruction directly pass the gcc 7 compilation.

By the way, have a glance for gcc7/8/9/10 manual, from the gcc 10 manual [1], seems gcc 10 was the first introduced the pauth as the extension name support.
https://gcc.gnu.org/onlinedocs/gcc-10.5.0/gcc/AArch64-Options.html#g_t-march-and--mcpu-Feature-Modifiers

Quoted 22 lines
Quoted 10 lines

This results in the NV ERETAx code breaking the build, as it relies on this extention to make use of the PACGA instruction.

Work around it by hand-assembling the instruction using a mind-bending trick lifted from an old patch by Will. Magic.

Fixes e09faab353a6 ("KVM: arm64: nv: Add emulation for ERETAx instructions")

I usually try to enclose these in a version check like "#if CONFIG_AS_VERSION >= 22800" so we can be sure we clean them up whenever the minimum version changes, but I assume you'll remember this one.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Signature · 2 lines
Thx and BRs,
Aiqun(Maria) Yu
author maintainer
#3 lore raw
Quoted 16 lines · Aiqun Yu (Maria)
Quoted 5 lines · Arnd Bergmann
Quoted 3 lines · Marc Zyngier

We still support GCC 8.x, and it appears that this toolchain does not understand "pauth" as a valid architectural extension. After all, it's only been 8 years since ARMv8.3 was released...

Quoted 7 lines

Just to clarify: I'm fairly sure that all supported toolchains support ARMv8.3 and PACGA, the problem with ".arch_extension pauth\n" seems to be that it was retroactively made an optional feature for earlier architecture versions a few years after ARMv8.3, so most binutils versions we support understand pacga as an armv8.3 feature but reject the pauth name for the extension.

Quoted 6 lines

Kind of agree with Arnd here.
Shall the fix just remove the ".arch_extension pauth"?

I've tried gcc 7 failed with the pauth name for the extension. After I remove the ".arch_extension pauth" and use "pacga" instruction directly pass the gcc 7 compilation.

And breaks with LLVM:

CC arch/arm64/kvm/pauth.o
arch/arm64/kvm/pauth.c:40:9: error: instruction requires: pauth

                     "pacga %0, %1, %2" : "=r" (pac) : "r" (ptr), "r" (mod));
                      ^

<inline asm>:2:1: note: instantiated into assembly here pacga x19, x1, x9
^
1 error generated.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#4 lore raw
Quoted 15 lines · Marc Zyngier
Quoted 13 lines · Aiqun Yu (Maria)
Quoted 12 lines · Arnd Bergmann
Quoted 3 lines · Marc Zyngier

We still support GCC 8.x, and it appears that this toolchain does not understand "pauth" as a valid architectural extension. After all, it's only been 8 years since ARMv8.3 was released...

Just to clarify: I'm fairly sure that all supported toolchains support ARMv8.3 and PACGA, the problem with ".arch_extension pauth\n" seems to be that it was retroactively made an optional feature for earlier architecture versions a few years after ARMv8.3, so most binutils versions we support understand pacga as an armv8.3 feature but reject the pauth name for the extension.

Quoted 6 lines
Quoted 6 lines

Kind of agree with Arnd here.
Shall the fix just remove the ".arch_extension pauth"?

I've tried gcc 7 failed with the pauth name for the extension. After I remove the ".arch_extension pauth" and use "pacga" instruction directly pass the gcc 7 compilation.

It really depends on the binutils version, not gcc of course.

Quoted 9 lines

And breaks with LLVM:

CC arch/arm64/kvm/pauth.o
arch/arm64/kvm/pauth.c:40:9: error: instruction requires: pauth

                     "pacga %0, %1, %2" : "=r" (pac) : "r" (ptr), "r" (mod));
                      ^

<inline asm>:2:1: note: instantiated into assembly here pacga x19, x1, x9
^

It works when building with LLVM_IAS=0, which we obviously don't want to mandate here. The variant below works for both clang+ias (including all still supported versions) and gcc+binutils, but at that point it gets obscure enough that your .inst version is easier to understand.

Arnd

arch/arm64/kvm/pauth.c +6 -1
@@ -36,7 +36,12 @@ static u64 compute_pac(struct kvm_vcpu *vcpu, u64 ptr,
        __ptrauth_key_install_nosync(APGA, ikey);
        isb();
 
-       asm volatile(ARM64_ASM_PREAMBLE ".arch_extension pauth\n"
+       asm volatile(ARM64_ASM_PREAMBLE
+#ifdef CONFIG_AS_IS_LLVM
+                    ".arch_extension pauth\n"
+#else
+                    ".arch armv8.3-a\n"
+#endif
                     "pacga %0, %1, %2" : "=r" (pac) : "r" (ptr), "r" (mod));
        isb();
 
author maintainer
#5 lore raw
Quoted 17 lines · Arnd Bergmann
Quoted 15 lines · Marc Zyngier
Quoted 13 lines · Aiqun Yu (Maria)
Quoted 12 lines · Arnd Bergmann
Quoted 3 lines · Marc Zyngier

We still support GCC 8.x, and it appears that this toolchain does not understand "pauth" as a valid architectural extension. After all, it's only been 8 years since ARMv8.3 was released...

Just to clarify: I'm fairly sure that all supported toolchains support ARMv8.3 and PACGA, the problem with ".arch_extension pauth\n" seems to be that it was retroactively made an optional feature for earlier architecture versions a few years after ARMv8.3, so most binutils versions we support understand pacga as an armv8.3 feature but reject the pauth name for the extension.

Quoted 8 lines
Quoted 6 lines
Quoted 6 lines

Kind of agree with Arnd here.
Shall the fix just remove the ".arch_extension pauth"?

I've tried gcc 7 failed with the pauth name for the extension. After I remove the ".arch_extension pauth" and use "pacga" instruction directly pass the gcc 7 compilation.

It really depends on the binutils version, not gcc of course.

Right. I'll amend the commit message to reflect that.

Quoted 11 lines
Quoted 10 lines

And breaks with LLVM:

CC arch/arm64/kvm/pauth.o
arch/arm64/kvm/pauth.c:40:9: error: instruction requires: pauth

                     "pacga %0, %1, %2" : "=r" (pac) : "r" (ptr), "r" (mod));
                      ^

<inline asm>:2:1: note: instantiated into assembly here pacga x19, x1, x9
^

Quoted 5 lines

It works when building with LLVM_IAS=0, which we obviously don't want to mandate here. The variant below works for both clang+ias (including all still supported versions) and gcc+binutils, but at that point it gets obscure enough that your .inst version is easier to understand.

Exactly. Either we have a good way to abstract this behind the scenes (which I don't see right now), or we just assume control of the instruction generation, which is what my patch does.

In general, I question the value of the ".arch_extension" requirement for something like Linux, where we already have a pretty fine grained control of what we want to see being output by the compiler. but that ship has sailed long ago.

Thanks,

	M.
Quoted 20 lines

arnd

arch/arm64/kvm/pauth.c +6 -1
@@ -36,7 +36,12 @@ static u64 compute_pac(struct kvm_vcpu *vcpu, u64 ptr,
        __ptrauth_key_install_nosync(APGA, ikey);
        isb();
 
-       asm volatile(ARM64_ASM_PREAMBLE ".arch_extension pauth\n"
+       asm volatile(ARM64_ASM_PREAMBLE
+#ifdef CONFIG_AS_IS_LLVM
+                    ".arch_extension pauth\n"
+#else
+                    ".arch armv8.3-a\n"
+#endif
                     "pacga %0, %1, %2" : "=r" (pac) : "r" (ptr), "r" (mod));
        isb();
 
Signature · 1 line
Without deviation from the norm, progress is not possible.
#6 lore raw
Quoted 47 lines · Marc Zyngier
Quoted 17 lines · Arnd Bergmann
Quoted 15 lines · Marc Zyngier
Quoted 13 lines · Aiqun Yu (Maria)
Quoted 12 lines · Arnd Bergmann
Quoted 3 lines · Marc Zyngier

We still support GCC 8.x, and it appears that this toolchain does not understand "pauth" as a valid architectural extension. After all, it's only been 8 years since ARMv8.3 was released...

Just to clarify: I'm fairly sure that all supported toolchains support ARMv8.3 and PACGA, the problem with ".arch_extension pauth\n" seems to be that it was retroactively made an optional feature for earlier architecture versions a few years after ARMv8.3, so most binutils versions we support understand pacga as an armv8.3 feature but reject the pauth name for the extension.

Quoted 8 lines
Quoted 6 lines
Quoted 6 lines

Kind of agree with Arnd here.
Shall the fix just remove the ".arch_extension pauth"?

I've tried gcc 7 failed with the pauth name for the extension. After I remove the ".arch_extension pauth" and use "pacga" instruction directly pass the gcc 7 compilation.

It really depends on the binutils version, not gcc of course.

Right. I'll amend the commit message to reflect that.

Quoted 11 lines
Quoted 10 lines

And breaks with LLVM:

CC arch/arm64/kvm/pauth.o
arch/arm64/kvm/pauth.c:40:9: error: instruction requires: pauth

                     "pacga %0, %1, %2" : "=r" (pac) : "r" (ptr), "r" (mod));
                      ^

<inline asm>:2:1: note: instantiated into assembly here pacga x19, x1, x9
^

Quoted 5 lines

It works when building with LLVM_IAS=0, which we obviously don't want to mandate here. The variant below works for both clang+ias (including all still supported versions) and gcc+binutils, but at that point it gets obscure enough that your .inst version is easier to understand.

Quoted 3 lines

Exactly. Either we have a good way to abstract this behind the scenes (which I don't see right now), or we just assume control of the instruction generation, which is what my patch does.

Ack.

Quoted 31 lines

In general, I question the value of the ".arch_extension" requirement for something like Linux, where we already have a pretty fine grained control of what we want to see being output by the compiler. but that ship has sailed long ago.

Thanks,

	M.
Quoted 20 lines

arnd

arch/arm64/kvm/pauth.c +6 -1
@@ -36,7 +36,12 @@ static u64 compute_pac(struct kvm_vcpu *vcpu, u64 ptr,
        __ptrauth_key_install_nosync(APGA, ikey);
        isb();
 
-       asm volatile(ARM64_ASM_PREAMBLE ".arch_extension pauth\n"
+       asm volatile(ARM64_ASM_PREAMBLE
+#ifdef CONFIG_AS_IS_LLVM
+                    ".arch_extension pauth\n"
+#else
+                    ".arch armv8.3-a\n"
+#endif
                     "pacga %0, %1, %2" : "=r" (pac) : "r" (ptr), "r" (mod));
        isb();
 
Signature · 2 lines
Thx and BRs,
Aiqun(Maria) Yu
reviewer
#7 lore raw
In reply to Marc Zyngier
Quoted 30 lines · Marc Zyngier

We still support GCC 8.x, and it appears that this toolchain does not understand "pauth" as a valid architectural extension. After all, it's only been 8 years since ARMv8.3 was released...

This results in the NV ERETAx code breaking the build, as it relies on this extention to make use of the PACGA instruction.

Work around it by hand-assembling the instruction using a mind-bending trick lifted from an old patch by Will. Magic.

Fixes e09faab353a6 ("KVM: arm64: nv: Add emulation for ERETAx instructions")
---
 arch/arm64/kvm/pauth.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)
arch/arm64/kvm/pauth.c +5 -0
index a3a5c404375b..8baf2a2cbdd3 100644
@@ -17,6 +17,22 @@
 #include <asm/kvm_emulate.h>
 #include <asm/pointer_auth.h>
 
+/*
+ * "// This is some of my finest work" (Will Deacon, 2019-02-12)
+ *
+ * The jury is still out on that one.
+ */

arch/arm64/kvm/pauth.c line 34

Quoted 10 lines
arch/arm64/kvm/pauth.c +10 -0
+#define REG(r)	"(0%x[" #r "] - ((0%x[" #r "] >> 4) * 6))"
+
+/* PACGA Xd, Xn, Xm */
+#define PACGA(d,n,m)						\
+	asm volatile(".inst 0x9AC03000   |"			\
+		     "(" REG(Rd) "<< 0)  |"			\
+		     "(" REG(Rn) "<< 5)  |"			\
+		     "(" REG(Rm) "<< 16)\n"			\
+		     : [Rd] "=r" ((d))				\
+		     : [Rn] "r" ((n)), [Rm] "r" ((m)))

Can you please use <asm/gpr-num.h> rather than open-coding this new REG() helper? That way this'll be consistent with the way we assemble MRS/MSR in <asm/sysreg.h>, and it avoids the few seconds of confusion trying to figure out the maths in the REG() helper (neat trick though!).

Fixlet for that below; I've build tested this and the resulting object file is the same before and after (diff shows that only the name of the file changed):

| [mark@lakrids:~/src/linux]% usekorg 13.2.0 aarch64-linux-objdump -d pauth-maz.o | grep pacga
|  204:   9ac03335        pacga   x21, x25, x0
| [mark@lakrids:~/src/linux]% usekorg 13.2.0 aarch64-linux-objdump -d pauth-rutland.o | grep pacga
|  204:   9ac03335        pacga   x21, x25, x0
| [mark@lakrids:~/src/linux]% diff <(usekorg 13.2.0 aarch64-linux-objdump -d pauth-maz.o) <(usekorg 13.2.0 aarch64-linux-objdump -d pauth-rutland.o)
| 2c2
| < pauth-maz.o:     file format elf64-littleaarch64
| ---
| > pauth-rutland.o:     file format elf64-littleaarch64

If you'd prefer I can send this as a patch.

Otherwise, this looks good to me; thanks for the fix!

Mark.

---->8----

arch/arm64/kvm/pauth.c +10 -14
index 8baf2a2cbdd32..e518baf570c86 100644
@@ -14,24 +14,20 @@
 
 #include <linux/kvm_host.h>
 
+#include <asm/gpr-num.h>
 #include <asm/kvm_emulate.h>
 #include <asm/pointer_auth.h>
 
-/*
- * "// This is some of my finest work" (Will Deacon, 2019-02-12)
- *
- * The jury is still out on that one.
- */
-#define REG(r)	"(0%x[" #r "] - ((0%x[" #r "] >> 4) * 6))"
-
 /* PACGA Xd, Xn, Xm */
-#define PACGA(d,n,m)						\
-	asm volatile(".inst 0x9AC03000   |"			\
-		     "(" REG(Rd) "<< 0)  |"			\
-		     "(" REG(Rn) "<< 5)  |"			\
-		     "(" REG(Rm) "<< 16)\n"			\
-		     : [Rd] "=r" ((d))				\
-		     : [Rn] "r" ((n)), [Rm] "r" ((m)))
+#define PACGA(d,n,m)				\
+	asm volatile(				\
+	__DEFINE_ASM_GPR_NUMS			\
+	"	.inst 0x9AC03000          |"	\
+	"	(.L__gpr_num_%[Rd] << 0)  |"	\
+	"	(.L__gpr_num_%[Rn] << 5)  |"	\
+	"	(.L__gpr_num_%[Rm] << 16)\n"	\
+	: [Rd] "=r" ((d))			\
+	: [Rn] "r" ((n)), [Rm] "r" ((m)))
 
 static u64 compute_pac(struct kvm_vcpu *vcpu, u64 ptr,
 		       struct ptrauth_key ikey)
author maintainer
#8 lore raw
Quoted 39 lines · Mark Rutland
Quoted 37 lines · Marc Zyngier

We still support GCC 8.x, and it appears that this toolchain does not understand "pauth" as a valid architectural extension. After all, it's only been 8 years since ARMv8.3 was released...

This results in the NV ERETAx code breaking the build, as it relies on this extention to make use of the PACGA instruction.

Work around it by hand-assembling the instruction using a mind-bending trick lifted from an old patch by Will. Magic.

Fixes e09faab353a6 ("KVM: arm64: nv: Add emulation for ERETAx instructions")
---
 arch/arm64/kvm/pauth.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)
arch/arm64/kvm/pauth.c +12 -0
index a3a5c404375b..8baf2a2cbdd3 100644
@@ -17,6 +17,22 @@
 #include <asm/kvm_emulate.h>
 #include <asm/pointer_auth.h>
 
+/*
+ * "// This is some of my finest work" (Will Deacon, 2019-02-12)
+ *
+ * The jury is still out on that one.
+ */
+#define REG(r)	"(0%x[" #r "] - ((0%x[" #r "] >> 4) * 6))"
+
+/* PACGA Xd, Xn, Xm */
+#define PACGA(d,n,m)						\
+	asm volatile(".inst 0x9AC03000   |"			\
+		     "(" REG(Rd) "<< 0)  |"			\
+		     "(" REG(Rn) "<< 5)  |"			\
Quoted 8 lines

arch/arm64/kvm/pauth.c line 30

Quoted 3 lines
arch/arm64/kvm/pauth.c +3 -0
+		     "(" REG(Rm) "<< 16)\n"			\
+		     : [Rd] "=r" ((d))				\
+		     : [Rn] "r" ((n)), [Rm] "r" ((m)))

Can you please use <asm/gpr-num.h> rather than open-coding this new REG() helper? That way this'll be consistent with the way we assemble MRS/MSR in <asm/sysreg.h>, and it avoids the few seconds of confusion trying to figure out the maths in the REG() helper (neat trick though!).

Ah, I had completely forgot about this guy. Good call. I'll fold that patchlet in and re-push the result.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.