Skip to contentSkip to conversation

arm64: Start allocating upper bits of AT_HWCAP2

PATCH Applied Mark Brown 4 participants 10 messages Jul 7, 2022 – Jul 20, 2022

Subsystems (2)
lore
Share & tools
lore

Mark Brown

author
#0 lore raw

v5 cover arm64: Start allocating upper bits of AT_HWCAP2

Currently for arm64 we expose hwcaps to userspace using the low 32 bits of AT_HWCAP and AT_HWCAP2. Due to the ever expanding capabilties of the architecture we have now allocated all the available bits in this scheme so we need to expand, either using the higher bits or adding a new AT_HWCAP3. Discussions suggest that the favoured approach for now is to start allocating the upper bits of AT_HWCAP2.

The final patch starts allocating from the upper bits of AT_HWCAP2, reporting the presence of FEAT_EBF16, this will conflict with my series converting ID_AA64ISAR1_EL1 to be generated. I've got a version based on top of that which I can send if that's convenient.

v5:
- Correct bit ranges for glibc reservation.
v4:
- Document status of upper bits in AT_HWCAP.
- Instead of creating AT_HWCAP3 just convert elf_hwcap to a bitmask and start allocating the upper bits of AT_HWCAP2.
v3:
- Expand comment on CPU_FEATURES_PER_HWCAP.
v2:
- Rebase onto v5.19-rc3.

Mark Brown (3):
arm64/hwcap: Document allocation of upper bits of AT_HWCAP arm64/cpufeature: Store elf_hwcaps as a bitmap rather than unsigned
long
arm64/hwcap: Support FEAT_EBF16

 Documentation/arm64/elf_hwcaps.rst  |  4 ++++
 arch/arm64/include/asm/cpufeature.h |  2 +-
 arch/arm64/include/asm/hwcap.h      |  3 ++-
 arch/arm64/include/uapi/asm/hwcap.h |  4 ++++
 arch/arm64/kernel/cpufeature.c      | 13 ++++++-------
 arch/arm64/kernel/cpuinfo.c         |  1 +
 6 files changed, 18 insertions(+), 9 deletions(-)

base-commit: a111daf0c53ae91e71fd2bfe7497862d14132e3e

Signature · 7 lines
2.30.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
author
#1 lore raw

PATCH v5 1/3 arm64/hwcap: Document allocation of upper bits of AT_HWCAP

The top two bits of AT_HWCAP are reserved for use by glibc and the rest of the top 32 bits are being kept unallocated for potential use by glibc. Document this in the header.

---
 arch/arm64/include/uapi/asm/hwcap.h | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/include/uapi/asm/hwcap.h +3 -0
index 4bb2cc8ac446..fd7577cf8e77 100644
@@ -19,6 +19,9 @@
 
 /*
  * HWCAP flags - for AT_HWCAP
+ *
+ * Bits 62 and 63 are reserved for use by libc.
+ * Bits 32-61 are unallocated for potential use by libc.
  */
 #define HWCAP_FP		(1 << 0)
 #define HWCAP_ASIMD		(1 << 1)
Signature · 7 lines
2.30.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Acked-by on v5 1/3
#2 lore raw

Re: [PATCH v5 1/3] arm64/hwcap: Document allocation of upper bits of AT_HWCAP

Quoted 5 lines · Mark Brown

The top two bits of AT_HWCAP are reserved for use by glibc and the rest of the top 32 bits are being kept unallocated for potential use by glibc. Document this in the header.

Quoted 21 lines
---
 arch/arm64/include/uapi/asm/hwcap.h | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/include/uapi/asm/hwcap.h +3 -0
index 4bb2cc8ac446..fd7577cf8e77 100644
@@ -19,6 +19,9 @@
 
 /*
  * HWCAP flags - for AT_HWCAP
+ *
+ * Bits 62 and 63 are reserved for use by libc.
+ * Bits 32-61 are unallocated for potential use by libc.
  */
 #define HWCAP_FP		(1 << 0)
 #define HWCAP_ASIMD		(1 << 1)
Signature · 1 line
2.30.2

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

maintainer Acked-by on v5 1/3
#3 lore raw

Re: [PATCH v5 1/3] arm64/hwcap: Document allocation of upper bits of AT_HWCAP

In reply to Mark Brown
Quoted 5 lines · Mark Brown

The top two bits of AT_HWCAP are reserved for use by glibc and the rest of the top 32 bits are being kept unallocated for potential use by glibc. Document this in the header.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author
#4 lore raw

PATCH v5 2/3 arm64/cpufeature: Store elf_hwcaps as a bitmap rather than unsigned long

In reply to Mark Brown

When we added support for AT_HWCAP2 we took advantage of the fact that we have limited hwcaps to the low 32 bits and stored it along with AT_HWCAP in a single unsigned integer. Thanks to the ever expanding capabilities of the architecture we have now allocated all 64 of the bits in an unsigned long so in preparation for adding more hwcaps convert elf_hwcap to be a bitmap instead, with 64 bits allocated to each AT_HWCAP.

There should be no functional change from this patch.

---
 arch/arm64/include/asm/cpufeature.h |  2 +-
 arch/arm64/include/asm/hwcap.h      |  2 +-
 arch/arm64/kernel/cpufeature.c      | 12 +++++-------
 3 files changed, 7 insertions(+), 9 deletions(-)
arch/arm64/include/asm/cpufeature.h +1 -1
index 14a8f3d93add..7abd6c0fa53d 100644
@@ -11,7 +11,7 @@
 #include <asm/hwcap.h>
 #include <asm/sysreg.h>
 
-#define MAX_CPU_FEATURES	64
+#define MAX_CPU_FEATURES	128
 #define cpu_feature(x)		KERNEL_HWCAP_ ## x
 
 #ifndef __ASSEMBLY__
arch/arm64/include/asm/hwcap.h +1 -1
index aa443d8f8cfb..9b5fdc114f8c 100644
@@ -85,7 +85,7 @@
 #define KERNEL_HWCAP_PACA		__khwcap_feature(PACA)
 #define KERNEL_HWCAP_PACG		__khwcap_feature(PACG)
 
-#define __khwcap2_feature(x)		(const_ilog2(HWCAP2_ ## x) + 32)
+#define __khwcap2_feature(x)		(const_ilog2(HWCAP2_ ## x) + 64)
 #define KERNEL_HWCAP_DCPODP		__khwcap2_feature(DCPODP)
 #define KERNEL_HWCAP_SVE2		__khwcap2_feature(SVE2)
 #define KERNEL_HWCAP_SVEAES		__khwcap2_feature(SVEAES)
arch/arm64/kernel/cpufeature.c +5 -7
index 8d88433de81d..5ba226f3721c 100644
@@ -91,7 +91,7 @@
 #include <asm/virt.h>
 
 /* Kernel representation of AT_HWCAP and AT_HWCAP2 */
-static unsigned long elf_hwcap __read_mostly;
+static DECLARE_BITMAP(elf_hwcap, MAX_CPU_FEATURES) __read_mostly;
 
 #ifdef CONFIG_COMPAT
 #define COMPAT_ELF_HWCAP_DEFAULT	\
@@ -3098,14 +3098,12 @@ static bool __maybe_unused __system_matches_cap(unsigned int n)
 
 void cpu_set_feature(unsigned int num)
 {
-	WARN_ON(num >= MAX_CPU_FEATURES);
-	elf_hwcap |= BIT(num);
+	set_bit(num, elf_hwcap);
 }
 
 bool cpu_have_feature(unsigned int num)
 {
-	WARN_ON(num >= MAX_CPU_FEATURES);
-	return elf_hwcap & BIT(num);
+	return test_bit(num, elf_hwcap);
 }
 EXPORT_SYMBOL_GPL(cpu_have_feature);
 
@@ -3116,12 +3114,12 @@ unsigned long cpu_get_elf_hwcap(void)
 	 * note that for userspace compatibility we guarantee that bits 62
 	 * and 63 will always be returned as 0.
 	 */
-	return lower_32_bits(elf_hwcap);
+	return elf_hwcap[0];
 }
 
 unsigned long cpu_get_elf_hwcap2(void)
 {
-	return upper_32_bits(elf_hwcap);
+	return elf_hwcap[1];
 }
 
 static void __init setup_system_capabilities(void)
Signature · 7 lines
2.30.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer Reviewed-by on v5 2/3
#5 lore raw

Re: [PATCH v5 2/3] arm64/cpufeature: Store elf_hwcaps as a bitmap rather than unsigned long

Quoted 21 lines · Mark Brown

When we added support for AT_HWCAP2 we took advantage of the fact that we have limited hwcaps to the low 32 bits and stored it along with AT_HWCAP in a single unsigned integer. Thanks to the ever expanding capabilities of the architecture we have now allocated all 64 of the bits in an unsigned long so in preparation for adding more hwcaps convert elf_hwcap to be a bitmap instead, with 64 bits allocated to each AT_HWCAP.

There should be no functional change from this patch.

---
 arch/arm64/include/asm/cpufeature.h |  2 +-
 arch/arm64/include/asm/hwcap.h      |  2 +-
 arch/arm64/kernel/cpufeature.c      | 12 +++++-------
 3 files changed, 7 insertions(+), 9 deletions(-)
arch/arm64/include/asm/cpufeature.h +0 -0
index 14a8f3d93add..7abd6c0fa53d 100644

arch/arm64/include/asm/cpufeature.h line 14

Quoted 5 lines
arch/arm64/include/asm/cpufeature.h +1 -1
@@ -11,7 +11,7 @@
 #include <asm/hwcap.h>
 #include <asm/sysreg.h>
 
-#define MAX_CPU_FEATURES	64
+#define MAX_CPU_FEATURES	128

Should this be (32 + 64)? I guess it's easier as you did since we have 128 bits in total, only that we reserved the top 32 in ELF_HWCAP.

Quoted 49 lines
arch/arm64/include/asm/cpufeature.h +0 -0
 #define cpu_feature(x)		KERNEL_HWCAP_ ## x
 
 #ifndef __ASSEMBLY__
arch/arm64/include/asm/hwcap.h +1 -1
index aa443d8f8cfb..9b5fdc114f8c 100644
@@ -85,7 +85,7 @@
 #define KERNEL_HWCAP_PACA		__khwcap_feature(PACA)
 #define KERNEL_HWCAP_PACG		__khwcap_feature(PACG)
 
-#define __khwcap2_feature(x)		(const_ilog2(HWCAP2_ ## x) + 32)
+#define __khwcap2_feature(x)		(const_ilog2(HWCAP2_ ## x) + 64)
 #define KERNEL_HWCAP_DCPODP		__khwcap2_feature(DCPODP)
 #define KERNEL_HWCAP_SVE2		__khwcap2_feature(SVE2)
 #define KERNEL_HWCAP_SVEAES		__khwcap2_feature(SVEAES)
arch/arm64/kernel/cpufeature.c +3 -5
index 8d88433de81d..5ba226f3721c 100644
@@ -91,7 +91,7 @@
 #include <asm/virt.h>
 
 /* Kernel representation of AT_HWCAP and AT_HWCAP2 */
-static unsigned long elf_hwcap __read_mostly;
+static DECLARE_BITMAP(elf_hwcap, MAX_CPU_FEATURES) __read_mostly;
 
 #ifdef CONFIG_COMPAT
 #define COMPAT_ELF_HWCAP_DEFAULT	\
@@ -3098,14 +3098,12 @@ static bool __maybe_unused __system_matches_cap(unsigned int n)
 
 void cpu_set_feature(unsigned int num)
 {
-	WARN_ON(num >= MAX_CPU_FEATURES);
-	elf_hwcap |= BIT(num);
+	set_bit(num, elf_hwcap);
 }
 
 bool cpu_have_feature(unsigned int num)
 {
-	WARN_ON(num >= MAX_CPU_FEATURES);
-	return elf_hwcap & BIT(num);
+	return test_bit(num, elf_hwcap);
 }
 EXPORT_SYMBOL_GPL(cpu_have_feature);
 
@@ -3116,12 +3114,12 @@ unsigned long cpu_get_elf_hwcap(void)
 	 * note that for userspace compatibility we guarantee that bits 62
 	 * and 63 will always be returned as 0.

arch/arm64/kernel/cpufeature.c unsigned long cpu_get_elf_hwcap(void) · line 3123

Quoted 10 lines
arch/arm64/kernel/cpufeature.c +2 -2
 	 */
-	return lower_32_bits(elf_hwcap);
+	return elf_hwcap[0];
 }
 
 unsigned long cpu_get_elf_hwcap2(void)
 {
-	return upper_32_bits(elf_hwcap);
+	return elf_hwcap[1];
 }

This is fine on the assumption that DECLARE_BITMAP is always an array of longs. I couldn't see any documentation on this but at least the set_bit() etc. API only works on 'long *' pointers.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author
#6 lore raw

Re: [PATCH v5 2/3] arm64/cpufeature: Store elf_hwcaps as a bitmap rather than unsigned long

Quoted 7 lines · Catalin Marinas

arch/arm64/include/asm/cpufeature.h line 14

Quoted 2 lines · Mark Brown
arch/arm64/include/asm/cpufeature.h +1 -1
-#define MAX_CPU_FEATURES	64
+#define MAX_CPU_FEATURES	128

Should this be (32 + 64)? I guess it's easier as you did since we have 128 bits in total, only that we reserved the top 32 in ELF_HWCAP.

We need to have the full 64 bits allocated for AT_HWCAP storage even if nothing uses the top 32 so that we can return the elements in the array, otherwise we need logic for fishing variably sized silces of bits out of the bitmap which seems more trouble than it's worth.

Quoted 9 lines

arch/arm64/kernel/cpufeature.c unsigned long cpu_get_elf_hwcap(void) · line 3123

Quoted 5 lines

unsigned long cpu_get_elf_hwcap2(void)
{

arch/arm64/kernel/cpufeature.c +1 -1
-	return upper_32_bits(elf_hwcap);
+	return elf_hwcap[1];
 }

This is fine on the assumption that DECLARE_BITMAP is always an array of longs. I couldn't see any documentation on this but at least the set_bit() etc. API only works on 'long *' pointers.

If someone changes DECLARE_BITMAP() they're going to have to be very, very careful.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author
#7 lore raw

PATCH v5 3/3 arm64/hwcap: Support FEAT_EBF16

In reply to Mark Brown

The v9.2 feature FEAT_EBF16 provides support for an extended BFloat16 mode. Allow userspace to discover system support for this feature by adding a hwcap for it.

---
 Documentation/arm64/elf_hwcaps.rst  | 4 ++++
 arch/arm64/include/asm/hwcap.h      | 1 +
 arch/arm64/include/uapi/asm/hwcap.h | 1 +
 arch/arm64/kernel/cpufeature.c      | 1 +
 arch/arm64/kernel/cpuinfo.c         | 1 +
 5 files changed, 8 insertions(+)
Documentation/arm64/elf_hwcaps.rst +4 -0
index 3d116fb536c5..31fc10b833dd 100644
@@ -301,6 +301,10 @@ HWCAP2_WFXT
 
     Functionality implied by ID_AA64ISAR2_EL1.WFXT == 0b0010.
 
+HWCAP2_EBF16
+
+    Functionality implied by ID_AA64ISAR1_EL1.BF16 == 0b0010.
+
 4. Unused AT_HWCAP bits
 -----------------------
 
arch/arm64/include/asm/hwcap.h +1 -0
index 9b5fdc114f8c..cef4ae7a3d8b 100644
@@ -118,6 +118,7 @@
 #define KERNEL_HWCAP_SME_F32F32		__khwcap2_feature(SME_F32F32)
 #define KERNEL_HWCAP_SME_FA64		__khwcap2_feature(SME_FA64)
 #define KERNEL_HWCAP_WFXT		__khwcap2_feature(WFXT)
+#define KERNEL_HWCAP_EBF16		__khwcap2_feature(EBF16)
 
 /*
  * This yields a mask that user programs can use to figure out what
arch/arm64/include/uapi/asm/hwcap.h +1 -0
index fd7577cf8e77..1ad2568a2569 100644
@@ -91,5 +91,6 @@
 #define HWCAP2_SME_F32F32	(1 << 29)
 #define HWCAP2_SME_FA64		(1 << 30)
 #define HWCAP2_WFXT		(1UL << 31)
+#define HWCAP2_EBF16		(1UL << 32)
 
 #endif /* _UAPI__ASM_HWCAP_H */
arch/arm64/kernel/cpufeature.c +1 -0
index 5ba226f3721c..c84f97797985 100644
@@ -2623,6 +2623,7 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = {
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_FRINTTS_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_FRINT),
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_SB_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_SB),
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_BF16_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_BF16),
+	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_BF16_SHIFT, 4, FTR_UNSIGNED, 2, CAP_HWCAP, KERNEL_HWCAP_EBF16),
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_DGH_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_DGH),
 	HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_I8MM_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_I8MM),
 	HWCAP_CAP(SYS_ID_AA64MMFR2_EL1, ID_AA64MMFR2_AT_SHIFT, 4, FTR_UNSIGNED, 1, CAP_HWCAP, KERNEL_HWCAP_USCAT),
arch/arm64/kernel/cpuinfo.c +1 -0
index 8eff0a34ffd4..2ca131243c30 100644
@@ -107,6 +107,7 @@ static const char *const hwcap_str[] = {
 	[KERNEL_HWCAP_SME_F32F32]	= "smef32f32",
 	[KERNEL_HWCAP_SME_FA64]		= "smefa64",
 	[KERNEL_HWCAP_WFXT]		= "wfxt",
+	[KERNEL_HWCAP_EBF16]		= "ebf16",
 };
 
 #ifdef CONFIG_COMPAT
Signature · 7 lines
2.30.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer Reviewed-by on v5 3/3
#8 lore raw

Re: [PATCH v5 3/3] arm64/hwcap: Support FEAT_EBF16

arch/arm64/include/uapi/asm/hwcap.h line 94

Quoted 9 lines · Mark Brown
arch/arm64/include/uapi/asm/hwcap.h +1 -0
index fd7577cf8e77..1ad2568a2569 100644
@@ -91,5 +91,6 @@
 #define HWCAP2_SME_F32F32	(1 << 29)
 #define HWCAP2_SME_FA64		(1 << 30)
 #define HWCAP2_WFXT		(1UL << 31)
+#define HWCAP2_EBF16		(1UL << 32)

This works for me:

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

maintainer Applied
#9 lore raw
In reply to Mark Brown
Quoted 8 lines · Mark Brown

Currently for arm64 we expose hwcaps to userspace using the low 32 bits of AT_HWCAP and AT_HWCAP2. Due to the ever expanding capabilties of the architecture we have now allocated all the available bits in this scheme so we need to expand, either using the higher bits or adding a new AT_HWCAP3. Discussions suggest that the favoured approach for now is to start allocating the upper bits of AT_HWCAP2.

[...]

Applied to arm64 (for-next/cpufeature), thanks!

[1/3] arm64/hwcap: Document allocation of upper bits of AT_HWCAP

      https://git.kernel.org/arm64/c/d3e4a9d30804
[2/3] arm64/cpufeature: Store elf_hwcaps as a bitmap rather than unsigned long
      https://git.kernel.org/arm64/c/60c868eff2bc
[3/3] arm64/hwcap: Support FEAT_EBF16
      https://git.kernel.org/arm64/c/a6a468f50d6a

Cheers,

Signature · 10 lines
Will

https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel