Skip to contentSkip to conversation

arm64: expose CPU prefetch and cache modulation controls

RFC PATCH In review Koba Ko 3 participants 9 messages Aug 25 – Oct 15

Subsystems (3)
lore
Share & tools
lore

Koba Ko

author
#0 lore raw

v2 cover arm64: expose CPU prefetch and cache modulation controls

This series adds CONFIG_ARM64_CPUMOD, a default-off arm64 interface for controlled performance characterization of selected implementation-defined CPU prefetch and cache-management fields. It exposes a small set of named, per-CPU attributes under the path below, with writable control fields range-checked before register state is changed:

/sys/devices/system/cpu/cpuN/cpumod/

The directory is created for online CPUs whose MIDR matches a recognized Grace or Vera profile, with common and profile-specific attributes selected for that CPU. Register accesses execute on the CPU that owns the sysfs instance, and CPU hotplug creates or removes the per-CPU subtree as the CPU transitions online or offline. Unsupported CPU profiles are skipped without blocking module load or CPU hotplug.

The register-backed attributes are root-only read/write files. Reads return the live value from the synchronous target-CPU callback rather than relying on persistent per-attribute sysfs storage. affected_cpus remains world-readable and read-only.

Changes since v1:
- Remove redundant GPL boilerplate from the new source file while retaining the SPDX identifier and NVIDIA copyright notice.
- Address Rob Herring's ABI feedback by adding standard Documentation/ABI/testing/sysfs-devices-system-cpu records and keeping the arm64 guide as a pointer to that ABI, without a second attribute list.
- Address Breno Leitao's access/readback feedback by returning live synchronous target-CPU register reads, changing the six register-backed controls to mode 0600, documenting affected_cpus as 0444, and adding deterministic selftest mode checks.

Koba Ko (3):
arm64: add CPU prefetch and cache modulation sysfs interface Documentation/arch/arm64: document arm_cpumod
selftests: arm64: add arm_cpumod kselftest

 .../ABI/testing/sysfs-devices-system-cpu      |  77 +++
 Documentation/arch/arm64/arm-cpumod.rst       |  66 +++
 Documentation/arch/arm64/index.rst            |   1 +
 arch/arm64/Kconfig                            |  17 +
 arch/arm64/kernel/Makefile                    |   1 +
 arch/arm64/kernel/arm_cpumod.c                | 458 +++++++++++++++
 arch/arm64/kernel/arm_cpumod_internal.h       |  93 +++
 tools/testing/selftests/arm64/Makefile        |   2 +-
 .../selftests/arm64/arm_cpumod/Makefile       |   5 +
 .../selftests/arm64/arm_cpumod/arm_cpumod.sh  | 543 ++++++++++++++++++
 10 files changed, 1262 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/arch/arm64/arm-cpumod.rst
 create mode 100644 arch/arm64/kernel/arm_cpumod.c
 create mode 100644 arch/arm64/kernel/arm_cpumod_internal.h
 create mode 100644 tools/testing/selftests/arm64/arm_cpumod/Makefile
 create mode 100755 tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh
Signature · 1 line
2.43.0
author
#7 lore raw

RFC PATCH v2 2/3 Documentation/arch/arm64: document arm_cpumod

Document the ABI intent and scope for the new arm64 cpumod interface. The sysfs directory is per-CPU and is created only for recognized Grace and Vera profiles:

/sys/devices/system/cpu/cpuN/cpumod/

CPUs with unsupported MIDRs are skipped without blocking module load or CPU hotplug and do not receive a cpumod directory.

Describe the common attributes, the Grace- and Vera-specific attributes, and the accepted range for each writable field. Document affected_cpus as world-readable read-only (0444), and the six register-backed controls as root-only read/write (0600). Out-of-range writes are documented as failing before target CPU register state is changed.

Document the CPU hotplug lifecycle explicitly: the cpumod directory is present only while a supported CPU is online, is removed when that CPU goes offline, and is recreated when it returns online. Reads and writes therefore require an online target CPU.

Also document the firmware prerequisite: EL1 must be allowed to access the relevant implementation control registers. The interface remains an RFC for controlled performance characterization, is not a general production tuning ABI, and should stay disabled by default on production systems.

---
 .../ABI/testing/sysfs-devices-system-cpu      | 77 +++++++++++++++++++
 Documentation/arch/arm64/arm-cpumod.rst       | 66 ++++++++++++++++
 Documentation/arch/arm64/index.rst            |  1 +
 3 files changed, 144 insertions(+)
 create mode 100644 Documentation/arch/arm64/arm-cpumod.rst
Documentation/ABI/testing/sysfs-devices-system-cpu +77 -0
index 82d10d556cc89..7275399e33435 100644
@@ -806,3 +806,80 @@ Date:		Nov 2022
 Contact:	Linux kernel mailing list <[email protected]>
 Description:
 		(RO) the list of CPUs that can be brought online.
+
+What:		/sys/devices/system/cpu/cpuX/cpumod/affected_cpus
+Date:		August 2026
+Contact:	Koba Ko <[email protected]>
+Contact:	Linux ARM Kernel mailing list <[email protected]>
+Description:
+		(RO) The decimal logical CPU identifier for this cpumod sysfs
+		instance. The file is present only while CPU X is online and its
+		MIDR matches a supported Grace or Vera profile. The file is
+		world-readable and read-only (0444).
+
+What:		/sys/devices/system/cpu/cpuX/cpumod/pf_dis
+Date:		August 2026
+Contact:	Koba Ko <[email protected]>
+Contact:	Linux ARM Kernel mailing list <[email protected]>
+Description:
+		(RW) Hardware prefetch disable control for CPU X. The file is
+		present only while a supported Grace or Vera CPU is online. Reads
+		return a decimal value. Writes accept decimal values 0 and 1.
+		Invalid writes fail with -EINVAL without changing the target CPU
+		register. The file is root-only read/write (0600).
+
+What:		/sys/devices/system/cpu/cpuX/cpumod/pf_mode
+Date:		August 2026
+Contact:	Koba Ko <[email protected]>
+Contact:	Linux ARM Kernel mailing list <[email protected]>
+Description:
+		(RW) Hardware prefetch aggressiveness mode for CPU X. The file is
+		present only while a supported Grace or Vera CPU is online. Reads
+		return a decimal value. Writes accept decimal values 0 through 9;
+		values 10 through 15 are reserved and rejected. Invalid writes fail
+		with -EINVAL without changing the target CPU register. The file is
+		root-only read/write (0600).
+
+What:		/sys/devices/system/cpu/cpuX/cpumod/cbusy_filter_threshold
+Date:		August 2026
+Contact:	Koba Ko <[email protected]>
+Contact:	Linux ARM Kernel mailing list <[email protected]>
+Description:
+		(RW) Grace CPU cache-busy filter threshold. The file is present only
+		while a supported Grace CPU is online. Reads return a decimal value.
+		Writes accept decimal values 0 through 3. Invalid writes fail with
+		-EINVAL without changing the target CPU register. The file is
+		root-only read/write (0600).
+
+What:		/sys/devices/system/cpu/cpuX/cpumod/cbusy_filter_window
+Date:		August 2026
+Contact:	Koba Ko <[email protected]>
+Contact:	Linux ARM Kernel mailing list <[email protected]>
+Description:
+		(RW) Grace CPU cache-busy filter window. The file is present only
+		while a supported Grace CPU is online. Reads return a decimal value.
+		Writes accept decimal values 0 through 3. Invalid writes fail with
+		-EINVAL without changing the target CPU register. The file is
+		root-only read/write (0600).
+
+What:		/sys/devices/system/cpu/cpuX/cpumod/cmc_min_ways
+Date:		August 2026
+Contact:	Koba Ko <[email protected]>
+Contact:	Linux ARM Kernel mailing list <[email protected]>
+Description:
+		(RW) Grace CPU cache-management minimum-way control. The file is
+		present only while a supported Grace CPU is online. Reads return a
+		decimal value. Writes accept decimal values 0 through 7. Invalid
+		writes fail with -EINVAL without changing the target CPU register.
+		The file is root-only read/write (0600).
+
+What:		/sys/devices/system/cpu/cpuX/cpumod/l2spr_cmc_max_ways
+Date:		August 2026
+Contact:	Koba Ko <[email protected]>
+Contact:	Linux ARM Kernel mailing list <[email protected]>
+Description:
+		(RW) Vera CPU cache-management maximum-way control. The file is
+		present only while a supported Vera CPU is online. Reads return a
+		decimal value. Writes accept decimal values 0 through 7. Invalid
+		writes fail with -EINVAL without changing the target CPU register.
+		The file is root-only read/write (0600).
Documentation/arch/arm64/arm-cpumod.rst new file +66 -0

66 lines not on this page · raw message

Documentation/arch/arm64/index.rst +1 -0
index 98052b4ef4a1e..9e085ce55d9c4 100644
@@ -8,6 +8,7 @@ ARM64 Architecture
     :maxdepth: 1
 
     acpi_object_usage
+    arm-cpumod
     amu
     arm-acpi
     arm-cca
Signature · 1 line
2.43.0
author
#1 lore raw

RFC PATCH v2 1/3 arm64: add CPU prefetch and cache modulation sysfs interface

In reply to Koba Ko

Performance characterization on arm64 systems sometimes needs a way to inspect and bracket selected implementation-defined prefetch and cache controls without exposing raw register contents. Add an RFC-only arm64 cpumod interface that presents a small set of named, range-checked per-CPU sysfs attributes under each CPU device.

Establish the Kconfig and Makefile plumbing, CPU profile detection for Grace and Vera from the target CPU's MIDR, register-field descriptors, per-CPU kobject and attribute helpers, locking, and target-CPU read/write callbacks. Common attributes are exposed for all supported profiles, while Grace- and Vera-specific fields are exposed only for the matching profile. Place the option at the end of the top-level Kernel Features menu because these implementation controls are not tied to an Arm architecture revision.

Per-CPU cpumod sysfs state follows the CPU lifecycle. Module initialization populates subtrees for every online CPU while holding cpus_read_lock(), so initial enumeration and CPU hotplug state registration are serialized against concurrent topology changes. Offline CPUs are intentionally skipped during initial setup because target-CPU MIDR detection requires an IPI-capable online CPU; a later online callback creates their subtree.

Profile detection reads MIDR through a synchronous callback on the CPU owning the sysfs subtree and returns nonzero only when that dispatch fails, so no IPI error can be hidden. After a successful read, retain CPUMOD_PROFILE_UNKNOWN as the unsupported-profile sentinel. Subtree creation skips that profile during both initial online enumeration and later CPUHP online callbacks without blocking module load or CPU hotplug. Object and sysfs setup failures remain fatal and are propagated.

Keep the dynamically allocated kobject state in a private per-CPU pointer instead of the CPU device's generic driver-data slot. The online callback is idempotent when that pointer already records a subtree. The offline callback clears the private pointer before dropping the kobject reference, allowing the release callback to retain ownership of the final free.

Register-backed controls are root-only read/write files. Reads synchronously execute on the target CPU and return the live register field value from that callback. Writes remain range-checked and continue to log the target-CPU readback after the register update.

Register the dynamic CPUHP state with cpuhp_setup_state_nocalls_cpuslocked(). The nocalls form avoids replaying startup callbacks for CPUs already initialized by the explicit online enumeration. On init failure, release the CPU read lock, destroy all subtrees created so far, and return the error. Module exit removes the CPUHP state with the matching nocalls helper and tears down all remaining per-CPU state across possible CPUs.

---
 arch/arm64/Kconfig                      |  17 +
 arch/arm64/kernel/Makefile              |   1 +
 arch/arm64/kernel/arm_cpumod.c          | 458 ++++++++++++++++++++++++
 arch/arm64/kernel/arm_cpumod_internal.h |  93 +++++
 4 files changed, 569 insertions(+)
 create mode 100644 arch/arm64/kernel/arm_cpumod.c
 create mode 100644 arch/arm64/kernel/arm_cpumod_internal.h
arch/arm64/Kconfig +17 -0
index b3afe0688919b..5e4ada4884b89 100644
@@ -2417,6 +2417,23 @@ config ARM64_CONTPTE
 	  bit, for any mappings that meet the size and alignment requirements.
 	  This reduces TLB pressure and improves performance.
 
+config ARM64_CPUMOD
+	tristate "Arm CPU prefetch modulation sysfs controls"
+	help
+	  Expose selected CPU implementation control register fields through
+	  per-CPU sysfs attributes for performance characterization.
+
+	  The exposed controls allow hardware prefetch and cache-management
+	  policy fields to be inspected and adjusted on CPUs where firmware
+	  permits EL1 access to the relevant implementation control registers.
+	  Systems where firmware traps or blocks these writes cannot use this
+	  interface.
+
+	  This interface is intended for controlled performance evaluation and
+	  should not be enabled by default on production systems.
+
+	  If unsure, say N.
+
 endmenu # "Kernel Features"
 
 menu "Boot options"
arch/arm64/kernel/Makefile +1 -0
index d2690c3ec5288..f180a52284e7d 100644
@@ -69,6 +69,7 @@ obj-$(CONFIG_VMCORE_INFO)		+= vmcore_info.o
 obj-$(CONFIG_ARM_SDE_INTERFACE)		+= sdei.o
 obj-$(CONFIG_ARM64_PTR_AUTH)		+= pointer_auth.o
 obj-$(CONFIG_ARM64_MPAM)		+= mpam.o
+obj-$(CONFIG_ARM64_CPUMOD)		+= arm_cpumod.o
 obj-$(CONFIG_ARM64_MTE)			+= mte.o
 obj-y					+= vdso-wrap.o
 obj-$(CONFIG_COMPAT_VDSO)		+= vdso32-wrap.o
arch/arm64/kernel/arm_cpumod.c new file +458 -0

458 lines not on this page · raw message

arch/arm64/kernel/arm_cpumod_internal.h new file +93 -0

93 lines not on this page · raw message

Signature · 1 line
2.43.0
author
#8 lore raw

RFC PATCH v2 3/3 selftests: arm64: add arm_cpumod kselftest

In reply to Koba Ko

Add a kselftest for the arm_cpumod sysfs ABI. The test loads or reuses the module and selects either one target CPU or a snapshot of all online CPUs. In single-CPU mode, skip an offline target before module validation. Observe the per-CPU online attribute first. If it is absent, immediately require the CPU directory to still exist before treating the CPU as non-hotpluggable and online. If the attribute exists, it must be readable and contain 1.

After module load, recheck the online state before classifying a selected CPU without a cpumod directory as an unsupported profile. Fail if the CPU went offline or disappeared, log and skip CPUs that remain online without a directory, and require at least one selected CPU to expose a supported Grace or Vera profile. Return KSFT_SKIP if none do.

For supported CPUs, recognize the exposed Grace or Vera profile-specific layout and validate the matching attributes. Only ARM_CPUMOD_EXPECT_PROFILE supplies an external profile expectation.

Verify read paths by checking that attributes return decimal values and that affected_cpus identifies the current CPU as a decimal CPU number. Validate affected_cpus mode 444 and every visible register-backed control mode 600 with stat before functional write checks. Optional valid-write testing writes each current value back. Optional invalid-write testing checks selected out-of-range values and confirms that the prior value remains unchanged.

Both write paths are opt-in. Valid writeback can alter implementation-defined CPU controls; invalid-write checks exercise rejection through the same firmware-sensitive interface, but are expected not to update state. Cover all-online-CPU mode through /sys/devices/system/cpu/online and, when the test loaded the module, verify cleanup by removing it and checking that the selected CPU subtrees disappear. The test does not offline or online CPUs.

---
 tools/testing/selftests/arm64/Makefile        |   2 +-
 .../selftests/arm64/arm_cpumod/Makefile       |   5 +
 .../selftests/arm64/arm_cpumod/arm_cpumod.sh  | 543 ++++++++++++++++++
 3 files changed, 549 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/arm64/arm_cpumod/Makefile
 create mode 100755 tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh
tools/testing/selftests/arm64/Makefile +1 -1
index e456f3b62fa13..0873be5af0369 100644
@@ -4,7 +4,7 @@
 ARCH ?= $(shell uname -m 2>/dev/null || echo not)
 
 ifneq (,$(filter $(ARCH),aarch64 arm64))
-ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs
+	ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs arm_cpumod
 else
 ARM64_SUBTARGETS :=
 endif
tools/testing/selftests/arm64/arm_cpumod/Makefile new file +5 -0
new file mode 100644
index 0000000000000..db0341cdbc621
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+
+TEST_PROGS := arm_cpumod.sh
+
+include ../../lib.mk
tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh new file +543 -0

543 lines not on this page · raw message

Signature · 1 line
2.43.0
maintainer
#2 lore raw

Re: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface

In reply to Koba Ko
Quoted 77 lines · Koba Ko

Performance characterization on arm64 systems sometimes needs a way to inspect and bracket selected implementation-defined prefetch and cache controls without exposing raw register contents. Add an RFC-only arm64 cpumod interface that presents a small set of named, range-checked per-CPU sysfs attributes under each CPU device.

Establish the Kconfig and Makefile plumbing, CPU profile detection for Grace and Vera from the target CPU's MIDR, register-field descriptors, per-CPU kobject and attribute helpers, locking, and target-CPU read/write callbacks. Common attributes are exposed for all supported profiles, while Grace- and Vera-specific fields are exposed only for the matching profile. Place the option at the end of the top-level Kernel Features menu because these implementation controls are not tied to an Arm architecture revision.

Per-CPU cpumod sysfs state follows the CPU lifecycle. Module initialization populates subtrees for every online CPU while holding cpus_read_lock(), so initial enumeration and CPU hotplug state registration are serialized against concurrent topology changes. Offline CPUs are intentionally skipped during initial setup because target-CPU MIDR detection requires an IPI-capable online CPU; a later online callback creates their subtree.

Profile detection reads MIDR through a synchronous callback on the CPU owning the sysfs subtree and returns nonzero only when that dispatch fails, so no IPI error can be hidden. After a successful read, retain CPUMOD_PROFILE_UNKNOWN as the unsupported-profile sentinel. Subtree creation skips that profile during both initial online enumeration and later CPUHP online callbacks without blocking module load or CPU hotplug. Object and sysfs setup failures remain fatal and are propagated.

Keep the dynamically allocated kobject state in a private per-CPU pointer instead of the CPU device's generic driver-data slot. The online callback is idempotent when that pointer already records a subtree. The offline callback clears the private pointer before dropping the kobject reference, allowing the release callback to retain ownership of the final free.

Register-backed controls are root-only read/write files. Reads synchronously execute on the target CPU and return the live register field value from that callback. Writes remain range-checked and continue to log the target-CPU readback after the register update.

Register the dynamic CPUHP state with cpuhp_setup_state_nocalls_cpuslocked(). The nocalls form avoids replaying startup callbacks for CPUs already initialized by the explicit online enumeration. On init failure, release the CPU read lock, destroy all subtrees created so far, and return the error. Module exit removes the CPUHP state with the matching nocalls helper and tears down all remaining per-CPU state across possible CPUs.

---
 arch/arm64/Kconfig                      |  17 +
 arch/arm64/kernel/Makefile              |   1 +
 arch/arm64/kernel/arm_cpumod.c          | 458 ++++++++++++++++++++++++
 arch/arm64/kernel/arm_cpumod_internal.h |  93 +++++
 4 files changed, 569 insertions(+)
 create mode 100644 arch/arm64/kernel/arm_cpumod.c
 create mode 100644 arch/arm64/kernel/arm_cpumod_internal.h
arch/arm64/Kconfig +6 -0
index b3afe0688919b..5e4ada4884b89 100644
@@ -2417,6 +2417,23 @@ config ARM64_CONTPTE
 	  bit, for any mappings that meet the size and alignment requirements.
 	  This reduces TLB pressure and improves performance.
 
+config ARM64_CPUMOD
+	tristate "Arm CPU prefetch modulation sysfs controls"
+	help
+	  Expose selected CPU implementation control register fields through
+	  per-CPU sysfs attributes for performance characterization.
+

arch/arm64/Kconfig config ARM64_CONTPTE · line 2435

Quoted 10 lines
arch/arm64/Kconfig +10 -0
+	  The exposed controls allow hardware prefetch and cache-management
+	  policy fields to be inspected and adjusted on CPUs where firmware
+	  permits EL1 access to the relevant implementation control registers.
+	  Systems where firmware traps or blocks these writes cannot use this
+	  interface.
+
+	  This interface is intended for controlled performance evaluation and
+	  should not be enabled by default on production systems.
+
+	  If unsure, say N.

We don't tend to support these sort of implementation-defined features as:

- They're generally not something we can virtualise
- The registers should not be accessible to the kernel in production
- It's not clear what other bits are exposed in the registers
- These sort of controls can't always be toggled while the memory system is active
- We can't test them or maintain the zoo of interfaces we'd end up with if we started merging them

So exposing the registers directly to userspace doesn't feel like a particularly good idea to me.

Will

author
#3 lore raw

Re: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface

Hi Will,

Thanks for the feedback. I understand that your concern is about exposing implementation-defined registers to userspace, not simply choosing sysfs versus debugfs. I'm not planning to submit v3.

For controlled performance characterization, is there an existing kernel mechanism or a different design you would recommend? Or do you think these experimental controls are better kept out of tree?

Thanks,
Koba

#4 lore raw

Re: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface

Quoted 3 lines · Koba Ko

For controlled performance characterization, is there an existing kernel mechanism or a different design you would recommend? Or do you think these experimental controls are better kept out of tree?

As a user of this patchset, I am trying hard to avoid carrying it out of tree, either as patches or as a driver. Out-of-tree code makes kernel rebases and rollouts slower, and debugging more complex.

Is there a way to keep a generic interface available under CONFIG_EXPERT, or something similar, so that users who want to experiment can use it?

We could even taint the kernel when the interface is used, to make it clear that the user is messing with things they should not (!?).

Thanks,
--breno

author
#5 lore raw

Re: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface

Hi Breno,

Thanks for explaining the rebase, rollout and debugging costs of carrying this out of tree. That's useful context from someone using the patchset.

CONFIG_EXPERT plus tainting on use could make experimental use more explicit. Those safeguards alone would not establish that firmware access, virtualization and runtime register updates are handled safely; the local restrictions still need validation against the hardware and firmware requirements.

I'd like to hear Will's view on whether this opt-in approach could be acceptable in principle, and what requirements it would need to meet. I'm not planning to submit v3.

Thanks,
Koba

maintainer
#6 lore raw

Re: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface

Quoted 9 lines · Koba Ko

Thanks for explaining the rebase, rollout and debugging costs of carrying this out of tree. That's useful context from someone using the patchset.

CONFIG_EXPERT plus tainting on use could make experimental use more explicit. Those safeguards alone would not establish that firmware access, virtualization and runtime register updates are handled safely; the local restrictions still need validation against the hardware and firmware requirements.

Quoted 3 lines

I'd like to hear Will's view on whether this opt-in approach could be acceptable in principle, and what requirements it would need to meet. I'm not planning to submit v3.

No, this isn't something we should be maintaining in the upstream kernel and if we take this then we'll suddenly grow a zoo full of this stuff.

If Nvidia has imp-def controls that they believe are useful to expose to userspace, then I suggest working with Arm to get something architectural that we can support across different pieces of hardware and which would integrate properly with the rest of the architecture (including things like virtualisation and debug).

Will