arm64: expose CPU prefetch and cache modulation controls
Subsystems (3)
v2 cover arm64: expose CPU prefetch and cache modulation controls
This series adds CONFIG_ARM64_CPUMOD, a default-off arm64 interface for controlled performance characterization of selected implementation-defined CPU prefetch and cache-management fields. It exposes a small set of named, per-CPU attributes under the path below, with writable control fields range-checked before register state is changed:
/sys/devices/system/cpu/cpuN/cpumod/
The directory is created for online CPUs whose MIDR matches a recognized Grace or Vera profile, with common and profile-specific attributes selected for that CPU. Register accesses execute on the CPU that owns the sysfs instance, and CPU hotplug creates or removes the per-CPU subtree as the CPU transitions online or offline. Unsupported CPU profiles are skipped without blocking module load or CPU hotplug.
The register-backed attributes are root-only read/write files. Reads return the live value from the synchronous target-CPU callback rather than relying on persistent per-attribute sysfs storage. affected_cpus remains world-readable and read-only.
Changes since v1:
- Remove redundant GPL boilerplate from the new source file while retaining the SPDX identifier and NVIDIA copyright notice.
- Address Rob Herring's ABI feedback by adding standard Documentation/ABI/testing/sysfs-devices-system-cpu records and keeping the arm64 guide as a pointer to that ABI, without a second attribute list.
- Address Breno Leitao's access/readback feedback by returning live synchronous target-CPU register reads, changing the six register-backed controls to mode 0600, documenting affected_cpus as 0444, and adding deterministic selftest mode checks.
Koba Ko (3):
arm64: add CPU prefetch and cache modulation sysfs interface Documentation/arch/arm64: document arm_cpumod
selftests: arm64: add arm_cpumod kselftest
.../ABI/testing/sysfs-devices-system-cpu | 77 +++ Documentation/arch/arm64/arm-cpumod.rst | 66 +++ Documentation/arch/arm64/index.rst | 1 + arch/arm64/Kconfig | 17 + arch/arm64/kernel/Makefile | 1 + arch/arm64/kernel/arm_cpumod.c | 458 +++++++++++++++ arch/arm64/kernel/arm_cpumod_internal.h | 93 +++ tools/testing/selftests/arm64/Makefile | 2 +- .../selftests/arm64/arm_cpumod/Makefile | 5 + .../selftests/arm64/arm_cpumod/arm_cpumod.sh | 543 ++++++++++++++++++ 10 files changed, 1262 insertions(+), 1 deletion(-) create mode 100644 Documentation/arch/arm64/arm-cpumod.rst create mode 100644 arch/arm64/kernel/arm_cpumod.c create mode 100644 arch/arm64/kernel/arm_cpumod_internal.h create mode 100644 tools/testing/selftests/arm64/arm_cpumod/Makefile create mode 100755 tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh
Signature · 1 line
2.43.0
RFC PATCH v2 2/3 Documentation/arch/arm64: document arm_cpumod
Document the ABI intent and scope for the new arm64 cpumod interface. The sysfs directory is per-CPU and is created only for recognized Grace and Vera profiles:
/sys/devices/system/cpu/cpuN/cpumod/
CPUs with unsupported MIDRs are skipped without blocking module load or CPU hotplug and do not receive a cpumod directory.
Describe the common attributes, the Grace- and Vera-specific attributes, and the accepted range for each writable field. Document affected_cpus as world-readable read-only (0444), and the six register-backed controls as root-only read/write (0600). Out-of-range writes are documented as failing before target CPU register state is changed.
Document the CPU hotplug lifecycle explicitly: the cpumod directory is present only while a supported CPU is online, is removed when that CPU goes offline, and is recreated when it returns online. Reads and writes therefore require an online target CPU.
Also document the firmware prerequisite: EL1 must be allowed to access the relevant implementation control registers. The interface remains an RFC for controlled performance characterization, is not a general production tuning ABI, and should stay disabled by default on production systems.
--- .../ABI/testing/sysfs-devices-system-cpu | 77 +++++++++++++++++++ Documentation/arch/arm64/arm-cpumod.rst | 66 ++++++++++++++++ Documentation/arch/arm64/index.rst | 1 + 3 files changed, 144 insertions(+) create mode 100644 Documentation/arch/arm64/arm-cpumod.rst
Documentation/ABI/testing/sysfs-devices-system-cpu +77 -0
@@ -806,3 +806,80 @@ Date: Nov 2022 Contact: Linux kernel mailing list <[email protected]> Description: (RO) the list of CPUs that can be brought online. + +What: /sys/devices/system/cpu/cpuX/cpumod/affected_cpus +Date: August 2026 +Contact: Koba Ko <[email protected]> +Contact: Linux ARM Kernel mailing list <[email protected]> +Description: + (RO) The decimal logical CPU identifier for this cpumod sysfs + instance. The file is present only while CPU X is online and its + MIDR matches a supported Grace or Vera profile. The file is + world-readable and read-only (0444). + +What: /sys/devices/system/cpu/cpuX/cpumod/pf_dis +Date: August 2026 +Contact: Koba Ko <[email protected]> +Contact: Linux ARM Kernel mailing list <[email protected]> +Description: + (RW) Hardware prefetch disable control for CPU X. The file is + present only while a supported Grace or Vera CPU is online. Reads + return a decimal value. Writes accept decimal values 0 and 1. + Invalid writes fail with -EINVAL without changing the target CPU + register. The file is root-only read/write (0600). + +What: /sys/devices/system/cpu/cpuX/cpumod/pf_mode +Date: August 2026 +Contact: Koba Ko <[email protected]> +Contact: Linux ARM Kernel mailing list <[email protected]> +Description: + (RW) Hardware prefetch aggressiveness mode for CPU X. The file is + present only while a supported Grace or Vera CPU is online. Reads + return a decimal value. Writes accept decimal values 0 through 9; + values 10 through 15 are reserved and rejected. Invalid writes fail + with -EINVAL without changing the target CPU register. The file is + root-only read/write (0600). + +What: /sys/devices/system/cpu/cpuX/cpumod/cbusy_filter_threshold +Date: August 2026 +Contact: Koba Ko <[email protected]> +Contact: Linux ARM Kernel mailing list <[email protected]> +Description: + (RW) Grace CPU cache-busy filter threshold. The file is present only + while a supported Grace CPU is online. Reads return a decimal value. + Writes accept decimal values 0 through 3. Invalid writes fail with + -EINVAL without changing the target CPU register. The file is + root-only read/write (0600). + +What: /sys/devices/system/cpu/cpuX/cpumod/cbusy_filter_window +Date: August 2026 +Contact: Koba Ko <[email protected]> +Contact: Linux ARM Kernel mailing list <[email protected]> +Description: + (RW) Grace CPU cache-busy filter window. The file is present only + while a supported Grace CPU is online. Reads return a decimal value. + Writes accept decimal values 0 through 3. Invalid writes fail with + -EINVAL without changing the target CPU register. The file is + root-only read/write (0600). + +What: /sys/devices/system/cpu/cpuX/cpumod/cmc_min_ways +Date: August 2026 +Contact: Koba Ko <[email protected]> +Contact: Linux ARM Kernel mailing list <[email protected]> +Description: + (RW) Grace CPU cache-management minimum-way control. The file is + present only while a supported Grace CPU is online. Reads return a + decimal value. Writes accept decimal values 0 through 7. Invalid + writes fail with -EINVAL without changing the target CPU register. + The file is root-only read/write (0600). + +What: /sys/devices/system/cpu/cpuX/cpumod/l2spr_cmc_max_ways +Date: August 2026 +Contact: Koba Ko <[email protected]> +Contact: Linux ARM Kernel mailing list <[email protected]> +Description: + (RW) Vera CPU cache-management maximum-way control. The file is + present only while a supported Vera CPU is online. Reads return a + decimal value. Writes accept decimal values 0 through 7. Invalid + writes fail with -EINVAL without changing the target CPU register. + The file is root-only read/write (0600).
Documentation/arch/arm64/arm-cpumod.rst new file +66 -0
66 lines not on this page · raw message
Documentation/arch/arm64/index.rst +1 -0
@@ -8,6 +8,7 @@ ARM64 Architecture :maxdepth: 1 acpi_object_usage + arm-cpumod amu arm-acpi arm-cca
Signature · 1 line
2.43.0
RFC PATCH v2 1/3 arm64: add CPU prefetch and cache modulation sysfs interface
In reply to Koba KoPerformance characterization on arm64 systems sometimes needs a way to inspect and bracket selected implementation-defined prefetch and cache controls without exposing raw register contents. Add an RFC-only arm64 cpumod interface that presents a small set of named, range-checked per-CPU sysfs attributes under each CPU device.
Establish the Kconfig and Makefile plumbing, CPU profile detection for Grace and Vera from the target CPU's MIDR, register-field descriptors, per-CPU kobject and attribute helpers, locking, and target-CPU read/write callbacks. Common attributes are exposed for all supported profiles, while Grace- and Vera-specific fields are exposed only for the matching profile. Place the option at the end of the top-level Kernel Features menu because these implementation controls are not tied to an Arm architecture revision.
Per-CPU cpumod sysfs state follows the CPU lifecycle. Module initialization populates subtrees for every online CPU while holding cpus_read_lock(), so initial enumeration and CPU hotplug state registration are serialized against concurrent topology changes. Offline CPUs are intentionally skipped during initial setup because target-CPU MIDR detection requires an IPI-capable online CPU; a later online callback creates their subtree.
Profile detection reads MIDR through a synchronous callback on the CPU owning the sysfs subtree and returns nonzero only when that dispatch fails, so no IPI error can be hidden. After a successful read, retain CPUMOD_PROFILE_UNKNOWN as the unsupported-profile sentinel. Subtree creation skips that profile during both initial online enumeration and later CPUHP online callbacks without blocking module load or CPU hotplug. Object and sysfs setup failures remain fatal and are propagated.
Keep the dynamically allocated kobject state in a private per-CPU pointer instead of the CPU device's generic driver-data slot. The online callback is idempotent when that pointer already records a subtree. The offline callback clears the private pointer before dropping the kobject reference, allowing the release callback to retain ownership of the final free.
Register-backed controls are root-only read/write files. Reads synchronously execute on the target CPU and return the live register field value from that callback. Writes remain range-checked and continue to log the target-CPU readback after the register update.
Register the dynamic CPUHP state with cpuhp_setup_state_nocalls_cpuslocked(). The nocalls form avoids replaying startup callbacks for CPUs already initialized by the explicit online enumeration. On init failure, release the CPU read lock, destroy all subtrees created so far, and return the error. Module exit removes the CPUHP state with the matching nocalls helper and tears down all remaining per-CPU state across possible CPUs.
--- arch/arm64/Kconfig | 17 + arch/arm64/kernel/Makefile | 1 + arch/arm64/kernel/arm_cpumod.c | 458 ++++++++++++++++++++++++ arch/arm64/kernel/arm_cpumod_internal.h | 93 +++++ 4 files changed, 569 insertions(+) create mode 100644 arch/arm64/kernel/arm_cpumod.c create mode 100644 arch/arm64/kernel/arm_cpumod_internal.h
arch/arm64/Kconfig +17 -0
@@ -2417,6 +2417,23 @@ config ARM64_CONTPTE bit, for any mappings that meet the size and alignment requirements. This reduces TLB pressure and improves performance. +config ARM64_CPUMOD + tristate "Arm CPU prefetch modulation sysfs controls" + help + Expose selected CPU implementation control register fields through + per-CPU sysfs attributes for performance characterization. + + The exposed controls allow hardware prefetch and cache-management + policy fields to be inspected and adjusted on CPUs where firmware + permits EL1 access to the relevant implementation control registers. + Systems where firmware traps or blocks these writes cannot use this + interface. + + This interface is intended for controlled performance evaluation and + should not be enabled by default on production systems. + + If unsure, say N. + endmenu # "Kernel Features" menu "Boot options"
arch/arm64/kernel/Makefile +1 -0
@@ -69,6 +69,7 @@ obj-$(CONFIG_VMCORE_INFO) += vmcore_info.o obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o obj-$(CONFIG_ARM64_MPAM) += mpam.o +obj-$(CONFIG_ARM64_CPUMOD) += arm_cpumod.o obj-$(CONFIG_ARM64_MTE) += mte.o obj-y += vdso-wrap.o obj-$(CONFIG_COMPAT_VDSO) += vdso32-wrap.o
arch/arm64/kernel/arm_cpumod.c new file +458 -0
458 lines not on this page · raw message
arch/arm64/kernel/arm_cpumod_internal.h new file +93 -0
93 lines not on this page · raw message
Signature · 1 line
2.43.0
RFC PATCH v2 3/3 selftests: arm64: add arm_cpumod kselftest
In reply to Koba KoAdd a kselftest for the arm_cpumod sysfs ABI. The test loads or reuses the module and selects either one target CPU or a snapshot of all online CPUs. In single-CPU mode, skip an offline target before module validation. Observe the per-CPU online attribute first. If it is absent, immediately require the CPU directory to still exist before treating the CPU as non-hotpluggable and online. If the attribute exists, it must be readable and contain 1.
After module load, recheck the online state before classifying a selected CPU without a cpumod directory as an unsupported profile. Fail if the CPU went offline or disappeared, log and skip CPUs that remain online without a directory, and require at least one selected CPU to expose a supported Grace or Vera profile. Return KSFT_SKIP if none do.
For supported CPUs, recognize the exposed Grace or Vera profile-specific layout and validate the matching attributes. Only ARM_CPUMOD_EXPECT_PROFILE supplies an external profile expectation.
Verify read paths by checking that attributes return decimal values and that affected_cpus identifies the current CPU as a decimal CPU number. Validate affected_cpus mode 444 and every visible register-backed control mode 600 with stat before functional write checks. Optional valid-write testing writes each current value back. Optional invalid-write testing checks selected out-of-range values and confirms that the prior value remains unchanged.
Both write paths are opt-in. Valid writeback can alter implementation-defined CPU controls; invalid-write checks exercise rejection through the same firmware-sensitive interface, but are expected not to update state. Cover all-online-CPU mode through /sys/devices/system/cpu/online and, when the test loaded the module, verify cleanup by removing it and checking that the selected CPU subtrees disappear. The test does not offline or online CPUs.
--- tools/testing/selftests/arm64/Makefile | 2 +- .../selftests/arm64/arm_cpumod/Makefile | 5 + .../selftests/arm64/arm_cpumod/arm_cpumod.sh | 543 ++++++++++++++++++ 3 files changed, 549 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/arm64/arm_cpumod/Makefile create mode 100755 tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh
tools/testing/selftests/arm64/Makefile +1 -1
@@ -4,7 +4,7 @@ ARCH ?= $(shell uname -m 2>/dev/null || echo not) ifneq (,$(filter $(ARCH),aarch64 arm64)) -ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs + ARM64_SUBTARGETS ?= tags signal pauth fp mte bti abi gcs arm_cpumod else ARM64_SUBTARGETS := endif
tools/testing/selftests/arm64/arm_cpumod/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_PROGS := arm_cpumod.sh + +include ../../lib.mk
tools/testing/selftests/arm64/arm_cpumod/arm_cpumod.sh new file +543 -0
543 lines not on this page · raw message
Signature · 1 line
2.43.0
Will Deacon
maintainerRe: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface
In reply to Koba KoQuoted 77 lines · Koba Ko
Performance characterization on arm64 systems sometimes needs a way to inspect and bracket selected implementation-defined prefetch and cache controls without exposing raw register contents. Add an RFC-only arm64 cpumod interface that presents a small set of named, range-checked per-CPU sysfs attributes under each CPU device.
Establish the
KconfigandMakefileplumbing, CPU profile detection for Grace and Vera from the target CPU's MIDR, register-field descriptors, per-CPU kobject and attribute helpers, locking, and target-CPU read/write callbacks. Common attributes are exposed for all supported profiles, while Grace- and Vera-specific fields are exposed only for the matching profile. Place the option at the end of the top-level Kernel Features menu because these implementation controls are not tied to an Arm architecture revision.Per-CPU cpumod sysfs state follows the CPU lifecycle. Module initialization populates subtrees for every online CPU while holding
cpus_read_lock(), so initial enumeration and CPU hotplug state registration are serialized against concurrent topology changes. Offline CPUs are intentionally skipped during initial setup because target-CPU MIDR detection requires an IPI-capable online CPU; a later online callback creates their subtree.Profile detection reads MIDR through a synchronous callback on the CPU owning the sysfs subtree and returns nonzero only when that dispatch fails, so no IPI error can be hidden. After a successful read, retain
CPUMOD_PROFILE_UNKNOWNas the unsupported-profile sentinel. Subtree creation skips that profile during both initial online enumeration and later CPUHP online callbacks without blocking module load or CPU hotplug. Object and sysfs setup failures remain fatal and are propagated.Keep the dynamically allocated kobject state in a private per-CPU pointer instead of the CPU device's generic driver-data slot. The online callback is idempotent when that pointer already records a subtree. The offline callback clears the private pointer before dropping the kobject reference, allowing the release callback to retain ownership of the final free.
Register-backed controls are root-only read/write files. Reads synchronously execute on the target CPU and return the live register field value from that callback. Writes remain range-checked and continue to log the target-CPU readback after the register update.
Register the dynamic CPUHP state with
cpuhp_setup_state_nocalls_cpuslocked(). The nocalls form avoids replaying startup callbacks for CPUs already initialized by the explicit online enumeration. On init failure, release the CPU read lock, destroy all subtrees created so far, and return the error. Module exit removes the CPUHP state with the matching nocalls helper and tears down all remaining per-CPU state across possible CPUs.Reviewed-by Fenghua Yu <[email protected]>Reviewed-by Tushar Dave <[email protected]>Reviewed-by Kai-Heng Feng <[email protected]>Signed-off-by Koba Ko <[email protected]>--- arch/arm64/Kconfig | 17 + arch/arm64/kernel/Makefile | 1 + arch/arm64/kernel/arm_cpumod.c | 458 ++++++++++++++++++++++++ arch/arm64/kernel/arm_cpumod_internal.h | 93 +++++ 4 files changed, 569 insertions(+) create mode 100644 arch/arm64/kernel/arm_cpumod.c create mode 100644 arch/arm64/kernel/arm_cpumod_internal.harch/arm64/Kconfig +6
-0@@ -2417,6 +2417,23 @@ config ARM64_CONTPTE bit, for any mappings that meet the size and alignment requirements. This reduces TLB pressure and improves performance. +config ARM64_CPUMOD + tristate "Arm CPU prefetch modulation sysfs controls" + help + Expose selected CPU implementation control register fields through + per-CPU sysfs attributes for performance characterization. +
arch/arm64/Kconfig config ARM64_CONTPTE · line 2435
Quoted 10 lines
arch/arm64/Kconfig +10
-0+ The exposed controls allow hardware prefetch and cache-management + policy fields to be inspected and adjusted on CPUs where firmware + permits EL1 access to the relevant implementation control registers. + Systems where firmware traps or blocks these writes cannot use this + interface. + + This interface is intended for controlled performance evaluation and + should not be enabled by default on production systems. + + If unsure, say N.
We don't tend to support these sort of implementation-defined features as:
- They're generally not something we can virtualise
- The registers should not be accessible to the kernel in production
- It's not clear what other bits are exposed in the registers
- These sort of controls can't always be toggled while the memory system is active
- We can't test them or maintain the zoo of interfaces we'd end up with if we started merging them
So exposing the registers directly to userspace doesn't feel like a particularly good idea to me.
Will
Re: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface
Hi Will,
Thanks for the feedback. I understand that your concern is about exposing implementation-defined registers to userspace, not simply choosing sysfs versus debugfs. I'm not planning to submit v3.
For controlled performance characterization, is there an existing kernel mechanism or a different design you would recommend? Or do you think these experimental controls are better kept out of tree?
Thanks,
Koba
Re: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface
Quoted 3 lines · Koba Ko
For controlled performance characterization, is there an existing kernel mechanism or a different design you would recommend? Or do you think these experimental controls are better kept out of tree?
As a user of this patchset, I am trying hard to avoid carrying it out of tree, either as patches or as a driver. Out-of-tree code makes kernel rebases and rollouts slower, and debugging more complex.
Is there a way to keep a generic interface available under CONFIG_EXPERT, or something similar, so that users who want to experiment can use it?
We could even taint the kernel when the interface is used, to make it clear that the user is messing with things they should not (!?).
Thanks,
--breno
Re: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface
Hi Breno,
Thanks for explaining the rebase, rollout and debugging costs of carrying this out of tree. That's useful context from someone using the patchset.
CONFIG_EXPERT plus tainting on use could make experimental use more explicit. Those safeguards alone would not establish that firmware access, virtualization and runtime register updates are handled safely; the local restrictions still need validation against the hardware and firmware requirements.
I'd like to hear Will's view on whether this opt-in approach could be acceptable in principle, and what requirements it would need to meet. I'm not planning to submit v3.
Thanks,
Koba
Will Deacon
maintainerRe: [RFC PATCH v2 1/3] arm64: add CPU prefetch and cache modulation sysfs interface
Quoted 9 lines · Koba Ko
Thanks for explaining the rebase, rollout and debugging costs of carrying this out of tree. That's useful context from someone using the patchset.
CONFIG_EXPERTplus tainting on use could make experimental use more explicit. Those safeguards alone would not establish that firmware access, virtualization and runtime register updates are handled safely; the local restrictions still need validation against the hardware and firmware requirements.
Quoted 3 lines
I'd like to hear Will's view on whether this opt-in approach could be acceptable in principle, and what requirements it would need to meet. I'm not planning to submit v3.
No, this isn't something we should be maintaining in the upstream kernel and if we take this then we'll suddenly grow a zoo full of this stuff.
If Nvidia has imp-def controls that they believe are useful to expose to userspace, then I suggest working with Arm to get something architectural that we can support across different pieces of hardware and which would integrate properly with the rest of the architecture (including things like virtualisation and debug).
Will