x86,fs/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

Hi All,

This series adds support for AMD's Privilege-Level Zero Association (PLZA), which allows the CPU to use a different CLOSID and/or RMID association while executing in kernel mode than it does in user mode.

The implementation is integrated through a generic kernel mode ("kmode") framework in fs/resctrl, enabling future architectures to support similar functionality without requiring changes to the core resctrl code. The intent is to provide a common infrastructure that architecture-specific implementations can hook into, minimizing future resctrl core modifications.

The features are documented in:

AMD64 Zen6 Platform Quality of Service (PQOS) Extensions, Publication # 69193 Revision 1.00, Issue Date March 2026

available at https://bugzilla.kernel.org/show_bug.cgi?id=206537

The patches are based on top of commit (tip/master):

c6279551a44c Merge branch into tip/master: 'x86/sev'

Any feedback is appreciated.

Background
==========

When memory bandwidth associated with a CLOSID is aggressively throttled, and a task with that CLOSID moves into kernel mode, the kernel operations are also aggressively throttled. This can stall forward progress and eventually degrade overall system performance.

Privilege-Level Zero Association (PLZA) allows the user to specify a CLOSID and/or RMID for execution at Privilege Level Zero. When PLZA is enabled on a CPU, kernel work at PL0 uses the CLOSID and/or RMID from MSR PQR_PLZA_ASSOC; otherwise, the CPU uses the CLOSID and RMID from PQR_ASSOC.

Design
======

A new sysfs file, info/kernel_mode, selects the kernel mode for resource allocation and monitoring of kernel work and, when the mode needs one, the resource group that backs it. Reads list the supported modes, with the active mode in square brackets. Writes use the same line without the brackets. Look at the thread below for design discussion.
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/

Two kernel modes are exposed:

- inherit_user: kernel work uses the allocation and monitoring associations of the user task on whose behalf it runs. It takes no parameters.

- global_enable_per_cpu: kernel work uses the allocation and/or monitoring association of a backing CTRL_MON or MON group. ctrl= and mon= select assign or inherit for each. group= names the group and must be the last parameter, because a group name may contain ';'.

When mon=assign, a CTRL_MON group is shown as <CTRL_MON>// and a MON group as <CTRL_MON>/<MON>/. When mon=inherit, or monitoring is unsupported, a CTRL_MON group is shown as <CTRL_MON>/.

An omitted ctrl=, mon=, or group= selects the default: assign where the platform supports it, and the default CTRL_MON group. A partial write therefore restores the omitted fields and programs every online CPU. ctrl=inherit together with mon=inherit is rejected. A monitor group requires mon=assign. A failed write leaves the current association in place.

Per-rdtgroup files kmode_cpus and kmode_cpus_list scope the association to a subset of online CPUs. They are visible only on the group that backs the active association. A write enables the association on CPUs added to the mask and disables it on CPUs removed from the mask. An empty mask disables the association on every currently online CPU. A new global_enable_per_cpu association sets the mask to every online CPU. A CPU that comes online later joins the association, including after a narrowed or empty mask. A CPU that goes offline leaves it only when that CPU is in the mask.

A group that backs the association, or a control group whose monitor group does, cannot change mode, including entering pseudo-lock setup. Reparenting a backing monitor group is rejected. The backing group cannot be renamed until the association is cleared.

The arch hook, resctrl_arch_configure_kmode_global(), keeps the fs/resctrl layer arch-neutral. enable=false disables the mode on the given CPUs and still writes the same CLOSID, RMID, and per-ID enables, which is what the PLZA domain rule requires. It does not by itself switch the active mode back to inherit_user.

resctrl_set_kmode_support() lets architecture code register each extra mode during resctrl initialization. inherit_user is registered by default. On AMD, PLZA registers global_enable_per_cpu when the feature is available and the platform can allocate or monitor.

Only AMD PLZA is wired up here; Intel and ARM can add their own support later by implementing the hooks.

Examples
========

(See Documentation/filesystems/resctrl.rst, "kernel_mode", "kmode_cpus", and "Examples on working with kernel_mode", for the full UAPI.)

# Mount resctrl

  # mount -t resctrl resctrl /sys/fs/resctrl
  # cd /sys/fs/resctrl

# Read the supported modes. The active mode is bracketed for display # only; do not include brackets when writing.

  # cat info/kernel_mode
  [inherit_user]
  global_enable_per_cpu:ctrl=assign;mon=assign;group=//

# Create a CTRL_MON group and back kernel-mode allocation with it. # mon=inherit, so the group is shown as <CTRL_MON>/.

  # mkdir ctrl1
  # echo "global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1/" \
          > info/kernel_mode
  # cat info/kernel_mode
  inherit_user
  [global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1/]

  # kmode_cpus and kmode_cpus_list are visible only on the backing group.
  # ls ctrl1/kmode_cpus*
  ctrl1/kmode_cpus  ctrl1/kmode_cpus_list

# Restrict the association to a CPU subset. The association is enabled

  # on CPUs added to the mask and disabled on CPUs removed from it.
  # echo 0-3 > ctrl1/kmode_cpus_list
  # cat ctrl1/kmode_cpus
  f
  # cat ctrl1/kmode_cpus_list
  0-3

# Return to inherit_user. The inactive global_enable_per_cpu line

  # shows the defaults again.
  # echo "inherit_user" > info/kernel_mode
  # cat info/kernel_mode
  [inherit_user]
  global_enable_per_cpu:ctrl=assign;mon=assign;group=//

Tested on AMD with PLZA; builds on x86 without PLZA and does not expose global_enable_per_cpu unless the feature is available.

Layout
======

01-03 x86: PLZA CPU feature, command-line option, and MSR definitions. 04-07 fs/resctrl: kernel mode enum, arch hook, kmode state, and
resctrl_set_kmode_support().
08 x86/resctrl: register global_enable_per_cpu when PLZA is
available.
09 fs/resctrl: document last_cmd_status under a top-level info
section.

  10     fs/resctrl: info/kernel_mode read interface.
  11     fs/resctrl: hidden rdtgroup files.
  12     fs/resctrl: per-rdtgroup kmode_cpus/kmode_cpus_list.
  13     x86,fs/resctrl: CPU hotplug for the active association, and
         clear PLZA across kexec.
  14     fs/resctrl: deactivate the association when a group is removed.
  15     fs/resctrl: info/kernel_mode write interface.
  16     fs/resctrl: reject mode changes and reparenting while a group
         backs the association.
  17     fs/resctrl: show or hide kmode_cpus on the backing group.
  18     fs/resctrl: kmode_cpus/kmode_cpus_list writes.
  19     fs/resctrl: documentation and examples.

Changelog
=========
v7:
- Used "CLOSID and/or RMID association" throughout the series because feature operates on the MSR_IA32_PQR_PLZA_ASSOC association register. The association concept is already known in the resctrl subsystem. Please let me know if you see a better alternative.
- Rebase onto tip/master c6279551a44c ("Merge branch into tip/master:

    'x86/sev'").

- Document last_cmd_status in a new "Top level files" section of the info directory, before kernel_mode.
- info/kernel_mode includes the monitor component of group= only when mon=assign. A CTRL_MON group then reads as <CTRL_MON>//. When mon=inherit, or monitoring is unsupported, it reads as <CTRL_MON>/.
- An omitted ctrl=, mon=, or group= selects the defaults (assign where the platform supports it, and the default CTRL_MON group). A partial write restores the omitted fields and programs every online CPU.
- Reject ctrl=inherit together with mon=inherit. With both enables clear, CPL0 keeps the user task's IDs, which is inherit_user.
- group= is the last parameter, so a group name may contain ';'.
- Drop resctrl_kmode_switch(). Deactivation restores inherit_user and the default ctrl and mon values.
- Move kmode_cpus visibility to after activation and deactivation, and move the mode-change and reparent checks to after the kernel_mode write interface.
- Clear MSR_IA32_PQR_PLZA_ASSOC from clear_closid_rmid() so a kexec boot does not keep a previous kernel's PLZA state.

v6:
- Addressed all the comments as discussed in v5. Anything missing is not intentional. Please feel free to comment on any mistake.
- Rename assign_global_enable_per_cpu to global_enable_per_cpu and describe the interface as a kernel mode rather than a policy.
- Flatten kernel mode state into resctrl_kmode. The arch hook is resctrl_arch_configure_kmode_global().
- An omitted ctrl=, mon=, or group= keeps the value shown by reading info/kernel_mode. inherit_user takes no parameters. ctrl=inherit;mon=inherit remains global_enable_per_cpu.
- Selecting a new global_enable_per_cpu association sets that group's kmode_cpus to every online CPU. A CPU that comes online later joins the association even if user space had removed it from the mask.
- kmode_cpus files are created hidden and shown only while the group backs the active association.
- A monitor group requires mon=assign. Removing a control group detaches a child that backs the association before the parent's CLOSID is freed.
- Reject a mode change when the group or one of its monitor groups backs the association.
- Split kmode_cpus visibility and the mode-change rejection into their own patches.
- Use the parsing similar to ctrlmondata.c:parse_line(), ctrlmondata.c:resctrl_io_alloc_parse_line(), monitor.c:resctrl_parse_mbm_assignment().
- Re-wrote the changelog as a feature and not as a bug-fix.
- Remove implementation details in function headers.

v5:
- Collapse the two v4 global-assign modes into a single assign_global_enable_per_cpu policy with ctrl= and mon= options.
- Use "association" terminology consistently in code, errors, and documentation.
- Register assign_global_enable_per_cpu once during x86 resource discovery when PLZA is available.
- Reject mon=inherit when group= selects a monitor group.
- Split hidden-file support, deactivation-on-teardown, and PLZA mode registration into separate patches for easier review.
- Fix msr_pqr_plza_assoc truncation on 32-bit systems by using u64.
- Refresh documentation and add a readable end-to-end example walkthrough.
- Block pseudo-lock setup on kernel-mode-associated groups.
- Reject mode changes and rename while a group backs the active association.

v4:
- Reorder and split the series into 15 patches: separate read-only info/kernel_mode display from the write path; add hotplug support when a CPU comes online; add an end-to-end documentation/examples patch.
- Introduced resctrl_set_kmode_support() so architecture code can register supported kernel-mode policies during resctrl initialization.
- info/kernel_mode write: validate group type; run fail paths before tearing down the active association so errors retain the old state.
- kmode_cpus / kmode_cpus_list: writable with incremental enable/disable deltas; empty masks allowed; offline CPUs rejected.
- Hotplug: newly online CPUs are added to the associated group's kmode_cpu_mask and programmed when assign_global_enable_per_cpu is active.

v3:
- Generalise the layer beyond AMD: rename "PLZA mode" to "kernel mode" (kmode) in code, sysfs, and Documentation.
- Reset the association when the associated rdtgroup is removed, instead of leaving stale state.

v2:
- Similar to RFC with a new proposal; interface names were not final.
- Separated Global Bandwidth Enforcement (GLBE) from PLZA; this series only adds PLZA support.
- Used "kmode" instead of "PLZA" in the generic layer.

Previous versions:
v6: https://lore.kernel.org/lkml/[email protected]/
v5: https://lore.kernel.org/lkml/[email protected]/
v4: https://lore.kernel.org/lkml/[email protected]/
v3: https://lore.kernel.org/lkml/[email protected]/
v2: https://lore.kernel.org/lkml/[email protected]/
v1: https://lore.kernel.org/lkml/[email protected]/

Babu Moger (19):
x86/cpufeatures: Support Privilege Level Zero Association (PLZA) x86/resctrl: Add PLZA support to command-line options x86/resctrl: Add PLZA MSR definition and field layout fs/resctrl: Introduce kernel mode enum arm,x86,fs/resctrl: Introduce arch hook to program global per-CPU
kernel mode
fs/resctrl: Introduce data structures for kernel mode configuration fs/resctrl: Introduce resctrl_set_kmode_support() to register
supported modes
x86/resctrl: Register RESCTRL_GLOBAL_ENABLE_PER_CPU when PLZA is
available
fs/resctrl: Rearrange top-level files in info directory fs/resctrl: Add interface to display kernel mode status fs/resctrl: Add support for hidden resource group files fs/resctrl: Introduce kmode_cpus/kmode_cpus_list per rdtgroup x86,fs/resctrl: Add CPU hotplug support for kernel mode fs/resctrl: Deactivate kernel mode when the group is removed fs/resctrl: Extend info/kernel_mode to configure the kernel mode fs/resctrl: Reject mode changes for groups backing kernel mode fs/resctrl: Control visibility of global per-CPU kernel mode group
files
fs/resctrl: Allow userspace to write kmode_cpus/kmode_cpus_list fs/resctrl: Add kernel mode documentation and examples

 .../admin-guide/kernel-parameters.txt         |   2 +-
 Documentation/filesystems/resctrl.rst         | 255 ++++++-
 arch/x86/include/asm/cpufeatures.h            |   1 +
 arch/x86/include/asm/msr-index.h              |   1 +
 arch/x86/kernel/cpu/resctrl/core.c            |   9 +
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c     |  38 +
 arch/x86/kernel/cpu/resctrl/internal.h        |  49 ++
 arch/x86/kernel/cpu/scattered.c               |   1 +
 drivers/resctrl/mpam_resctrl.c                |   7 +
 fs/resctrl/internal.h                         |  32 +
 fs/resctrl/rdtgroup.c                         | 693 ++++++++++++++++++
 include/linux/resctrl.h                       |  65 ++
 12 files changed, 1141 insertions(+), 12 deletions(-)

2.43.0

When memory bandwidth associated with a CLOSID is aggressively throttled, and a task with that CLOSID moves into kernel mode, the kernel operations are also aggressively throttled. This can stall forward progress and eventually degrade overall system performance.

AMD hardware supports a feature Privilege Level Zero Association (PLZA), which allows the CPU's CLOSID and/or RMID association to be changed during the transition from user mode to kernel mode. This allows the kernel to run using a different CLOSID and/or RMID than user space, which can improve system performance and/or allow kernel work to be monitored separately.

Note that PLZA feature associates CLOSID and/or RMID to kernel work, and therefore influences resource allocation and/or monitoring for all resources, not only memory bandwidth.

The feature is detected via CPUID_Fn80000020_EBX_x00 [Bit 9]: Privilege Level Zero Association (PLZA).

The PLZA feature details are documented in [1] available from [2].

[1] AMD64 Zen6 Platform Quality of Service (PQOS) Extensions:

    Publication # 69193 Revision: 1.00, Issue Date: March 2026

---
v7: Added Reviewed-by: tag(Reinette).

v6: Highlight that CLOSID/RMID association affects all resources,
not just memory bandwidth.

v5: Added Acked-by from Boris.
Updated the changelog to include cache allocation and monitoring. Moved the Link tag last.
Updated subject line for cpufeatures.

v4: Split the patch into 2. This patch only handles x86 changes.
Re-wrote the changelog along the ABMC changes.

v3: Code did not change. Patch order changed.
Added documentation link.

v2: Rebased on top of the latest tip.

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 arch/x86/kernel/cpu/scattered.c    | 1 +
 2 files changed, 2 insertions(+)

index 303467a498b8..3090fb28ce06 100644
@@ -536,6 +536,7 @@
 						      * and purposes if CLEAR_CPU_BUF_VM is set).
 						      */
 #define X86_FEATURE_X2AVIC_EXT		(21*32+20) /* AMD SVM x2AVIC support for 4k vCPUs */
+#define X86_FEATURE_PLZA		(21*32+21) /* Privilege Level Zero Association */
 
 /*
  * BUG word(s)

index c990245d29d6..be387ba1cbea 100644
@@ -62,6 +62,7 @@ static const struct cpuid_bit cpuid_bits[] = {
 	{ X86_FEATURE_BMEC,			CPUID_EBX,  3, 0x80000020, 0 },
 	{ X86_FEATURE_ABMC,			CPUID_EBX,  5, 0x80000020, 0 },
 	{ X86_FEATURE_SDCIAE,			CPUID_EBX,  6, 0x80000020, 0 },
+	{ X86_FEATURE_PLZA,			CPUID_EBX,  9, 0x80000020, 0 },
 	{ X86_FEATURE_AMD_WORKLOAD_CLASS,	CPUID_EAX, 22, 0x80000021, 0 },
 	{ X86_FEATURE_TSA_SQ_NO,		CPUID_ECX,  1, 0x80000021, 0 },
 	{ X86_FEATURE_TSA_L1_NO,		CPUID_ECX,  2, 0x80000021, 0 },

2.43.0

Add "plza" to the rdt= command-line parameter so PLZA (Privilege Level Zero Association) exposure to resctrl can be enabled or disabled at boot. PLZA is exposed by default when supported; rdt=!plza suppresses it.

---
v7: Removed repetative lines.
Added Reviewed-by: (Reinette).

v6: Updated the changelog.

v5: Updated the changelog to mention about rdt= options.

v4: Split the patch 1 from v3 into 2 patches.
This patch contains changes for resctrl subsystem.

---
 Documentation/admin-guide/kernel-parameters.txt | 2 +-
 Documentation/filesystems/resctrl.rst           | 1 +
 arch/x86/kernel/cpu/resctrl/core.c              | 2 ++
 3 files changed, 4 insertions(+), 1 deletion(-)

index 0b3c140babd9..b8c680f7f638 100644
@@ -6460,7 +6460,7 @@ Kernel parameters
 	rdt=		[HW,X86,RDT]
 			Turn on/off individual RDT features. List is:
 			cmt, mbmtotal, mbmlocal, l3cat, l3cdp, l2cat, l2cdp,
-			mba, smba, bmec, abmc, sdciae, energy[:guid],
+			mba, smba, bmec, abmc, sdciae, plza, energy[:guid],
 			perf[:guid].
 			E.g. to turn on cmt and turn off mba use:
 				rdt=cmt,!mba

index 2750f2eabf70..d13a4998ee7e 100644
@@ -28,6 +28,7 @@ SMBA (Slow Memory Bandwidth Allocation)				""
 BMEC (Bandwidth Monitoring Event Configuration)			""
 ABMC (Assignable Bandwidth Monitoring Counters)			""
 SDCIAE (Smart Data Cache Injection Allocation Enforcement)	""
+PLZA (Privilege Level Zero Association)				""
 =============================================================== ================================
 
 Historically, new features were made visible by default in /proc/cpuinfo. This

index 3294b18ba45f..e64e9191184e 100644
@@ -772,6 +772,7 @@ enum {
 	RDT_FLAG_BMEC,
 	RDT_FLAG_ABMC,
 	RDT_FLAG_SDCIAE,
+	RDT_FLAG_PLZA,
 };
 
 #define RDT_OPT(idx, n, f)	\

@@ -799,6 +800,7 @@ static struct rdt_options rdt_options[]  __ro_after_init = {
 	RDT_OPT(RDT_FLAG_BMEC,	    "bmec",	X86_FEATURE_BMEC),
 	RDT_OPT(RDT_FLAG_ABMC,	    "abmc",	X86_FEATURE_ABMC),
 	RDT_OPT(RDT_FLAG_SDCIAE,    "sdciae",	X86_FEATURE_SDCIAE),
+	RDT_OPT(RDT_FLAG_PLZA,	    "plza",	X86_FEATURE_PLZA),
 };
 #define NUM_RDT_OPTIONS ARRAY_SIZE(rdt_options)
 

2.43.0

Privilege Level Zero Association (PLZA) enables a CPU to use a different CLOSID and/or RMID association when executing in kernel mode than those used in user mode.

PLZA is programmed per-CPU through MSR_IA32_PQR_PLZA_ASSOC, which contains the kernel mode CLOSID and RMID, an enable bit for each, and a bit that enables PLZA on the CPU. All fields except the PLZA enable bit must be programmed identically on every CPU in a domain.

Make the register and its field layout available in preparation for resctrl support of PLZA.

---
v7: Updated changelog(Reinette) with minor changes.
Minor code comment update.

v6: Updated the code comments to match general format (members first and
details after).
changelog update.

v5: Sashiko reported:
https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=3
Fix the union truncate issue on 32 bit systems by using u64. Updated the changelog and code comments.

v4: Re-wrote the changelog and code comment.

v3: No code changes. Patch order changed. Improved changelog.

v2: No changes. Just rebasing on top of the latest tip branch.

---
 arch/x86/include/asm/msr-index.h       |  1 +
 arch/x86/kernel/cpu/resctrl/internal.h | 49 ++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)

index c0914829a925..13f81a0587b4 100644
@@ -1306,6 +1306,7 @@
 /* - AMD: */
 #define MSR_IA32_MBA_BW_BASE		0xc0000200
 #define MSR_IA32_SMBA_BW_BASE		0xc0000280
+#define MSR_IA32_PQR_PLZA_ASSOC		0xc00003fc
 #define MSR_IA32_L3_QOS_ABMC_CFG	0xc00003fd
 #define MSR_IA32_L3_QOS_EXT_CFG		0xc00003ff
 #define MSR_IA32_EVT_CFG_BASE		0xc0000400

2.43.0

Kernel work need not use the CLOSID and RMID of the user task on whose behalf it runs. On x86, Privilege Level Zero Association (PLZA) enables a CPU to use a different CLOSID and/or RMID association while executing at privilege level zero.

Resctrl needs common definitions of how kernel work is managed so that architectures can advertise the kernel modes they support and user space can query them and select the one to use.

Introduce enum resctrl_kernel_mode:

- RESCTRL_INHERIT_USER: Kernel work uses the CLOSID and RMID of the user task.

- RESCTRL_GLOBAL_ENABLE_PER_CPU: All the kernel work uses the CLOSID and/or RMID of a single resource group, which may also have user tasks assigned to it. The mode can be enabled or disabled per-CPU; on a CPU where it is disabled, kernel work inherits the user task's CLOSID and RMID.

---
v7: Changelog update (Reinette) with minor changes.
Code comment update.

v6: changelog update and code comment update.
Renamed RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU to RESCTRL_GLOBAL_ENABLE_PER_CPU.

v5: Collapse the two global-assign modes into a single
RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU value. Update the changelog to match the enum names and descriptions in the code.

v4: Updated the changelog to be generic as possible.
Moved the enum resctrl_kernel_mode to include/linux/resctrl.h. Updated the code comments to be generic.
Removed resctrl_kmode_cfg from the code. This definition can be fs specific only and architectures dont need to know. Changed enum name to resctrl_kernel_mode from resctrl_kernel_modes.

v3: Removed resctrl_kmode definition.
Changed the kernel mode definitions to enum resctrl_kernel_modes. Used BIT() to set/test the features.
Added details to changelog.

v2: New patch to handle PLZA interfaces with /sys/fs/resctrl/info/ directory.

    https://lore.kernel.org/lkml/[email protected]/
---
 include/linux/resctrl.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

index b6b4519a5190..6f3e57ec4aa3 100644
@@ -716,6 +716,31 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable);
  */
 bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r);
 
+/**
+ * enum resctrl_kernel_mode - Kernel modes
+ *
+ * @RESCTRL_INHERIT_USER:
+ *     Kernel work inherits the CLOSID and RMID of the user task.
+ *
+ * @RESCTRL_GLOBAL_ENABLE_PER_CPU:
+ *     All kernel work uses the CLOSID and/or RMID of a single resource
+ *     group, which may also have user tasks assigned to it. The mode can
+ *     be enabled/disabled per-CPU, all online CPUs are enabled by default.
+ *     When the mode is disabled on a CPU then the kernel work on that CPU
+ *     inherits the CLOSID and RMID from the user task, effectively the
+ *     kernel mode becomes RESCTRL_INHERIT_USER on that CPU.
+ *
+ *     On x86, CLOSID and RMID are configured independently; either or both
+ *     may differ from the user task.
+ */
+enum resctrl_kernel_mode {
+	RESCTRL_INHERIT_USER,
+	RESCTRL_GLOBAL_ENABLE_PER_CPU,
+	RESCTRL_KMODE_LAST = RESCTRL_GLOBAL_ENABLE_PER_CPU
+};
+
+#define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
+
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 

2.43.0

RESCTRL_GLOBAL_ENABLE_PER_CPU enables a CPU to use a different CLOSID and/or RMID association when executing in kernel mode than those used in user mode. On x86, this mode corresponds to PLZA (Privilege Level Zero Association).

PLZA is programmed per-CPU through MSR_IA32_PQR_PLZA_ASSOC, which contains the kernel mode CLOSID and RMID, an enable bit for each, and a bit that enables PLZA on the CPU. All fields except the PLZA enable bit must be programmed identically on every CPU in a domain.

The mode is enabled independently on each CPU, so the filesystem must program hardware when the mode is activated, when CPUs are added or removed, and when it is deactivated. CLOSID and RMID can be programmed independently, so the architecture callback provides separate enable options for each.

Add resctrl_arch_configure_kmode_global() with an x86 PLZA implementation and an empty MPAM stub for ARM systems that do not support this mode.

---
v7: Updated the changelog with consistant terms.
Update changelog based on comments from Reinette with minor changes.

v6: Rename to resctrl_arch_configure_kmode_global(). Rewrite changelog
for RESCTRL_GLOBAL_ENABLE_PER_CPU only (not a bugfix, not a code restatement). Add arm to the subject. Document parameters first; closid/rmid are always programmed. Split fs/arch contract from call scenarios and name the mode explicitly.

v5: Fix kernel-doc for resctrl_arch_configure_kmode(), wire assign_ctrl
through to PLZA closid_en, and update the changelog.
Add calling context in API doc.

v4: Added assign_mon parameter in resctrl_arch_configure_kmode() to program the RMID
as discussed in below.
https://lore.kernel.org/lkml/[email protected]/
Reintroduce independent monitoring assignment via assign_mon after removing the task-based PLZA approach in v3.
Changed cpumask type to "const struct cpumask *cpu_mask".
Added MPAM stub to avoid any linking issues when resctrl_arch_configure_kmode() is called from FS layer. Thanks to Qinyun.
Re-wrote the changelog to be generic.
Updated code comments.
v3: Removed task based PLZA implementation so related changes are removed.
Per-task RMID handling is replaced by group type selection in the fs layer. Updated the change log with details.
Removed resctrl_arch_set_kmode() as arch only provides the modes supported. It is FS which decided which mode to apply.
v2: Updated the commit message to include the sequence of steps to enable PLZA.
Added mode code comments for clarity.
Added kmode to function names to be generic.

---
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 38 +++++++++++++++++++++++
 drivers/resctrl/mpam_resctrl.c            |  7 +++++
 include/linux/resctrl.h                   | 28 +++++++++++++++++
 3 files changed, 73 insertions(+)

index 62044489b052..79b6fdcbc553 100644
@@ -137,3 +137,41 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable)
 
 	return 0;
 }
+
+static void resctrl_kmode_set_one_amd(void *arg)
+{
+	union msr_pqr_plza_assoc *plza = arg;
+
+	wrmsrq(MSR_IA32_PQR_PLZA_ASSOC, plza->full);
+}
+
+/*
+ * resctrl_arch_configure_kmode_global - Program Privilege Level Zero
+ *					 Association (PLZA).
+ * @cpu_mask:	CPUs whose MSR_IA32_PQR_PLZA_ASSOC should be updated.
+ * @closid:	CLOSID programmed into MSR_IA32_PQR_PLZA_ASSOC. Always
+ *		written; required even when @assign_ctrl is false.
+ * @assign_ctrl: Whether PLZA should program the kernel mode CLOSID.
+ * @rmid:	RMID programmed into MSR_IA32_PQR_PLZA_ASSOC. Always
+ *		written; required even when @assign_mon is false.
+ * @assign_mon:	Whether PLZA should program the kernel mode RMID.
+ * @enable:	Whether PLZA should program the kernel mode.
+ *
+ * Program PLZA on @cpu_mask for %RESCTRL_GLOBAL_ENABLE_PER_CPU. See
+ * union msr_pqr_plza_assoc for the hardware field meanings.
+ */
+void resctrl_arch_configure_kmode_global(const struct cpumask *cpu_mask,
+					 u32 closid, bool assign_ctrl,
+					 u32 rmid, bool assign_mon,
+					 bool enable)
+{
+	union msr_pqr_plza_assoc plza = { 0 };
+
+	plza.split.rmid = rmid;
+	plza.split.rmid_en = assign_mon;
+	plza.split.closid = closid;
+	plza.split.closid_en = assign_ctrl;
+	plza.split.plza_en = enable;
+
+	on_each_cpu_mask(cpu_mask, resctrl_kmode_set_one_amd, &plza, 1);
+}

index 5468367776e2..548bfb336be5 100644
@@ -139,6 +139,13 @@ bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r)
 	return false;
 }
 
+void resctrl_arch_configure_kmode_global(const struct cpumask *cpu_mask,
+					 u32 closid, bool assign_ctrl,
+					 u32 rmid, bool assign_mon,
+					 bool enable)
+{
+}
+
 void resctrl_arch_pre_mount(void)
 {
 }

index 6f3e57ec4aa3..e43415ae6e0e 100644
@@ -741,6 +741,34 @@ enum resctrl_kernel_mode {
 
 #define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
 
+/**
+ * resctrl_arch_configure_kmode_global() - Program RESCTRL_GLOBAL_ENABLE_PER_CPU
+ *
+ * @cpu_mask:	Online CPUs on which to update the CLOSID and/or RMID.
+ * @closid:	CLOSID of the resource group backing %RESCTRL_GLOBAL_ENABLE_PER_CPU
+ *		mode. Always provided, even when @assign_ctrl is false.
+ * @assign_ctrl: true to update the @closid for kernel mode; false to inherit the
+ *		CLOSID from the user task.
+ * @rmid:	RMID of the resource group backing %RESCTRL_GLOBAL_ENABLE_PER_CPU
+ *		mode. Always provided, even when @assign_mon is false.
+ * @assign_mon:	true to update the @rmid for kernel mode; false to inherit the
+ *		RMID from the user task.
+ * @enable:	true to enable %RESCTRL_GLOBAL_ENABLE_PER_CPU on CPUs in @cpu_mask;
+ *		false to disable @RESCTRL_GLOBAL_ENABLE_PER_CPU and reset the
+ *		kernel mode to %RESCTRL_INHERIT_USER.
+ *
+ * The callback programs %RESCTRL_GLOBAL_ENABLE_PER_CPU. A configuration starts
+ * when %RESCTRL_GLOBAL_ENABLE_PER_CPU is activated and ends when the mode is
+ * deactivated or userspace changes the backing resource group, @assign_ctrl, or
+ * @assign_mon. Each call for a given configuration uses the same @closid,
+ * @assign_ctrl, @rmid, and @assign_mon; only @cpu_mask and @enable differ.
+ * Before a configuration ends, it is disabled on every CPU where it is enabled.
+ */
+void resctrl_arch_configure_kmode_global(const struct cpumask *cpu_mask,
+					 u32 closid, bool assign_ctrl,
+					 u32 rmid, bool assign_mon,
+					 bool enable);
+
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 

2.43.0

The resctrl filesystem needs to track the supported kernel modes, the active mode, and whether the backing group's CLOSID and RMID are assigned or inherited.

Introduce struct resctrl_kmode_cfg to represent supported kernel modes and their configuration.

---
v7: Changed kmode_sup -> supported, kmode_cur -> active.
Updated the changelog.
Removed resctrl_kmode_init().
Updated the code comments.

v6: Drop ctrl_en/mon_en and use resctrl_arch_alloc_capable() /
resctrl_arch_mon_capable() directly. Flatten caps/active into resctrl_kmode_cfg. Rename resctrl_kcfg to resctrl_kmode, ctrl_mode to ctrl, mon_mode to mon, and k_rdtgrp to rdtgrp. Comments describe the objects, not a member list. Rewrite changelog.

v5: Replace per-mode registration helpers with a single resctrl_kmode_cfg
structure. Derive ctrl_en and mon_en from resctrl_arch_alloc_capable() and resctrl_arch_mon_capable() during filesystem init. Update code comments and changelog.

v4: New patch to initialize supported kernel modes.
https://lore.kernel.org/lkml/[email protected]/
Changed the kmode_cur to enum resctrl_kernel_mode in resctrl_kmode_cfg. Moved the resctrl_kmode_cfg to filesystem code.

---
 fs/resctrl/internal.h | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

index e62a277dee85..38aa0c0cfb7e 100644
@@ -314,6 +314,33 @@ struct mbm_state {
 	u32	prev_bw;
 };
 
+/**
+ * enum kmode_state - Kernel mode association state.
+ * @KMODE_INHERIT:    Inherit the CLOSID or RMID from the user task.
+ * @KMODE_ASSIGN:     Use the CLOSID or RMID from the backing resource group.
+ */
+enum kmode_state {
+	KMODE_INHERIT,
+	KMODE_ASSIGN
+};
+
+/**
+ * struct resctrl_kmode_cfg - Kernel mode configuration
+ * @supported:	Bitmap of supported kernel modes.
+ * @active:	Currently active kernel mode.
+ * @ctrl:	CLOSID assign or inherit in %RESCTRL_GLOBAL_ENABLE_PER_CPU.
+ * @mon:	RMID assign or inherit in %RESCTRL_GLOBAL_ENABLE_PER_CPU.
+ * @rdtgrp:	Resource group backing %RESCTRL_GLOBAL_ENABLE_PER_CPU.
+ *		%NULL while the mode is not selected.
+ */
+struct resctrl_kmode_cfg {
+	DECLARE_BITMAP(supported, RESCTRL_NUM_KERNEL_MODES);
+	enum resctrl_kernel_mode	active;
+	enum kmode_state		ctrl;
+	enum kmode_state		mon;
+	struct rdtgroup			*rdtgrp;
+};
+
 extern struct mutex rdtgroup_mutex;
 
 static inline const char *rdt_kn_name(const struct kernfs_node *kn)

2.43.0

Architectures may support different kernel modes that need to be exposed to user space via resctrl filesystem.

Initialize the resctrl_kmode data structure and add resctrl_set_kmode_support() to allow architecture to register its supported kernel modes. This is preparatory infrastructure for future kernel mode support in resctrl.

---
v7: Moved the static initialization for resctrl_kmode in here as it is used
here first.
Updated the changelog and code comments.

v6: Use resctrl_kmode.kmode_sup after the struct flatten. Drop duplicate
kernel-doc from the .c definition and place the function next to resctrl_kmode_init(). Rewrite changelog (not a bugfix).

v5: New patch split from resctrl_kmode_cfg initialization. Register modes
in resctrl_kmode_cfg.caps.kmode_sup using RESCTRL_INHERIT_USER and RESCTRL_ASSIGN_GLOBAL_ENABLE_PER_CPU. Ignore out-of-range mode values.

v4: New patch to initialize supported kernel modes.

    https://lore.kernel.org/lkml/[email protected]/
---
 fs/resctrl/rdtgroup.c   | 14 ++++++++++++++
 include/linux/resctrl.h | 12 ++++++++++++
 2 files changed, 26 insertions(+)

index 443b41a4d467..28bd4c4d98f2 100644
@@ -76,6 +76,14 @@ static void rdtgroup_destroy_root(void);
 
 struct dentry *debugfs_resctrl;
 
+static struct resctrl_kmode_cfg resctrl_kmode = {
+	.supported = { BIT(RESCTRL_INHERIT_USER) },
+	.active = RESCTRL_INHERIT_USER,
+	.ctrl = KMODE_ASSIGN,
+	.mon = KMODE_ASSIGN,
+	.rdtgrp = NULL,
+};
+
 /*
  * Memory bandwidth monitoring event to use for the default CTRL_MON group
  * and each new CTRL_MON group created by the user.  Only relevant when

@@ -2297,6 +2305,12 @@ static void io_alloc_init(void)
 	}
 }
 
+void resctrl_set_kmode_support(enum resctrl_kernel_mode kmode)
+{
+	if (kmode < RESCTRL_NUM_KERNEL_MODES)
+		__set_bit(kmode, resctrl_kmode.supported);
+}
+
 void resctrl_file_fflags_init(const char *config, unsigned long fflags)
 {
 	struct rftype *rft;

index e43415ae6e0e..519c28c63bc7 100644
@@ -741,6 +741,18 @@ enum resctrl_kernel_mode {
 
 #define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
 
+/**
+ * resctrl_set_kmode_support() - Register a supported kernel mode
+ * @kmode:	Kernel mode supported by the architecture.
+ *
+ * %RESCTRL_INHERIT_USER is a default mode and does not need to be
+ * registered.
+ *
+ * Architectures use this interface during resctrl initialization to
+ * advertise supported kernel modes.
+ */
+void resctrl_set_kmode_support(enum resctrl_kernel_mode kmode);
+
 /**
  * resctrl_arch_configure_kmode_global() - Program RESCTRL_GLOBAL_ENABLE_PER_CPU
  *

2.43.0

PLZA (Privilege Level Zero Association) provides the global per-CPU kernel mode, RESCTRL_GLOBAL_ENABLE_PER_CPU, which enables a CPU to use a different CLOSID and/or RMID association when executing in kernel mode than those used in user mode.

Register the RESCTRL_GLOBAL_ENABLE_PER_CPU kernel mode with the resctrl kernel mode infrastructure when PLZA is supported.

---
v7: Changelog Update for consistancy.

v6: Use RESCTRL_GLOBAL_ENABLE_PER_CPU.
Rewrite changelog (not a bugfix).

v5: Update the patch to adapt new data structure and resctrl_set_kmode_support()
changes.

v4: New patch to set the supported features during arch init.

---
 arch/x86/kernel/cpu/resctrl/core.c | 4 ++++
 1 file changed, 4 insertions(+)

index e64e9191184e..14530c11bbd7 100644
@@ -990,6 +990,10 @@ static __init bool get_rdt_resources(void)
 	rdt_alloc_capable = get_rdt_alloc_resources();
 	rdt_mon_capable = get_rdt_mon_resources();
 
+	if (rdt_cpu_has(X86_FEATURE_PLZA) &&
+	    (rdt_alloc_capable || rdt_mon_capable))
+		resctrl_set_kmode_support(RESCTRL_GLOBAL_ENABLE_PER_CPU);
+
 	return (rdt_mon_capable || rdt_alloc_capable);
 }
 

2.43.0

The "last_cmd_status" file reports the status of the last command executed by the resctrl subsystem. Since it is documented after all other files in the info directory, it is easy to overlook.

Create a dedicated section for top-level files in the info directory and move the "last_cmd_status" documentation there.

---
v7: New patch to move the last_cmd_status in new section.

---
 Documentation/filesystems/resctrl.rst | 29 +++++++++++++++++----------
 1 file changed, 18 insertions(+), 11 deletions(-)

index d13a4998ee7e..2d672bbb69be 100644
@@ -529,18 +529,25 @@ with the following files:
 The upper bound for how many "CTRL_MON" + "MON" can be created
 is the smaller of the L3_MON and PERF_PKG_MON "num_rmids" values.
 
-Finally, in the top level of the "info" directory there is a file
-named "last_cmd_status". This is reset with every "command" issued
-via the file system (making new directories or writing to any of the
-control files). If the command was successful, it will read as "ok".
-If the command failed, it will provide more information that can be
-conveyed in the error returns from file operations. E.g.
-::
+Top level files
+---------------
+
+The top level of the "info" directory contains the following file(s):
+
+"last_cmd_status":
+	Status of the last command executed by the resctrl subsystem.
+
+	This file is reset with every "command" issued via the file
+	system (making new directories or writing to any of the control
+	files). If the command was successful, it will read as "ok".
+	If the command failed, it will provide more information that
+	can be conveyed in the error returns from file operations.
+	For example::
 
-	# echo L3:0=f7 > schemata
-	bash: echo: write error: Invalid argument
-	# cat info/last_cmd_status
-	mask f7 has non-consecutive 1-bits
+		# echo L3:0=f7 > schemata
+		bash: echo: write error: Invalid argument
+		# cat info/last_cmd_status
+		mask f7 has non-consecutive 1-bits
 
 Resource alloc and monitor groups
 =================================

2.43.0

Kernel mode support allows the CPU's CLOSID and/or RMID association to change when transitioning from user mode to kernel mode.

Add info/kernel_mode to expose the supported and active kernel modes through the resctrl filesystem. Supported modes are listed one per line, with the active mode enclosed in square brackets.

---
v7: Updated the changelog.

    kernel_mode prints the monitoring group only when mon == assign.

v6: Rewrote the changelog to make it as feature and not as a bug fix.
Update the code to use the new name "global_enable_per_cpu". Display ctrl or mon only when allocation or monitoring is supportd.

v5: Use resctrl_kcfg.caps and resctrl_kcfg.active after the struct split
from patch 0006. Rename resctrl_kmode_bind_str() to resctrl_kmode_state_str() and ctrl_bind/mon_bind to ctrl_state/mon_state. Used info_kn_lock()/info_kn_unlock().
Add documentation update in resctrl.rst.

v4: Fixed the display of inherit_ctrl_and_mon policy. It is not
associated with any group.
Added "uninitialized" for the non-active group. Rewrote the changelog.

v3: New patch to handle the changed interface file info/kernel_mode.
Changed the group name to "none" if kmode binding is not done. Reinette suggested "uninitialized". "none" seemed more relevent.

---
 Documentation/filesystems/resctrl.rst |  56 ++++++++++++++
 fs/resctrl/rdtgroup.c                 | 105 ++++++++++++++++++++++++++
 2 files changed, 161 insertions(+)

index 2d672bbb69be..033d405338c0 100644
@@ -549,6 +549,62 @@ The top level of the "info" directory contains the following file(s):
 		# cat info/last_cmd_status
 		mask f7 has non-consecutive 1-bits
 
+"kernel_mode":
+	Lists the kernel modes supported by the system, one per line, with
+	the active mode enclosed in square brackets.
+
+	A task running on a CPU is associated with an allocation and
+	monitoring ID pair (ctrl_hw_id, mon_hw_id), which determines the
+	resource allocation and monitoring applied to that task. Kernel
+	modes allow the CPU's allocation and/or monitoring association to
+	change during the transition from user mode to kernel mode.
+
+	Possible kernel modes are:
+
+	"inherit_user":
+		Default mode at boot. Kernel work uses the allocation
+		and monitoring of the user task on whose behalf it runs.
+
+	"global_enable_per_cpu":
+		Kernel work uses the allocation and/or monitoring of a backing
+		CTRL_MON or MON group. The mode can be enabled or disabled
+		per-CPU. The line contains group= and whichever of ctrl= and
+		mon= the platform supports::
+
+		  global_enable_per_cpu:ctrl=<assign|inherit>;mon=<assign|inherit>;group=<CTRL_MON>/<MON>/
+
+		"ctrl":
+			"assign" if kernel work uses the resource group's
+			allocation, or "inherit" if it uses the user task's
+			allocation. Present only on systems that support allocation.
+
+		"mon":
+			"assign" if kernel work uses the resource group's monitoring,
+			or "inherit" if it uses the user task's monitoring. Present
+			only on systems that support monitoring.
+
+		"group":
+			Path of the backing resource group, relative to the resctrl
+			mount point.
+
+			A MON group is shown as "<CTRL_MON>/<MON>/".
+
+			A CTRL_MON group is shown as "<CTRL_MON>//" when "mon=assign".
+
+			When "mon=inherit", or monitoring is unsupported, a CTRL_MON group
+			is shown as "<CTRL_MON>/".
+
+		When the mode is not active, the output lists the supported parameters
+		and their default values, which are used if the mode is activated
+		without new values.
+
+	Example of a system that supports allocation and monitoring, with default mode
+	set to inherit_user::
+
+		# cat info/kernel_mode
+		[inherit_user]
+		global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
 Resource alloc and monitor groups
 =================================
 

2.43.0

A resctrl file is visible to user space as soon as its resource group directory is activated. A file can only be hidden after it has been created, as is done for the info directory's BMEC event configuration files.

In the global_enable_per_cpu kernel mode, kernel work running on CPUs where the mode is active uses the CLOSID and/or RMID as part of a single backing resource group. The files needed to display and modify the CPUs on which this kernel mode is active are only meaningful in that group. Visible in every resource group, the files would return errors when accessed in any group other than the one backing the active kernel mode. Creating the files when the kernel mode is activated, and removing them when it is deactivated, would add failure paths to kernel mode activation that showing and hiding existing files does not have.

Allow a resource group file to be marked as hidden so that it is hidden when created, before its directory is activated, and is never visible to user space until resctrl restores its visibility. This provides the foundation in preparation for kernel mode support that shows the kmode_cpus and kmode_cpus_list files only in the resource group backing the active kernel mode.

---
v7: Updated the changelog (Reinette's input with minor update in second
paragrapj).

v6: Removed resctrl_hidden_files_set_visible() from the patch.
It will be separate patch.
Re-wrote the changelog.

v5: New patch to add support for creating hidden files.

---
 fs/resctrl/internal.h | 2 ++
 fs/resctrl/rdtgroup.c | 3 +++
 2 files changed, 5 insertions(+)

index 38aa0c0cfb7e..909511e985a1 100644
@@ -283,6 +283,7 @@ extern int max_name_width;
  * @kf_ops:	File operations
  * @flags:	File specific RFTYPE_FLAGS_* flags
  * @fflags:	File specific RFTYPE_* flags
+ * @hidden:	Hide file at creation; visibility may be restored later
  * @seq_show:	Show content of the file
  * @write:	Write to the file
  */

@@ -292,6 +293,7 @@ struct rftype {
 	const struct kernfs_ops	*kf_ops;
 	unsigned long		flags;
 	unsigned long		fflags;
+	bool			hidden;
 
 	int (*seq_show)(struct kernfs_open_file *of,
 			struct seq_file *sf, void *v);

index afbe0f94940a..9235f094245c 100644
@@ -313,6 +313,9 @@ static int rdtgroup_add_file(struct kernfs_node *parent_kn, struct rftype *rft)
 		return ret;
 	}
 
+	if (rft->hidden)
+		kernfs_show(kn, false);
+
 	return 0;
 }
 

2.43.0

The RESCTRL_GLOBAL_ENABLE_PER_CPU kernel mode allows CPUs to use different CLOSID and/or RMID associations when transitioning from user mode to kernel mode. When active, it is enabled or disabled on a per-CPU basis through a backing resource group.

Add kmode_cpus and kmode_cpus_list to display the CPUs in a resource group backing the active kernel mode.

---
v7: Changelog updated with consistant terms.
Code comment update.

v6: Drop rdtgrp::kmode. Rewrite changelog (not a bugfix; association).

    https://lore.kernel.org/lkml/[email protected]/

v5: Updated the changelog.
Removed the RDT_MODE_PSEUDO_LOCKED check and added rdtgroup->kmode check. Files are created hidden.

v4: Rewrote the change to be generic.

v3: New patch to add "kmode_cpus" and "kmode_cpus_list" to support
kernel_modes.

---
 fs/resctrl/internal.h |  3 +++
 fs/resctrl/rdtgroup.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)

index 909511e985a1..a473b408dd9d 100644
@@ -216,6 +216,8 @@ struct mongroup {
  * @mon:			mongroup related data
  * @mode:			mode of resource group
  * @mba_mbps_event:		input monitoring event id when mba_sc is enabled
+ * @kmode_cpu_mask:		list CPUs in the group when backing the active
+ *				RESCTRL_GLOBAL_ENABLE_PER_CPU kernel mode
  * @plr:			pseudo-locked region
  */
 struct rdtgroup {

@@ -229,6 +231,7 @@ struct rdtgroup {
 	struct mongroup			mon;
 	enum rdtgrp_mode		mode;
 	enum resctrl_event_id		mba_mbps_event;
+	struct cpumask			kmode_cpu_mask;
 	struct pseudo_lock_region	*plr;
 };
 

2.43.0

The RESCTRL_GLOBAL_ENABLE_PER_CPU kernel mode allows the CPU's CLOSID and/or RMID association to change when transitioning from user mode to kernel mode. The mode is enabled or disabled on a per-CPU basis through a backing resource group. CPUs that come online are brought into a state consistent with the active kernel mode, while CPUs that go offline are removed from its CPU mask.

Integrate RESCTRL_GLOBAL_ENABLE_PER_CPU mode with CPU hotplug handling so that CPUs automatically join the active kernel mode when they come online and are removed from it when they go offline. Also, ensure the register state remains in a consistent reset state across kexec boot transitions. This ensures that kernel mode behavior remains consistent across CPU hotplug events and kexec transitions.

Introduce resctrl_kmode_global_online_cpu() and resctrl_kmode_global_offline_cpu(), and hook them into the existing resctrl_online_cpu() and resctrl_offline_cpu() paths. Also update clear_closid_rmid() to reset MSR_IA32_PQR_PLZA_ASSOC register across kexec transitions.

---
v7: Changelog update.
Code comment update.
Added changes to reset the PLZA register during the bootup.

v6: Update the changelog in non-bugfix form.
Update the code to remove rdtgroup->kmode.
Introduce new wrapper resctrl_configure_kmode_global() to consolidate architecture calls.

v5: Added resctrl_kmode_offline_cpu() to disable the kernel mode
association and update kmode_cpu_mask when an associated CPU goes
offline.
This addresses sashiko comments:
https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=11
https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=10
Also moved the patch earlier than patch 10.
[RESEND PATCH v4 10/15] fs/resctrl: Reset the kernel mode binding when an rdtgroup is removed That seemed much more clear.

v4: New patch in the series. Patch taken from:
https://lore.kernel.org/lkml/[email protected]/
Updated the code to enable kernel mode by default whenever a CPU comes online when one of global-assign mode is enabled.

---
 arch/x86/kernel/cpu/resctrl/core.c |  3 ++
 fs/resctrl/rdtgroup.c              | 56 ++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+)

index 14530c11bbd7..d7faf30c2885 100644
@@ -704,6 +704,9 @@ static void clear_closid_rmid(int cpu)
 	state->cur_closid = RESCTRL_RESERVED_CLOSID;
 	state->cur_rmid = RESCTRL_RESERVED_RMID;
 	wrmsrq(MSR_IA32_PQR_ASSOC, val.q);
+
+	if (boot_cpu_has(X86_FEATURE_PLZA))
+		wrmsrq(MSR_IA32_PQR_PLZA_ASSOC, 0);
 }
 
 static int resctrl_arch_online_cpu(unsigned int cpu)

2.43.0

The RESCTRL_GLOBAL_ENABLE_PER_CPU allows CPUs to use different CLOSID and/or RMID association when transitioning from user mode to kernel mode. When active, the mode is enabled or disabled on a per-CPU basis as part a backing resource group.

When the backing group is removed, the configured kernel mode settings are no longer valid. Deactivate the kernel mode and restore it to RESCTRL_INHERIT_USER, the default mode in which kernel work inherits the user task's CLOSID and/or RMID.

---
v7: Removed rdtgroup_kmode_global_detach() and merged everything to
rdtgroup_kmode_global_deactivate().
Removed detach from rdtgroup_rmdir_ctrl().
Updated the changelog and code comments.

v6: Rewrite changelog (not a bugfix).
Shorten the code comments.

v5: Took care of couple of Sashiko comments on previous patch.
https://sashiko.dev/#/patchset/[email protected]?part=10
Re-wrote the changelog.
Renamed rdtgroup_config_kmode_reset() to rdtgroup_kmode_deactivate().

v4: Re-wrote the changelog.
Added the call free_all_child_rdtgrp() and rmdir_all_sub() Simplified the code comments.

v3: New patch to handle the kernel_mode clean up.

---
 Documentation/filesystems/resctrl.rst |  3 +++
 fs/resctrl/rdtgroup.c                 | 30 +++++++++++++++++++++++++++
 2 files changed, 33 insertions(+)

index 033d405338c0..e57361dd342e 100644
@@ -594,6 +594,9 @@ The top level of the "info" directory contains the following file(s):
 			When "mon=inherit", or monitoring is unsupported, a CTRL_MON group
 			is shown as "<CTRL_MON>/".
 
+			Removing the backing group deactivates the current kernel mode and
+			restores to "inherit_user".
+
 		When the mode is not active, the output lists the supported parameters
 		and their default values, which are used if the mode is activated
 		without new values.

index af4c929d7ed4..97d99406de00 100644
@@ -1178,6 +1178,29 @@ static void resctrl_kmode_global_offline_cpu(unsigned int cpu)
 	resctrl_configure_kmode_global(cpumask_of(cpu), rdtgrp, false);
 }
 
+/*
+ * rdtgroup_kmode_global_deactivate() - Deactivate global kernel mode
+ * @rdtgrp: Resctrl group backing the active kernel mode.
+ *
+ * If @rdtgrp backs the active RESCTRL_GLOBAL_ENABLE_PER_CPU mode, disable
+ * the mode, clear kernel mode state, and restore the default
+ * RESCTRL_INHERIT_USER configuration.
+ */
+static void rdtgroup_kmode_global_deactivate(struct rdtgroup *rdtgrp)
+{
+	if (resctrl_kmode.active != RESCTRL_GLOBAL_ENABLE_PER_CPU ||
+	    resctrl_kmode.rdtgrp != rdtgrp)
+		return;
+
+	resctrl_configure_kmode_global(&rdtgrp->kmode_cpu_mask, rdtgrp, false);
+	cpumask_clear(&rdtgrp->kmode_cpu_mask);
+
+	resctrl_kmode.rdtgrp = NULL;
+	resctrl_kmode.active = RESCTRL_INHERIT_USER;
+	resctrl_kmode.ctrl = KMODE_ASSIGN;
+	resctrl_kmode.mon = KMODE_ASSIGN;
+}
+
 void *rdt_kn_parent_priv(struct kernfs_node *kn)
 {
 	/*

@@ -3214,6 +3237,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp)
 
 	head = &rdtgrp->mon.crdtgrp_list;
 	list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) {
+		rdtgroup_kmode_global_deactivate(sentry);
 		rdtgroup_unassign_cntrs(sentry);
 		free_rmid(sentry->closid, sentry->mon.rmid);
 		list_del(&sentry->mon.crdtgrp_list);

@@ -3251,6 +3275,7 @@ static void rmdir_all_sub(void)
 		cpumask_or(&rdtgroup_default.cpu_mask,
 			   &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask);
 
+		rdtgroup_kmode_global_deactivate(rdtgrp);
 		rdtgroup_unassign_cntrs(rdtgrp);
 
 		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP ||

@@ -3346,6 +3371,7 @@ static void resctrl_fs_teardown(void)
 		return;
 
 	rmdir_all_sub();
+	rdtgroup_kmode_global_deactivate(&rdtgroup_default);
 	rdtgroup_unassign_cntrs(&rdtgroup_default);
 	mon_put_kn_priv();
 	rdt_pseudo_lock_release();

@@ -4381,6 +4407,8 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
 	u32 closid, rmid;
 	int cpu;
 
+	rdtgroup_kmode_global_deactivate(rdtgrp);
+
 	/* Give any tasks back to the parent group */
 	rdt_move_group_tasks(rdtgrp, prdtgrp, tmpmask);
 

@@ -4431,6 +4459,8 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
 	u32 closid, rmid;
 	int cpu;
 
+	rdtgroup_kmode_global_deactivate(rdtgrp);
+
 	/* Give any tasks back to the default group */
 	rdt_move_group_tasks(rdtgrp, &rdtgroup_default, tmpmask);
 

2.43.0

Kernel mode support allows the CPU's CLOSID and/or RMID associations to change during the transition from user mode to kernel mode. The resctrl file info/kernel_mode lists the current kernel mode configuration.

Extend info/kernel_mode with write support so user space can configure the active kernel mode. Validate kernel mode configuration before activation. Restrict backing groups to shareable mode so the kernel can use the group's CLOSID and/or RMID, which are shared with the group's tasks.

Document the interface in Documentation/filesystems/resctrl.rst.

---
v7: Refactored rdtgroup_kmode_global_activate and rdtgroup_kmode_global_deactivate().
Removed resctrl_kmode_switch().
Refactored resctrl_kmode_parse_group().
Handled the "group=" as the last parameter.

v6: Use flattened resctrl_kmode and global_enable_per_cpu. Omitted
parameters keep the values shown by reading the file. Drop the ctrl=inherit;mon=inherit no-op. Parse options with strsep(), as the other resctrl writers do, and split validation out of the write path.
https://lore.kernel.org/lkml/[email protected]/

v5: Adapt to the resctrl_kcfg caps and active split. Accept
assign_global_enable_per_cpu:ctrl=;mon=;group=<ctrl>/<mon>/ writes. Treat both ctrl=inherit and mon=inherit as a no-op. Use group=// for the default control group. Show associated kmode files through resctrl_hidden_files_set_visible().
Added kmode check in rdtgroup_locksetup_enter(), rdtgroup_mode_write() and rdtgroup_rename().
Simplify kernel-doc for write and activate helpers.

v4: Validate input before tearing down the active association so failures
retain the previous association. Allow assign_mon on CTRL_MON and MON groups; require a control group when control assignment is inherit.

v3: New patch to make info/kernel_mode writable.

---
 Documentation/filesystems/resctrl.rst |  40 ++++
 fs/resctrl/rdtgroup.c                 | 274 +++++++++++++++++++++++++-
 2 files changed, 313 insertions(+), 1 deletion(-)

index e57361dd342e..1a1877222624 100644
@@ -608,6 +608,46 @@ The top level of the "info" directory contains the following file(s):
 		[inherit_user]
 		global_enable_per_cpu:ctrl=assign;mon=assign;group=//
 
+	Writes use the same line format as reads, without square brackets and
+	with a trailing newline. Errors are reported in "info/last_cmd_status".
+
+	"inherit_user":
+		Written without parameters.
+
+	"global_enable_per_cpu":
+		"ctrl=", "mon=", and "group=" are optional.
+
+		An omitted "ctrl=", "mon=" and "group=" keeps the default values.
+
+		If "mon=inherit", or monitoring is unsupported, "group=" must
+		reference a CTRL_MON group.
+
+		The "ctrl=inherit" together with "mon=inherit" is rejected.
+
+		A MON group requires "mon=assign".
+
+		"group=" must be the last parameter. The group must be in shareable mode.
+
+		Selecting a new kernel mode, parameter, or group replaces the active
+		mode and updates "kmode_cpus" to include all online CPUs.
+
+		Empty values are rejected.
+
+		Example::
+
+			# mkdir ctrl1
+			# echo "global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1//" \
+				> info/kernel_mode
+
+			# cat info/kernel_mode
+			inherit_user
+			[global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1//]
+
+			# echo "inherit_user" > info/kernel_mode
+			# cat info/kernel_mode
+			[inherit_user]
+			global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
 Resource alloc and monitor groups
 =================================
 

2.43.0

The RESCTRL_GLOBAL_ENABLE_PER_CPU kernel mode uses the CLOSID and/or RMID of a backing group when transitioning from user mode to kernel mode. Since the CLOSID and RMID are shared with the group's tasks, the backing group must remain shareable while kernel mode is active.

Reject mode changes for a group backing the active kernel mode and for the parent of a backing monitor group.

Also reject reparenting a backing monitor group, as it would change the monitor group's parent and implicitly modify the active kernel mode configuration.

---
v7: Changelog update.
Moved the patch after the kernel_mode modify interface. Added code comments to justify reparenting.

v6: New patch to fix sashiko feedback.

    https://sashiko.dev/#/patchset/cover.1787772750.git.babu.moger%40amd.com
---
 fs/resctrl/rdtgroup.c | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

index 872755532fb4..1ad169e714a4 100644
@@ -1994,6 +1994,27 @@ static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)
 	return true;
 }
 
+/* True when @rdtgrp or one of its monitor groups backs the kernel mode. */
+static bool rdtgroup_has_kmode_global(struct rdtgroup *rdtgrp)
+{
+	struct rdtgroup *child;
+
+	if (!resctrl_kmode.rdtgrp)
+		return false;
+
+	if (resctrl_kmode.rdtgrp == rdtgrp)
+		return true;
+
+	if (rdtgrp->type != RDTCTRL_GROUP)
+		return false;
+
+	list_for_each_entry(child, &rdtgrp->mon.crdtgrp_list, mon.crdtgrp_list)
+		if (child == resctrl_kmode.rdtgrp)
+			return true;
+
+	return false;
+}
+
 /*
  * rdtgroup_mode_write - Modify the resource group's mode
  */

@@ -2034,6 +2055,12 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
 		goto out;
 	}
 
+	if (rdtgroup_has_kmode_global(rdtgrp)) {
+		rdt_last_cmd_puts("Cannot change mode for kernel mode backing group\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
 	if (!strcmp(buf, "shareable")) {
 		if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
 			ret = rdtgroup_locksetup_exit(rdtgrp);

@@ -4922,6 +4949,19 @@ static int rdtgroup_rename(struct kernfs_node *kn,
 		goto out;
 	}
 
+	/*
+	 * Changing the parent of a group backing the active kernel mode changes
+	 * the effective CLOSID and/or RMID configuration applied to kernel work.
+	 * Reject such moves and require kernel mode to be reconfigured explicitly
+	 * instead.
+	 */
+	if (rdtgrp == resctrl_kmode.rdtgrp &&
+	    rdtgrp->mon.parent != new_prdtgrp) {
+		rdt_last_cmd_puts("Cannot move a group that backs kernel mode\n");
+		ret = -EPERM;
+		goto out;
+	}
+
 	/*
 	 * Allocate the cpumask for use in mongrp_reparent() to avoid the
 	 * possibility of failing to allocate it after kernfs_rename() has

2.43.0

The kmode_cpus and kmode_cpus_list files are specific to the global per-CPU kernel mode. These files are created in a hidden state when an rdtgroup is created.

Introduce resctrl_kmode_global_files_show() to control the visibility of these files for individual resource groups. Show the files only for the group backing the active global per-CPU kernel mode and hide them for all other groups.

---
v7: Updated the prototype rdtgroup_kmode_global_files_show().
Moved the patch after both rdtgroup_kmode_global_activate() and rdtgroup_kmode_global_deactivate() is Introduced, so the change is symmetrical.
Updated the changelog.

v6: Helper named for the global per-CPU files instead of unhiding every
hidden file on the group.
https://lore.kernel.org/lkml/[email protected]/

---
 fs/resctrl/rdtgroup.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

index 1ad169e714a4..86bd6a66812b 100644
@@ -1136,6 +1136,24 @@ static int resctrl_kernel_mode_show(struct kernfs_open_file *of,
 	return 0;
 }
 
+/* Show or hide kmode_cpus and kmode_cpus_list on @kn. */
+static void resctrl_kmode_global_files_show(struct rdtgroup *rdtgrp, bool show)
+{
+	struct kernfs_node *kn_file;
+
+	kn_file = kernfs_find_and_get(rdtgrp->kn, "kmode_cpus");
+	if (kn_file) {
+		kernfs_show(kn_file, show);
+		kernfs_put(kn_file);
+	}
+
+	kn_file = kernfs_find_and_get(rdtgrp->kn, "kmode_cpus_list");
+	if (kn_file) {
+		kernfs_show(kn_file, show);
+		kernfs_put(kn_file);
+	}
+}
+
 /*
  * resctrl_kmode_global_online_cpu() - Program kernel mode association for @cpu
  * @cpu: CPU that has just been brought online

@@ -1194,6 +1212,7 @@ static void rdtgroup_kmode_global_deactivate(struct rdtgroup *rdtgrp)
 
 	resctrl_configure_kmode_global(&rdtgrp->kmode_cpu_mask, rdtgrp, false);
 	cpumask_clear(&rdtgrp->kmode_cpu_mask);
+	resctrl_kmode_global_files_show(rdtgrp, false);
 
 	resctrl_kmode.rdtgrp = NULL;
 	resctrl_kmode.active = RESCTRL_INHERIT_USER;

@@ -1214,6 +1233,7 @@ static void rdtgroup_kmode_global_activate(struct rdtgroup *rdtgrp,
 
 	cpumask_copy(&rdtgrp->kmode_cpu_mask, cpu_online_mask);
 	resctrl_configure_kmode_global(&rdtgrp->kmode_cpu_mask, rdtgrp, true);
+	resctrl_kmode_global_files_show(rdtgrp, true);
 }
 
 static int resctrl_kmode_parse_state(char *val, enum kmode_state *state)

2.43.0

The global per-CPU kernel mode allows CPUs to use different CLOSID and/or RMID associations when transitioning from user mode to kernel mode. The mode is enabled on a per-CPU basis, and kmode_cpus and kmode_cpus_list expose the CPUs used by active kernel mode.

Add write support so userspace can update the CPUs used by the active kernel mode.

Document the interface in Documentation/filesystems/resctrl.rst.

---
v7: Changelog update.
Last_cmd_status message update to keep the messages consistant. Minor changes to handle resctrl_kmode data structure changes.

v6: Use flattened resctrl_kmode and global_enable_per_cpu.
Fix the sashiko reported issue.
https://sashiko.dev/#/patchset/cover.1787772750.git.babu.moger%40amd.com?part=15
Update the changelog.

v5: Adapt to the resctrl_kcfg caps and active split. Use
resctrl_kcfg.active.ctrl_mode and mon_mode in kmode_cpus_write(). Use full ctrl=/mon= syntax in the kmode_cpus example.

v4: Empty masks are now allowed and updated masks are in rdtgroup->kmode_cpu_mask.
Updated the changelog.

v3: New patch to add "kmode_cpus" and "kmode_cpus_list" to support
kernel_modes.

---
 Documentation/filesystems/resctrl.rst |  30 +++++++
 fs/resctrl/rdtgroup.c                 | 114 +++++++++++++++++++++++++-
 2 files changed, 142 insertions(+), 2 deletions(-)

index 1a1877222624..2f0f6f01766f 100644
@@ -716,6 +716,36 @@ All groups contain the following files:
 "cpus_list":
 	Just like "cpus", only using ranges of CPUs instead of bitmasks.
 
+"kmode_cpus":
+	Accessible only within the resource group backing the active global
+	per-CPU kernel mode (see "info/kernel_mode").
+
+	Bitmask of the logical CPUs used by resource group backing the kernel
+	mode. When kernel mode is activated through info/kernel_mode, every
+	currently online CPU is included. CPUs that come online later are
+	automatically added to the group.
+
+	Writing a mask replaces the CPU mask. The mode is enabled on CPUs
+	added to the mask and disabled on CPUs removed from it. An empty mask
+	disables the mode on all currently online CPUs. CPUs that come online
+	later are still automatically added to the group. The mask must
+	contain only online CPUs; masks naming offline CPUs are rejected.
+	Errors are reported in "info/last_cmd_status".
+	Example::
+
+	  # mkdir ctrl1
+	  # echo "global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1//" \
+	        > info/kernel_mode
+	  # echo 0-3 > ctrl1/kmode_cpus_list
+	  # cat ctrl1/kmode_cpus
+	  f
+	  # cat ctrl1/kmode_cpus_list
+	  0-3
+
+"kmode_cpus_list":
+	Just like "kmode_cpus", only using ranges of CPUs instead of bitmasks.
+	Writable with the same semantics and restrictions as "kmode_cpus".
+
 
 When control is enabled all CTRL_MON groups will also contain:
 

2.43.0

Kernel mode configuration is exposed through info/kernel_mode, kmode_cpus, and kmode_cpus_list.

Provide examples showing how to query supported modes, configure kernel modes, and manage per-CPU kernel mode associations.

---
v7: Changelog update.
Documentation update.

v6: Renamed assign_global_enable_per_cpu to global_enable_per_cpu in the
examples. Replaced "policy" with "mode" throughout.

v5: Update the example for the collapsed kernel mode model:
inherit_user and assign_global_enable_per_cpu with ctrl=/mon= association options, and <CTRL_MON group>/<MON group>/ paths.

v4: New patch. Added examples in the documentation.

---
 Documentation/filesystems/resctrl.rst | 96 +++++++++++++++++++++++++++
 1 file changed, 96 insertions(+)

index 2f0f6f01766f..d161c51fe6e8 100644
@@ -1933,6 +1933,102 @@ View the llc occupancy snapshot::
   11234000
 
 
+Examples on working with kernel_mode
+====================================
+The kernel-mode interface spans three files:
+
+- ``info/kernel_mode`` - select the kernel mode, configuration and backing group
+- ``kmode_cpus`` / ``kmode_cpus_list`` - view or adjust the CPU scope of the
+  active mode (visible only on a backing group)
+
+The walkthrough below covers a typical workflow on a platform with PLZA
+(Privilege Level Zero Association) support:
+
+a. Discover supported kernel modes
+::
+
+  # mount -t resctrl resctrl /sys/fs/resctrl/
+
+  # cat /sys/fs/resctrl/info/kernel_mode
+  [inherit_user]
+  global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
+``inherit_user`` is the active mode. Kernel work inherits the resource
+allocation and monitoring of the current user task. The second line
+shows that ``global_enable_per_cpu`` is available. In later reads, the
+square brackets move to the active mode.
+
+b. Create a control group "ctrl1". Activate global_enable_per_cpu mode on the
+control group.
+::
+
+  # mkdir /sys/fs/resctrl/ctrl1
+  # echo "global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1/" > \
+        /sys/fs/resctrl/info/kernel_mode
+
+  # cat /sys/fs/resctrl/info/kernel_mode
+  inherit_user
+  [global_enable_per_cpu:ctrl=assign;mon=inherit;group=ctrl1/]
+
+``ctrl=assign`` uses ctrl1's allocation.
+
+``mon=inherit`` uses the monitoring of the user task.
+
+The group path uses the ``<CTRL_MON>/`` syntax; a control group alone is
+written as ``ctrl1/``.
+
+When the mode is active:
+
+- ``ctrl1/kmode_cpus`` and ``ctrl1/kmode_cpus_list`` become visible.
+- All currently online CPUs are added initially.
+- CPUs that come online later are added automatically.
+
+c. Narrow the CPU scope to 0-3
+::
+
+  # echo 0-3 > /sys/fs/resctrl/ctrl1/kmode_cpus_list
+  # cat /sys/fs/resctrl/ctrl1/kmode_cpus
+  f
+  # cat /sys/fs/resctrl/ctrl1/kmode_cpus_list
+  0-3
+
+The write replaces the CPU mask:
+
+- The mode is enabled on CPUs added to the mask.
+- The mode is disabled on CPUs removed from the mask.
+- Offline CPUs in the mask are rejected.
+- An empty mask disables the mode on all currently online CPUs, but later
+  hotplug CPUs are still added automatically.
+
+d. Create a monitor group and activate the mode on the monitor group.
+::
+
+  # mkdir /sys/fs/resctrl/ctrl1/mon_groups/mon1
+  # echo "global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1/mon1/" > \
+        /sys/fs/resctrl/info/kernel_mode
+
+  # cat /sys/fs/resctrl/info/kernel_mode
+  inherit_user
+  [global_enable_per_cpu:ctrl=assign;mon=assign;group=ctrl1/mon1/]
+
+Kernel mode work uses ctrl1's allocation (``ctrl=assign``) and mon1's monitoring
+(``mon=assign``).
+
+- Resets the CPU scope to all currently online CPUs.
+- Hides ``kmode_cpus`` and ``kmode_cpus_list`` on the previous group.
+- Shows them under ``ctrl1/mon_groups/mon1/``.
+
+e. Return to inherit_user
+::
+
+  # echo "inherit_user" > /sys/fs/resctrl/info/kernel_mode
+  # cat /sys/fs/resctrl/info/kernel_mode
+  [inherit_user]
+  global_enable_per_cpu:ctrl=assign;mon=assign;group=//
+
+Writing ``inherit_user`` clears the active kernel mode and hides ``kmode_cpus``
+and ``kmode_cpus_list`` because no group is backing the kernel mode.
+
 Examples on working with mbm_assign_mode
 ========================================
 

2.43.0