arm64: mpam: Document when to set arm64.nompam
Subsystems (3)
PATCH v3 1/1 arm64: mpam: Document when to set arm64.nompam
arm64.nompam skips the MPAM2_EL2 and MPAMHCR_EL2 writes in finalise_el2_state and leaves the ARM64_MPAM cpucap unset. Where EL3 firmware has cleared MPAM3_EL3.TRAPLOWER, the EL2 trap controls are left unwritten, at reset values that are UNKNOWN, and KVM still hides MPAM from the guest but no longer enables the traps that stop a guest from using it. KVM never saves or restores MPAM0_EL1, MPAM1_EL1 and MPAMSM_EL1, so what one guest writes reaches the next guest and the host. Nor can the kernel make the EL2 writes unconditionally: they trap to EL3 on the firmware the option exists for, and TRAPLOWER cannot be read below EL3.
Say so in mpam.rst, and add the rule and a pointer to the kernel-parameters entry.
---
Notes:
Changes since v2:
- Add MPAMSM_EL1 to the registers a guest can reach (Ben).
- Carry Bradley's Reviewed-by, dropped at v2 and reaffirmed there.
v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
.../admin-guide/kernel-parameters.txt | 3 ++- Documentation/arch/arm64/mpam.rst | 26 +++++++++++++++++++ 2 files changed, 28 insertions(+), 1 deletion(-)
Documentation/admin-guide/kernel-parameters.txt +2 -1
@@ -575,7 +575,8 @@ Kernel parameters Set instructions support arm64.nompam [ARM64] Unconditionally disable Memory Partitioning And - Monitoring support + Monitoring support. Only for a machine that does not + boot without it. See Documentation/arch/arm64/mpam.rst arm64.nomte [ARM64] Unconditionally disable Memory Tagging Extension support
Documentation/arch/arm64/mpam.rst +26 -0
@@ -87,6 +87,32 @@ The supported features are: MBWU monitors can be exposed to the user after support for more monitoring scopes is added to resctrl. +Command line parameters +======================= + +arm64.nompam +------------ +Firmware controls MPAM through two bits of MPAM3_EL3. MPAMEN enables +it: while set, the PARTID and PMG in the MPAMn_ELx registers label the +CPU's memory requests. TRAPLOWER, which resets to 1, traps accesses to +the MPAM system registers from the lower exception levels to EL3. +Firmware must clear it, or handle the trap and emulate MPAM as disabled. +Where it does neither, the CPUs still advertise MPAM in the ID registers, +the kernel's MPAM register accesses trap to EL3, and the boot fails. +``arm64.nompam`` exists for that firmware: it makes the kernel treat the +CPUs as not implementing MPAM, so no MPAM system register is accessed. +Set it only on a machine that does not boot without it. + +It is not a way to turn MPAM off. Where firmware has cleared TRAPLOWER, +the option leaves the trap controls in MPAM2_EL2 and MPAMHCR_EL2 +unwritten, and their reset values are UNKNOWN. KVM still hides MPAM from +guests but no longer enables the traps that stop a guest from using it, +so a guest may be able to read and write MPAM0_EL1, MPAM1_EL1 and +MPAMSM_EL1. KVM does not save or restore them, so what one guest writes +is still there when the next guest runs on that CPU, and when the host +does. With MPAMEN set, EL0 and EL1 requests then carry that PARTID and +PMG. + Reporting Bugs ============== If you are not seeing the counters or controls you expect please share the
Signature · 1 line
2.39.5
Hi Fuad,
Quoted 11 lines · Fuad Tabba
arm64.nompam skips the
MPAM2_EL2andMPAMHCR_EL2writes infinalise_el2_stateand leaves theARM64_MPAMcpucap unset. Where EL3 firmware has cleared MPAM3_EL3.TRAPLOWER, the EL2 trap controls are left unwritten, at reset values that are UNKNOWN, and KVM still hides MPAM from the guest but no longer enables the traps that stop a guest from using it. KVM never saves or restoresMPAM0_EL1,MPAM1_EL1andMPAMSM_EL1, so what one guest writes reaches the next guest and the host. Nor can the kernel make the EL2 writes unconditionally: they trap to EL3 on the firmware the option exists for, and TRAPLOWER cannot be read below EL3.
Quoted 7 lines
Say so in
mpam.rst, and add the rule and a pointer to the kernel-parameters entry.
Thanks,
Ben
Hi Catalin, Will,
Quoted 12 lines · Fuad Tabba
arm64.nompam skips the
MPAM2_EL2andMPAMHCR_EL2writes infinalise_el2_stateand leaves theARM64_MPAMcpucap unset. Where EL3 firmware has cleared MPAM3_EL3.TRAPLOWER, the EL2 trap controls are left unwritten, at reset values that are UNKNOWN, and KVM still hides MPAM from the guest but no longer enables the traps that stop a guest from using it. KVM never saves or restoresMPAM0_EL1,MPAM1_EL1andMPAMSM_EL1, so what one guest writes reaches the next guest and the host. Nor can the kernel make the EL2 writes unconditionally: they trap to EL3 on the firmware the option exists for, and TRAPLOWER cannot be read below EL3.
Quoted 2 lines
Say so in
mpam.rst, and add the rule and a pointer to the kernel-parameters entry.
Gentle ping on this one; it has Ben's and Bradley's R-b. I think it still stands regardless of the other cpufeature work going on now. Anything else needed for it to go in?
Cheers,
/fuad
Quoted 73 lines
Suggested-by Ben Horgan <[email protected]>Link https://lore.kernel.org/all/CA+EHjTxeWxZiuSmnKLGLxTBXP4oJT7-LuffbPAyCSZZ5TW=5Ew@mail.gmail.com/ open hereReviewed-by Bradley Morgan <[email protected]>Signed-off-by Fuad Tabba <[email protected]>---
Notes:
Changes since v2:
- AddMPAMSM_EL1to the registers a guest can reach (Ben).
- Carry Bradley's Reviewed-by, dropped at v2 and reaffirmed there.v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/.../admin-guide/kernel-parameters.txt | 3 ++- Documentation/arch/arm64/mpam.rst | 26 +++++++++++++++++++ 2 files changed, 28 insertions(+), 1 deletion(-)Documentation/admin-guide/kernel-parameters.txt +2
-1@@ -575,7 +575,8 @@ Kernel parameters Set instructions support arm64.nompam [ARM64] Unconditionally disable Memory Partitioning And - Monitoring support + Monitoring support. Only for a machine that does not + boot without it. See Documentation/arch/arm64/mpam.rst arm64.nomte [ARM64] Unconditionally disable Memory Tagging Extension supportDocumentation/arch/arm64/mpam.rst +26
-1@@ -87,6 +87,32 @@ The supported features are: MBWU monitors can be exposed to the user after support for more monitoring scopes is added to resctrl. +Command line parameters +======================= + +arm64.nompam +------------ +Firmware controls MPAM through two bits of MPAM3_EL3. MPAMEN enables +it: while set, the PARTID and PMG in the MPAMn_ELx registers label the +CPU's memory requests. TRAPLOWER, which resets to 1, traps accesses to +the MPAM system registers from the lower exception levels to EL3. +Firmware must clear it, or handle the trap and emulate MPAM as disabled. +Where it does neither, the CPUs still advertise MPAM in the ID registers, +the kernel's MPAM register accesses trap to EL3, and the boot fails. +``arm64.nompam`` exists for that firmware: it makes the kernel treat the +CPUs as not implementing MPAM, so no MPAM system register is accessed. +Set it only on a machine that does not boot without it. + +It is not a way to turn MPAM off. Where firmware has cleared TRAPLOWER, +the option leaves the trap controls in MPAM2_EL2 and MPAMHCR_EL2 +unwritten, and their reset values are UNKNOWN. KVM still hides MPAM from +guests but no longer enables the traps that stop a guest from using it, +so a guest may be able to read and write MPAM0_EL1, MPAM1_EL1 and +MPAMSM_EL1. KVM does not save or restore them, so what one guest writes +is still there when the next guest runs on that CPU, and when the host +does. With MPAMEN set, EL0 and EL1 requests then carry that PARTID and +PMG. + Reporting Bugs ============== If you are not seeing the counters or controls you expect please share the --2.39.5
Catalin Marinas
maintainerQuoted 12 lines · Fuad Tabba
Quoted 11 lines · Fuad Tabba
arm64.nompam skips the
MPAM2_EL2andMPAMHCR_EL2writes infinalise_el2_stateand leaves theARM64_MPAMcpucap unset. Where EL3 firmware has cleared MPAM3_EL3.TRAPLOWER, the EL2 trap controls are left unwritten, at reset values that are UNKNOWN, and KVM still hides MPAM from the guest but no longer enables the traps that stop a guest from using it. KVM never saves or restoresMPAM0_EL1,MPAM1_EL1andMPAMSM_EL1, so what one guest writes reaches the next guest and the host. Nor can the kernel make the EL2 writes unconditionally: they trap to EL3 on the firmware the option exists for, and TRAPLOWER cannot be read below EL3.
Quoted 6 lines
Quoted 2 lines
Say so in
mpam.rst, and add the rule and a pointer to the kernel-parameters entry.Gentle ping on this one; it has Ben's and Bradley's R-b. I think it still stands regardless of the other cpufeature work going on now. Anything else needed for it to go in?
Normally I expect a pull request from Ben or James but they rarely send one in time ;). I'll apply it directly.
Signature · 1 line
Catalin
Quoted 11 lines · Fuad Tabba
arm64.nompam skips the
MPAM2_EL2andMPAMHCR_EL2writes infinalise_el2_stateand leaves theARM64_MPAMcpucap unset. Where EL3 firmware has cleared MPAM3_EL3.TRAPLOWER, the EL2 trap controls are left unwritten, at reset values that are UNKNOWN, and KVM still hides MPAM from the guest but no longer enables the traps that stop a guest from using it. KVM never saves or restoresMPAM0_EL1,MPAM1_EL1andMPAMSM_EL1, so what one guest writes reaches the next guest and the host. Nor can the kernel make the EL2 writes unconditionally: they trap to EL3 on the firmware the option exists for, and TRAPLOWER cannot be read below EL3.
Quoted 1 line
[...]
Applied to arm64 (for-next/mpam), thanks!
[1/1] arm64: mpam: Document when to set arm64.nompam
https://git.kernel.org/arm64/c/eb0fde98fee4
(and this wasn't a pure mpam patch, should have picked it directly ;))