arm_mpam: Add MPAM-Fb firmware support
Subsystems (3)
v4 cover arm_mpam: Add MPAM-Fb firmware support
Hi,
this is version 4 of the MPAM-Fb code, for firmware based MSC accesses. Thanks to Jonathan's fearless review, a lot of the error handling has been improved, mostly courtesy of new kernel toys like scoped_guard and ACQUIRE(). Also the error handling pattern has been unified, to be consistent within this series, but also to match common kernel patterns. However I did not reorder the changes significantly, although some patches have been squashed to avoid touching a line multiple times. Also Ben, Niyas and Lee noticed bugs and other subtle issues, which have been fixed. On top of that, the "nrdy" handling optimisation has been dropped (for now).
Find a changelog below. Based on v7.2-rc1.
=======================
The Arm MPAM specification defines Memory System Components (MSCs), which are devices that are programmed through an MMIO register frame. In some occasions this turned out to be too limiting: the MSC might be located behind a separate bus system (for instance inside an on-board controller), it might be mapped secure-only, or in a different processor socket without direct MMIO mapping. Also the MMIO access might be too slow or it would need to be filtered or otherwise access controlled. Finally there might be bugs in the MSC integration, which require a mediating firmware to be accessible.
To accommodate all those different use cases, the MPAM-Fb specification
[1] describes an alternative way to access MSCs. Accesses to an MSC would be wrapped in a message and communicated to the system using a shared-memory/mailbox system mostly mimicking the Arm SCMI spec. For ACPI systems, this would be abstracted through an ACPI PCC channel, which provides the shared-memory region and the mailbox trigger. We can lean on existing ACPI parsing code to register with these two subsystems, but cannot rely on the existing SCMI code in the kernel. This means we somewhat need to open code a very simplified SCMI handler, which just provides enough functionality for the very basic subset of SCMI that the MPAM-Fb spec requires.
The first six patches rework all MSC access wrappers to propagate error information. Pure MMIO based MSC accesses would never fail, but the MPAM-Fb access can go wrong in multiple ways. The patches have been split up purely for reviewing reasons, if the number is a problem, we could as well squash them.
Patch 7/10 solves a nasty problem: At the moment we protect stateful MSC register accesses (mon_sel) through a spinlock. Unfortunately the mailbox subsystem and the slow nature of the communication through this channel forbid MPAM-Fb access in atomic context. So this patch keeps using a spinlock for MMIO based accesses, but reverts to a mutex otherwise. We just deny taking the lock for MPAM-Fb in atomic context, ideally we wouldn't need that (no need to IPI another core when the MSC access does not need to be local to one particular core), or we simply deny that part of the functionality (access through perf).
Patch 8/10 adds the code to redirect MSC accesses through the PCC shmem/mailbox system.
Patch 9/10 avoids the error IRQ handler to do an MSC access when using MPAM_Fb, since those accesses cannot run in atomic context. The final patch 10/10 then adds the code to detect and store the PCC channel information from the ACPI tables, and eventually enables MPAM-Fb accesses.
This would enable systems where some MSCs are not accessible via MMIO to use those components anyway.
Please have a look and test!
Cheers,
Andre
[1] https://developer.arm.com/documentation/den0144/latest
Changes in v4:
- unify error check patterns
- use scoped_guard(mutex) and ACQUIRE to simplify error handling
- use struct kref for PCC channel refcount
- drop unused mailbox rx_callback function
- increase size of props[] array to hold new msc_id property
- set mailbox TX timeout based on ACPI MPAM table field
- use devm_mutex_init() to allow automatic cleanup
- avoid overwriting error value for arm,not-ready-us property read
- prune MPAM-Fb protocol token to fit into bitmask inside register
- check return value for AIDR read
- use two separate MPAM-Fb protocol payload structs
- prune header list for mpam_fb.c
- drop "nrdy handled as an error" change
- fix message size field in MPAM-Fb protocol shmem area
- drop message size return from MPAM-Fb message build functions
- bring back IRQ flag in MPAM-Fb protocol header
Changes in v3:
- drop inner/outer mon_sel lock patch, replace with simpler version
- harmonise code patterns in error propagation changes
- drop mon_sel lock before erroring out in mpam_ris_hw_probe_csu_nrdy()
- refactor mpam_msc_read_mbwu_l() to return an error
- drop special NRDY handling in __ris_msmon_read()
- add IRQ numbers in error interrupt handler to help pinpoint failure
- refactor MPAM-Fb message generation to accommodate more than read/write
- check MPAM-Fb protocol version at probe time
- drop mpam_fb.h header, merge into mpam_internal.h
- translate MPAM-Fb error code in Linux codes where needed
- clear IRQ flag bit in MPAM-Fb protocol header
- drop unneeded endianness conversions when crafting MPAM-Fb message
- use C struct to model MPAM-Fb message payload
Changes in v2:
- add patches to add error propagation to MSC access wrappers
- drop former patch 1/5 (not needed)
- drop lock in mpam_reprogram_msc(), to avoid double lock
- add support for multiple MSCs per PCC channel
- adjust SCMI protocol code to use a PCC subtype 3 channel
- let PCC code handle the PCC channel negotiation (due to subtype 3)
- drop SCMI names in shmem offsets, and use existing PCC type 3 struct
- adjust shmem field offsets to match MPAM-Fb spec, not pure SCMI
- prevent MPAM-Fb calls inside atomic smp_call_function_any() payload
- skip all MSC accesses inside the IRQ handler when using MPAM-Fb
Andre Przywara (10):
arm_mpam: let low level MSC accessors return an error
arm_mpam: propagate MSC access errors for hw_probe functions
arm_mpam: propagate MSC access errors for MBWU counters
arm_mpam: propagate MSC access errors for msmon helpers
arm_mpam: propagate MSC access errors for __ris_msmon_read()
arm_mpam: propagate MSC access errors for state saving function
arm_mpam: prepare mon_sel locking for MPAM-Fb
arm_mpam: add MPAM-Fb MSC firmware access support
arm_mpam: prevent MPAM-Fb accesses inside IRQ handler
arm_mpam: detect and enable MPAM-Fb PCC support
drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 648 ++++++++++++++++++++++++-------- drivers/resctrl/mpam_fb.c | 197 ++++++++++ drivers/resctrl/mpam_internal.h | 60 ++- include/linux/arm_mpam.h | 2 +- 6 files changed, 739 insertions(+), 176 deletions(-) create mode 100644 drivers/resctrl/mpam_fb.c
Signature · 1 line
2.43.0
PATCH v4 1/10 arm_mpam: let low level MSC accessors return an error
The upcoming MPAM-Fb support does not use MMIO primitives to access an MSC, but employs a shared-memory/doorbell based firmware protocol. Its complexity means that is must be able to handle errors, whereas we always assume an MMIO based MSC access succeeds today.
Change the __mpam_read_reg() low level accessor function to return the requested data through a pointer, and return an error code instead. Also change the __mpam_write_reg() accessor function to return an error code. At the moment this is always 0, but this will change with alternative MSC access methods.
Change all direct users of those MSC wrappers to comply with the new prototypes, though the errors are not propagated all the way up yet.
--- drivers/resctrl/mpam_devices.c | 204 +++++++++++++++++++++------------ 1 file changed, 130 insertions(+), 74 deletions(-)
drivers/resctrl/mpam_devices.c +130 -74
@@ -177,59 +177,68 @@ static void mpam_assert_partid_sizes_fixed(void)WARN_ON_ONCE (!partid_max_published); } -static u32 __mpam_read_reg (struct mpam_msc *msc,u16 reg) +static int __mpam_read_reg (struct mpam_msc *msc,u16 reg,u32 *res) {WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); -return readl_relaxed (msc->mapped_hwpage + reg); + *res =readl_relaxed (msc->mapped_hwpage + reg); + +return 0 ; } -static inline u32 _mpam_read_partsel_reg (struct mpam_msc *msc,u16 reg) +static inline int _mpam_read_partsel_reg (struct mpam_msc *msc,u16 reg, +u32 *res) {lockdep_assert_held_once (&msc->part_sel_lock); -return __mpam_read_reg (msc, reg); +return __mpam_read_reg (msc, reg, res); } -#define mpam_read_partsel_reg (msc, reg) _mpam_read_partsel_reg(msc, MPAMF_##reg) +#define mpam_read_partsel_reg (msc, reg, res) _mpam_read_partsel_reg(msc, MPAMF_##reg, res) -static void __mpam_write_reg (struct mpam_msc *msc,u16 reg,u32 val) +static int __mpam_write_reg (struct mpam_msc *msc,u16 reg,u32 val) {WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility));writel_relaxed (val, msc->mapped_hwpage + reg); + +return 0 ; } -static inline void _mpam_write_partsel_reg (struct mpam_msc *msc,u16 reg,u32 val) +static inline int _mpam_write_partsel_reg (struct mpam_msc *msc,u16 reg,u32 val) {lockdep_assert_held_once (&msc->part_sel_lock); -__mpam_write_reg (msc, reg, val); +return __mpam_write_reg (msc, reg, val); }#define mpam_write_partsel_reg (msc, reg, val) _mpam_write_partsel_reg(msc, MPAMCFG_##reg, val) -static inline u32 _mpam_read_monsel_reg (struct mpam_msc *msc,u16 reg) +static inline int _mpam_read_monsel_reg (struct mpam_msc *msc,u16 reg, +u32 *res) {mpam_mon_sel_lock_held (msc); -return __mpam_read_reg (msc, reg); +return __mpam_read_reg (msc, reg, res); } -#define mpam_read_monsel_reg (msc, reg) _mpam_read_monsel_reg(msc, MSMON_##reg) +#define mpam_read_monsel_reg (msc, reg, res) _mpam_read_monsel_reg(msc, MSMON_##reg, res) -static inline void _mpam_write_monsel_reg (struct mpam_msc *msc,u16 reg,u32 val) +static inline int _mpam_write_monsel_reg (struct mpam_msc *msc,u16 reg,u32 val) {mpam_mon_sel_lock_held (msc); -__mpam_write_reg (msc, reg, val); +return __mpam_write_reg (msc, reg, val); }#define mpam_write_monsel_reg (msc, reg, val) _mpam_write_monsel_reg(msc, MSMON_##reg, val)static bool mpam_msc_check_aidr (struct mpam_msc *msc) { -u32 aidr =__mpam_read_reg (msc,MPAMF_AIDR ); -u32 major =FIELD_GET (MPAMF_AIDR_ARCH_MAJOR_REV , aidr); -u32 minor =FIELD_GET (MPAMF_AIDR_ARCH_MINOR_REV , aidr); +u32 aidr, major, minor; +if (__mpam_read_reg (msc,MPAMF_AIDR , &aidr)) +return false; + + major =FIELD_GET (MPAMF_AIDR_ARCH_MAJOR_REV , aidr); + minor =FIELD_GET (MPAMF_AIDR_ARCH_MINOR_REV , aidr);/* * v0.0 and >v2.x aren't supported, but anything else should be backward * compatible to v0.1 or v1.0.
@@ -242,25 +251,39 @@ static bool mpam_msc_check_aidr(struct mpam_msc *msc)return true; } -static u64 mpam_msc_read_idr (struct mpam_msc *msc) +static int mpam_msc_read_idr (struct mpam_msc *msc,u64 *res) { -u64 idr_high =0 , idr_low; +u32 idr_high =0 , idr_low; +int ret;lockdep_assert_held (&msc->part_sel_lock); - idr_low =mpam_read_partsel_reg (msc,IDR ); -if (FIELD_GET (MPAMF_IDR_EXT , idr_low)) - idr_high =mpam_read_partsel_reg (msc,IDR +4 ); + ret =mpam_read_partsel_reg (msc,IDR , &idr_low); +if (ret) +return ret; + +if (FIELD_GET (MPAMF_IDR_EXT , idr_low)) { + ret =mpam_read_partsel_reg (msc,IDR +4 , &idr_high); +if (ret) +return ret; + } -return (idr_high <<32 ) | idr_low; + *res = ((u64 )idr_high <<32 ) | idr_low; + +return 0 ; } -static void mpam_msc_clear_esr (struct mpam_msc *msc) +static int mpam_msc_clear_esr (struct mpam_msc *msc) { -u64 esr_low =__mpam_read_reg (msc,MPAMF_ESR ); +u32 esr_low; +int ret; + + ret =__mpam_read_reg (msc,MPAMF_ESR , &esr_low); +if (ret) +return ret;if (!esr_low) -return ; +return 0 ;/* * Clearing the high/low bits of MPAMF_ESR can not be atomic.
@@ -268,44 +291,57 @@ static void mpam_msc_clear_esr(struct mpam_msc *msc)* lower half prevent hardware from updating either half of the * register. */ -if (msc->has_extd_esr) -__mpam_write_reg (msc,MPAMF_ESR +4 ,0 ); -__mpam_write_reg (msc,MPAMF_ESR ,0 ); +if (msc->has_extd_esr) { + ret =__mpam_write_reg (msc,MPAMF_ESR +4 ,0 ); +if (ret) +return ret; + } + +return __mpam_write_reg (msc,MPAMF_ESR ,0 ); } -static u64 mpam_msc_read_esr (struct mpam_msc *msc) +static int mpam_msc_read_esr (struct mpam_msc *msc,u64 *res) { -u64 esr_high =0 , esr_low; +u32 esr_high =0 , esr_low; +int ret; - esr_low =__mpam_read_reg (msc,MPAMF_ESR ); -if (msc->has_extd_esr) - esr_high =__mpam_read_reg (msc,MPAMF_ESR +4 ); + ret =__mpam_read_reg (msc,MPAMF_ESR , &esr_low); +if (ret) +return ret; -return (esr_high <<32 ) | esr_low; +if (msc->has_extd_esr) { + ret =__mpam_read_reg (msc,MPAMF_ESR +4 , &esr_high); +if (ret) +return ret; + } + + *res = ((u64 )esr_high <<32 ) | esr_low; + +return 0 ; } -static void __mpam_part_sel_raw (u32 partsel,struct mpam_msc *msc) +static int __mpam_part_sel_raw (u32 partsel,struct mpam_msc *msc) {lockdep_assert_held (&msc->part_sel_lock); -mpam_write_partsel_reg (msc,PART_SEL , partsel); +return mpam_write_partsel_reg (msc,PART_SEL , partsel); } -static void __mpam_part_sel (u8 ris_idx,u16 partid,struct mpam_msc *msc) +static int __mpam_part_sel (u8 ris_idx,u16 partid,struct mpam_msc *msc) {u32 partsel =FIELD_PREP (MPAMCFG_PART_SEL_RIS , ris_idx) |FIELD_PREP (MPAMCFG_PART_SEL_PARTID_SEL , partid); -__mpam_part_sel_raw (partsel, msc); +return __mpam_part_sel_raw (partsel, msc); } -static void __mpam_intpart_sel (u8 ris_idx,u16 intpartid,struct mpam_msc *msc) +static int __mpam_intpart_sel (u8 ris_idx,u16 intpartid,struct mpam_msc *msc) {u32 partsel =FIELD_PREP (MPAMCFG_PART_SEL_RIS , ris_idx) |FIELD_PREP (MPAMCFG_PART_SEL_PARTID_SEL , intpartid) |MPAMCFG_PART_SEL_INTERNAL ; -__mpam_part_sel_raw (partsel, msc); +return __mpam_part_sel_raw (partsel, msc); }int mpam_register_requestor (u16 partid_max,u8 pmg_max)
@@ -774,13 +810,13 @@ static bool mpam_ris_hw_probe_csu_nrdy(struct mpam_msc_ris *ris)mpam_write_monsel_reg (msc,CFG_CSU_CTL , ctl_val);_mpam_write_monsel_reg (msc,MSMON_CSU ,MSMON___NRDY ); - now =_mpam_read_monsel_reg (msc,MSMON_CSU ); +_mpam_read_monsel_reg (msc,MSMON_CSU , &now); can_set = now &MSMON___NRDY ;_mpam_write_monsel_reg (msc,MSMON_CSU ,0 );/* Configuration change to try and coax hardware into setting nrdy */ mpam_write_monsel_reg (msc,CFG_CSU_FLT ,0x1 ); - now =_mpam_read_monsel_reg (msc,MSMON_CSU ); +_mpam_read_monsel_reg (msc,MSMON_CSU , &now); can_clear = !(now &MSMON___NRDY );mpam_mon_sel_unlock (msc);
@@ -800,7 +836,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)/* Cache Capacity Partitioning */ if (FIELD_GET (MPAMF_IDR_HAS_CCAP_PART , ris->idr)) { -u32 ccap_features =mpam_read_partsel_reg (msc,CCAP_IDR ); +u32 ccap_features; + +mpam_read_partsel_reg (msc,CCAP_IDR , &ccap_features); props->cmax_wd =FIELD_GET (MPAMF_CCAP_IDR_CMAX_WD , ccap_features); if (props->cmax_wd &&
@@ -823,7 +861,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)/* Cache Portion partitioning */ if (FIELD_GET (MPAMF_IDR_HAS_CPOR_PART , ris->idr)) { -u32 cpor_features =mpam_read_partsel_reg (msc,CPOR_IDR ); +u32 cpor_features; + +mpam_read_partsel_reg (msc,CPOR_IDR , &cpor_features); props->cpbm_wd =FIELD_GET (MPAMF_CPOR_IDR_CPBM_WD , cpor_features);if (props->cpbm_wd)
@@ -832,7 +872,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)/* Memory bandwidth partitioning */ if (FIELD_GET (MPAMF_IDR_HAS_MBW_PART , ris->idr)) { -u32 mbw_features =mpam_read_partsel_reg (msc,MBW_IDR ); +u32 mbw_features; + +mpam_read_partsel_reg (msc,MBW_IDR , &mbw_features);/* portion bitmap resolution */ props->mbw_pbm_bits =FIELD_GET (MPAMF_MBW_IDR_BWPBM_WD , mbw_features);
@@ -860,7 +902,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)/* Priority partitioning */ if (FIELD_GET (MPAMF_IDR_HAS_PRI_PART , ris->idr)) { -u32 pri_features =mpam_read_partsel_reg (msc,PRI_IDR ); +u32 pri_features; + +mpam_read_partsel_reg (msc,PRI_IDR , &pri_features); props->intpri_wd =FIELD_GET (MPAMF_PRI_IDR_INTPRI_WD , pri_features);if (props->intpri_wd &&FIELD_GET (MPAMF_PRI_IDR_HAS_INTPRI , pri_features)) {
@@ -879,7 +923,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)/* Performance Monitoring */ if (FIELD_GET (MPAMF_IDR_HAS_MSMON , ris->idr)) { -u32 msmon_features =mpam_read_partsel_reg (msc,MSMON_IDR ); +u32 msmon_features; + +mpam_read_partsel_reg (msc,MSMON_IDR , &msmon_features);/* * If the firmware max-nrdy-us property is missing, the
@@ -892,7 +938,8 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)if (FIELD_GET (MPAMF_MSMON_IDR_MSMON_CSU , msmon_features)) {u32 csumonidr; - csumonidr =mpam_read_partsel_reg (msc,CSUMON_IDR ); +mpam_read_partsel_reg (msc,CSUMON_IDR , &csumonidr); + props->num_csu_mon =FIELD_GET (MPAMF_CSUMON_IDR_NUM_MON , csumonidr);if (props->num_csu_mon) {bool hw_managed;
@@ -915,7 +962,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris) }if (FIELD_GET (MPAMF_MSMON_IDR_MSMON_MBWU , msmon_features)) {bool has_long; -u32 mbwumon_idr =mpam_read_partsel_reg (msc,MBWUMON_IDR ); +u32 mbwumon_idr; + +mpam_read_partsel_reg (msc,MBWUMON_IDR , &mbwumon_idr); props->num_mbwu_mon =FIELD_GET (MPAMF_MBWUMON_IDR_NUM_MON , mbwumon_idr);if (props->num_mbwu_mon) {
@@ -945,8 +994,11 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)*/ if (FIELD_GET (MPAMF_IDR_HAS_PARTID_NRW , ris->idr) && class->type !=MPAM_CLASS_UNKNOWN ) { -u32 nrwidr =mpam_read_partsel_reg (msc,PARTID_NRW_IDR ); -u16 partid_max =FIELD_GET (MPAMF_PARTID_NRW_IDR_INTPARTID_MAX , nrwidr); +u16 partid_max; +u32 nrwidr; + +mpam_read_partsel_reg (msc,PARTID_NRW_IDR , &nrwidr); + partid_max =FIELD_GET (MPAMF_PARTID_NRW_IDR_INTPARTID_MAX , nrwidr);mpam_set_feature (mpam_feat_partid_nrw, props); msc->partid_max =min (msc->partid_max, partid_max);
@@ -970,8 +1022,9 @@ static int mpam_msc_hw_probe(struct mpam_msc *msc)/* Grab an IDR value to find out how many RIS there are */ mutex_lock (&msc->part_sel_lock); - idr =mpam_msc_read_idr (msc); - msc->iidr =mpam_read_partsel_reg (msc,IIDR ); +mpam_msc_read_idr (msc, &idr); +mpam_read_partsel_reg (msc,IIDR , &msc->iidr); +mutex_unlock (&msc->part_sel_lock);mpam_enable_quirks (msc);
@@ -985,7 +1038,7 @@ static int mpam_msc_hw_probe(struct mpam_msc *msc)for (ris_idx =0 ; ris_idx <= msc->ris_max; ris_idx++) {mutex_lock (&msc->part_sel_lock);__mpam_part_sel (ris_idx,0 , msc); - idr =mpam_msc_read_idr (msc); +mpam_msc_read_idr (msc, &idr);mutex_unlock (&msc->part_sel_lock); partid_max =FIELD_GET (MPAMF_IDR_PARTID_MAX , idr);
@@ -1039,24 +1092,24 @@ static u64 mpam_msc_read_mbwu_l(struct mpam_msc *msc) {int retry =3 ;u32 mbwu_l_low; -u64 mbwu_l_high1, mbwu_l_high2; +u32 mbwu_l_high1, mbwu_l_high2;mpam_mon_sel_lock_held (msc);WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); - mbwu_l_high2 =__mpam_read_reg (msc,MSMON_MBWU_L +4 ); +__mpam_read_reg (msc,MSMON_MBWU_L +4 , &mbwu_l_high2);do { mbwu_l_high1 = mbwu_l_high2; - mbwu_l_low =__mpam_read_reg (msc,MSMON_MBWU_L ); - mbwu_l_high2 =__mpam_read_reg (msc,MSMON_MBWU_L +4 ); +__mpam_read_reg (msc,MSMON_MBWU_L , &mbwu_l_low); +__mpam_read_reg (msc,MSMON_MBWU_L +4 , &mbwu_l_high2); retry--; }while (mbwu_l_high1 != mbwu_l_high2 && retry >0 );if (mbwu_l_high1 == mbwu_l_high2) -return (mbwu_l_high1 <<32 ) | mbwu_l_low; +return ((u64 )mbwu_l_high1 <<32 ) | mbwu_l_low;pr_warn ("Failed to read a stable value\n" );return MSMON___L_NRDY ;
@@ -1120,14 +1173,14 @@ static void read_msmon_ctl_flt_vals(struct mon_read *m, u32 *ctl_val,switch (m->type) {case mpam_feat_msmon_csu: - *ctl_val =mpam_read_monsel_reg (msc,CFG_CSU_CTL ); - *flt_val =mpam_read_monsel_reg (msc,CFG_CSU_FLT ); +mpam_read_monsel_reg (msc,CFG_CSU_CTL , ctl_val); +mpam_read_monsel_reg (msc,CFG_CSU_FLT , flt_val);break ;case mpam_feat_msmon_mbwu_31counter:case mpam_feat_msmon_mbwu_44counter:case mpam_feat_msmon_mbwu_63counter: - *ctl_val =mpam_read_monsel_reg (msc,CFG_MBWU_CTL ); - *flt_val =mpam_read_monsel_reg (msc,CFG_MBWU_FLT ); +mpam_read_monsel_reg (msc,CFG_MBWU_CTL , ctl_val); +mpam_read_monsel_reg (msc,CFG_MBWU_FLT , flt_val);break ;default :pr_warn ("Unexpected monitor type %d\n" , m->type);
@@ -1206,6 +1259,7 @@ static u64 mpam_msmon_overflow_val(enum mpam_device_features type,static void __ris_msmon_read (void *arg) {u64 now; +u32 now32;bool nrdy = false;bool config_mismatch;bool overflow = false;
@@ -1268,9 +1322,9 @@ static void __ris_msmon_read(void *arg)switch (m->type) {case mpam_feat_msmon_csu: - now =mpam_read_monsel_reg (msc,CSU ); - nrdy = now &MSMON___NRDY ; - now =FIELD_GET (MSMON___VALUE , now); +mpam_read_monsel_reg (msc,CSU , &now32); + nrdy = now32 &MSMON___NRDY ; + now =FIELD_GET (MSMON___VALUE , now32);if (mpam_has_quirk (IGNORE_CSU_NRDY , msc) && m->waited_timeout) nrdy = false;
@@ -1288,9 +1342,9 @@ static void __ris_msmon_read(void *arg)else now =FIELD_GET (MSMON___L_VALUE , now); }else { - now =mpam_read_monsel_reg (msc,MBWU ); - nrdy = now &MSMON___NRDY ; - now =FIELD_GET (MSMON___VALUE , now); +mpam_read_monsel_reg (msc,MBWU , &now32); + nrdy = now32 &MSMON___NRDY ; + now =FIELD_GET (MSMON___VALUE , now32); } if (mpam_has_quirk(T241_MBW_COUNTER_SCALE_64, msc) &&
@@ -1685,16 +1739,18 @@ static int mpam_save_mbwu_state(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , i) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx);mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); - - cur_flt =mpam_read_monsel_reg (msc,CFG_MBWU_FLT ); - cur_ctl =mpam_read_monsel_reg (msc,CFG_MBWU_CTL ); +mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); +mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl);mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 );if (mpam_ris_has_mbwu_long_counter (ris)) { val =mpam_msc_read_mbwu_l (msc);mpam_msc_zero_mbwu_l (msc); }else { - val =mpam_read_monsel_reg (msc,MBWU ); +u32 val32; + +mpam_read_monsel_reg (msc,MBWU , &val32); + val = val32;mpam_write_monsel_reg (msc,MBWU ,0 ); }
@@ -2465,7 +2521,7 @@ static irqreturn_t __mpam_irq_handler(int irq, struct mpam_msc *msc) &msc->accessibility)))return IRQ_NONE ; - reg =mpam_msc_read_esr (msc); +mpam_msc_read_esr (msc, ®); errcode =FIELD_GET (MPAMF_ESR_ERRCODE , reg);if (!errcode)
Signature · 1 line
2.43.0
Re: [PATCH v4 01/10] arm_mpam: let low level MSC accessors return an error
Quoted 3 lines · On Thu, 23 Jul 2026 17:54:45 +0200 Andre Przywara <[email protected]>…
The upcoming MPAM-Fb support does not use MMIO primitives to access an MSC, but employs a shared-memory/doorbell based firmware protocol. Its complexity means that is must be able to handle errors, whereas we
that it must
Quoted 7 lines
always assume an MMIO based MSC access succeeds today.
Change the
__mpam_read_reg()low level accessor function to return the requested data through a pointer, and return an error code instead. Also change the__mpam_write_reg()accessor function to return an error code. At the moment this is always 0, but this will change with alternative MSC access methods.
Be consistent on paragraph formatting. So blank line here. Check whole series for such tiny consistency problems.
Quoted 4 lines
Change all direct users of those MSC wrappers to comply with the new prototypes, though the errors are not propagated all the way up yet.
Signed-off-by Andre Przywara <[email protected]>
Some comments inline. A few are 'whilst you are here let us make this more readable'. I was in two minds about the temporary lack of handling of uninitialised (looking) variables but in the interests of expediency I suppose we can rely on the functions never actually failing at this point in the series.
I will note that if you did adding the returns from outer calls to inner you wouldn't get this problem. However you might get other problems!
Quoted 22 lines
--- drivers/resctrl/mpam_devices.c | 204 +++++++++++++++++++++------------ 1 file changed, 130 insertions(+), 74 deletions(-)drivers/resctrl/mpam_devices.c +3
-4-static u64 mpam_msc_read_idr (struct mpam_msc *msc) +static int mpam_msc_read_idr (struct mpam_msc *msc,u64 *res) { -u64 idr_high =0 , idr_low; +u32 idr_high =0 , idr_low; +int ret;lockdep_assert_held (&msc->part_sel_lock); - idr_low =mpam_read_partsel_reg (msc,IDR ); -if (FIELD_GET (MPAMF_IDR_EXT , idr_low))
drivers/resctrl/mpam_devices.c static bool mpam_msc_check_aidr(struct mpam_msc *msc) · line 269
Quoted 10 lines
drivers/resctrl/mpam_devices.c +9
-1- idr_high =mpam_read_partsel_reg (msc,IDR +4 ); + ret =mpam_read_partsel_reg (msc,IDR , &idr_low); +if (ret) +return ret; + +if (FIELD_GET (MPAMF_IDR_EXT , idr_low)) { + ret =mpam_read_partsel_reg (msc,IDR +4 , &idr_high); +if (ret) +return ret; + }
As a whilst you are here type of comment. I'd find this more obvious if it were
} else {
idr_high = 0;
}rather than assign a default at top of what is now a rather more complex function.
Quoted 19 lines
drivers/resctrl/mpam_devices.c +6
-6-return (idr_high <<32 ) | idr_low; + *res = ((u64 )idr_high <<32 ) | idr_low; + +return 0 ; } -static u64 mpam_msc_read_esr (struct mpam_msc *msc) +static int mpam_msc_read_esr (struct mpam_msc *msc,u64 *res) { -u64 esr_high =0 , esr_low; +u32 esr_high =0 , esr_low; +int ret; - esr_low =__mpam_read_reg (msc,MPAMF_ESR ); -if (msc->has_extd_esr) - esr_high =__mpam_read_reg (msc,MPAMF_ESR +4 );
drivers/resctrl/mpam_devices.c static void mpam_msc_clear_esr(struct mpam_msc *msc) · line 316
Quoted 10 lines
drivers/resctrl/mpam_devices.c +8
-1+ ret =__mpam_read_reg (msc,MPAMF_ESR , &esr_low); +if (ret) +return ret; -return (esr_high <<32 ) | esr_low; +if (msc->has_extd_esr) { + ret =__mpam_read_reg (msc,MPAMF_ESR +4 , &esr_high); +if (ret) +return ret; + }
Same as the other example above. } else { esr_high = 0; } would be slightly clearer now this whole function has gotten more complex.
Quoted 9 lines
drivers/resctrl/mpam_devices.c +4
-0+ + *res = ((u64 )esr_high <<32 ) | esr_low; + +return 0 ; }int mpam_register_requestor (u16 partid_max,u8 pmg_max)
drivers/resctrl/mpam_devices.c static bool mpam_ris_hw_probe_csu_nrdy(struct mpam_msc_ris *ris) · line 813
Quoted 5 lines
drivers/resctrl/mpam_devices.c +1
-1@@ -774,13 +810,13 @@ static bool mpam_ris_hw_probe_csu_nrdy(struct mpam_msc_ris *ris)mpam_write_monsel_reg (msc,CFG_CSU_CTL , ctl_val);_mpam_write_monsel_reg (msc,MSMON_CSU ,MSMON___NRDY ); - now =_mpam_read_monsel_reg (msc,MSMON_CSU ); +_mpam_read_monsel_reg (msc,MSMON_CSU , &now);
If an error were to occur and "now" not get assigned, would this being accessing undefined data? If checking return values doesn't make sense in here it should still not result in something static analysis might get annoyed by. This gets cleaned up in next patch so maybe that is just about ok. Alternative would be to just initialize any such variables in this patch and drop those initializations in next. Doing it just above the code that is going to change anyway would be cleanest.
now = 0; _mpam_read_monsel_reg(msc, MSMON_CSU, &now);
for example.
Hmm. I suppose this is a bit of a special case because we know the calls don't return errors at this point in the set.
So whilst I don't like it I'll not insist that you fix all the instances of this.
Quoted 8 lines
drivers/resctrl/mpam_devices.c +1
-1can_set = now &MSMON___NRDY ;_mpam_write_monsel_reg (msc,MSMON_CSU ,0 );/* Configuration change to try and coax hardware into setting nrdy */ mpam_write_monsel_reg (msc,CFG_CSU_FLT ,0x1 ); - now =_mpam_read_monsel_reg (msc,MSMON_CSU ); +_mpam_read_monsel_reg (msc,MSMON_CSU , &now); can_clear = !(now &MSMON___NRDY );
Re: [PATCH v4 01/10] arm_mpam: let low level MSC accessors return an error
Hi Jonathan,
Quoted 23 lines · Jonathan Cameron
Quoted 3 lines · On Thu, 23 Jul 2026 17:54:45 +0200 Andre Przywara <[email protected]>…
The upcoming MPAM-Fb support does not use MMIO primitives to access an MSC, but employs a shared-memory/doorbell based firmware protocol. Its complexity means that is must be able to handle errors, whereas we
that it must
Quoted 7 lines
always assume an MMIO based MSC access succeeds today.
Change the
__mpam_read_reg()low level accessor function to return the requested data through a pointer, and return an error code instead. Also change the__mpam_write_reg()accessor function to return an error code. At the moment this is always 0, but this will change with alternative MSC access methods.Be consistent on paragraph formatting. So blank line here. Check whole series for such tiny consistency problems.
Quoted 3 lines
Change all direct users of those MSC wrappers to comply with the new prototypes, though the errors are not propagated all the way up yet.
Quoted 7 lines
Quoted 1 line
Signed-off-by Andre Przywara <[email protected]>Some comments inline. A few are 'whilst you are here let us make this more readable'. I was in two minds about the temporary lack of handling of uninitialised (looking) variables but in the interests of expediency I suppose we can rely on the functions never actually failing at this point in the series.
I probably should have made this explicit in at least the cover letter, but that was the intention: for most of the patches, the lower level functions only ever return 0, so no error path will be triggered at this point. This allows to code to grow more slowly, and remaining issues are then fixed in later patches (7/10-9/10), before only the last patch actually activates MPAM-Fb, and introduces the possibility of an error return. So there is no real functional change, it's all refactoring. Interestingly Sashiko missed that, many complaints are about things that cannot happen before patch 10/10.
I will add a comment to that effect to the commit message and the cover letter.
Quoted 2 lines
I will note that if you did adding the returns from outer calls to inner you wouldn't get this problem. However you might get other problems!
Thanks for the suggestion, I might try it the next time I encounter a similar problem! I briefly looked at applying it here, but already struggled to identify those outer functions. So I think this "lower level first" is still more comprehensible.
Quoted 34 lines
Quoted 22 lines
--- drivers/resctrl/mpam_devices.c | 204 +++++++++++++++++++++------------ 1 file changed, 130 insertions(+), 74 deletions(-)drivers/resctrl/mpam_devices.c +3
-4-static u64 mpam_msc_read_idr (struct mpam_msc *msc) +static int mpam_msc_read_idr (struct mpam_msc *msc,u64 *res) { -u64 idr_high =0 , idr_low; +u32 idr_high =0 , idr_low; +int ret;lockdep_assert_held (&msc->part_sel_lock); - idr_low =mpam_read_partsel_reg (msc,IDR ); -if (FIELD_GET (MPAMF_IDR_EXT , idr_low))drivers/resctrl/mpam_devices.c static bool mpam_msc_check_aidr(struct mpam_msc *msc) · line 269
Quoted 10 lines
drivers/resctrl/mpam_devices.c +9
-1- idr_high =mpam_read_partsel_reg (msc,IDR +4 ); + ret =mpam_read_partsel_reg (msc,IDR , &idr_low); +if (ret) +return ret; + +if (FIELD_GET (MPAMF_IDR_EXT , idr_low)) { + ret =mpam_read_partsel_reg (msc,IDR +4 , &idr_high); +if (ret) +return ret; + }As a whilst you are here type of comment. I'd find this more obvious if it were
Quoted 5 lines
} else { idr_high = 0; }rather than assign a default at top of what is now a rather more complex function.
I see what you mean, makes sense.
Quoted 44 lines
Quoted 19 lines
drivers/resctrl/mpam_devices.c +6
-6-return (idr_high <<32 ) | idr_low; + *res = ((u64 )idr_high <<32 ) | idr_low; + +return 0 ; } -static u64 mpam_msc_read_esr (struct mpam_msc *msc) +static int mpam_msc_read_esr (struct mpam_msc *msc,u64 *res) { -u64 esr_high =0 , esr_low; +u32 esr_high =0 , esr_low; +int ret; - esr_low =__mpam_read_reg (msc,MPAMF_ESR ); -if (msc->has_extd_esr) - esr_high =__mpam_read_reg (msc,MPAMF_ESR +4 );drivers/resctrl/mpam_devices.c static void mpam_msc_clear_esr(struct mpam_msc *msc) · line 316
Quoted 10 lines
drivers/resctrl/mpam_devices.c +8
-1+ ret =__mpam_read_reg (msc,MPAMF_ESR , &esr_low); +if (ret) +return ret; -return (esr_high <<32 ) | esr_low; +if (msc->has_extd_esr) { + ret =__mpam_read_reg (msc,MPAMF_ESR +4 , &esr_high); +if (ret) +return ret; + }Same as the other example above. } else {
esr_high= 0; } would be slightly clearer now this whole function has gotten more complex.Quoted 11 lines
drivers/resctrl/mpam_devices.c +4
-0+ + *res = ((u64 )esr_high <<32 ) | esr_low; + +return 0 ; }int mpam_register_requestor (u16 partid_max,u8 pmg_max)@@ -774,13 +810,13 @@ static bool mpam_ris_hw_probe_csu_nrdy(struct mpam_msc_ris *ris)mpam_write_monsel_reg (msc,CFG_CSU_CTL , ctl_val);
Quoted 6 lines
drivers/resctrl/mpam_devices.c static bool mpam_ris_hw_probe_csu_nrdy(struct mpam_msc_ris *ris) · line 813
Quoted 3 lines
drivers/resctrl/mpam_devices.c +1
-1_mpam_write_monsel_reg (msc,MSMON_CSU ,MSMON___NRDY ); - now =_mpam_read_monsel_reg (msc,MSMON_CSU ); +_mpam_read_monsel_reg (msc,MSMON_CSU , &now);If an error were to occur and "now" not get assigned, would this being accessing undefined data?
But there will be no error at this point, as __mpam_read_reg() unconditionally still returns 0. And this missing error code check will be fixed in a later patch, before proper error returns get enabled.
If checking return values doesn't make sense
Quoted 11 lines
in here it should still not result in something static analysis might get annoyed by. This gets cleaned up in next patch so maybe that is just about ok. Alternative would be to just initialize any such variables in this patch and drop those initializations in next. Doing it just above the code that is going to change anyway would be cleanest.
now = 0; _mpam_read_monsel_reg(msc, MSMON_CSU, &now);for example.
Quoted 7 lines
Hmm. I suppose this is a bit of a special case because we know the calls don't return errors at this point in the set.
So whilst I don't like it I'll not insist that you fix all the instances of this.
Reviewed-by Jonathan Cameron <[email protected]>
Thanks!
Cheers,
Andre
Quoted 13 lines
Quoted 13 lines
drivers/resctrl/mpam_devices.c +1
-1can_set = now &MSMON___NRDY ;_mpam_write_monsel_reg (msc,MSMON_CSU ,0 );/* Configuration change to try and coax hardware into setting nrdy */ mpam_write_monsel_reg (msc,CFG_CSU_FLT ,0x1 ); - now =_mpam_read_monsel_reg (msc,MSMON_CSU ); +_mpam_read_monsel_reg (msc,MSMON_CSU , &now); can_clear = !(now &MSMON___NRDY );
PATCH v4 2/10 arm_mpam: propagate MSC access errors for hw_probe functions
In reply to Andre PrzywaraAllow the functions probing for MSC hardware and features to return an error, and propagate read and write errors from the lower level up. This uses some "scoped cleanup" functions like scoped_guard() and ACQUIRE() to avoid the complexity of error handling when a lock has been taken. Since the mon_sel_lock is a bit special (even more so in an upcoming patch), we define a new GUARD type for it.
--- drivers/resctrl/mpam_devices.c | 127 +++++++++++++++++++++----------- drivers/resctrl/mpam_internal.h | 8 ++ 2 files changed, 91 insertions(+), 44 deletions(-)
drivers/resctrl/mpam_devices.c +83 -44
@@ -794,38 +794,46 @@ static bool mpam_ris_hw_probe_csu_nrdy(struct mpam_msc_ris *ris)bool can_set, can_clear;struct mpam_msc *msc = ris->vmsc->msc; -if (WARN_ON_ONCE (!mpam_mon_sel_lock (msc))) +ACQUIRE (mon_sel_lock, guard)(msc); +if (ACQUIRE_ERR (mon_sel_lock, &guard))return false; mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL ,0 ) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel)) +return false;/* Hardware might ignore nrdy if it's not enabled */ ctl_val =MSMON_CFG_CSU_CTL_TYPE_CSU ; ctl_val |= MSMON_CFG_x_CTL_MATCH_PARTID; ctl_val |= MSMON_CFG_x_CTL_MATCH_PMG; ctl_val |= MSMON_CFG_x_CTL_EN; -mpam_write_monsel_reg (msc,CFG_CSU_FLT ,0 ); -mpam_write_monsel_reg (msc,CFG_CSU_CTL , ctl_val); +if (mpam_write_monsel_reg (msc,CFG_CSU_FLT ,0 )) +return false; +if (mpam_write_monsel_reg (msc,CFG_CSU_CTL , ctl_val)) +return false; -_mpam_write_monsel_reg (msc,MSMON_CSU ,MSMON___NRDY ); -_mpam_read_monsel_reg (msc,MSMON_CSU , &now); +if (_mpam_write_monsel_reg (msc,MSMON_CSU ,MSMON___NRDY )) +return false; +if (_mpam_read_monsel_reg (msc,MSMON_CSU , &now)) +return false; can_set = now &MSMON___NRDY ; -_mpam_write_monsel_reg (msc,MSMON_CSU ,0 ); +if (_mpam_write_monsel_reg (msc,MSMON_CSU ,0 )) +return false;/* Configuration change to try and coax hardware into setting nrdy */ -mpam_write_monsel_reg (msc,CFG_CSU_FLT ,0x1 ); -_mpam_read_monsel_reg (msc,MSMON_CSU , &now); +if (mpam_write_monsel_reg (msc,CFG_CSU_FLT ,0x1 )) +return false; +if (_mpam_read_monsel_reg (msc,MSMON_CSU , &now)) +return false; can_clear = !(now &MSMON___NRDY ); -mpam_mon_sel_unlock (msc);return (!can_set || !can_clear); } -static void mpam_ris_hw_probe (struct mpam_msc_ris *ris) +static int mpam_ris_hw_probe (struct mpam_msc_ris *ris) { -int err; +int fw_has_nrdy_err, err;struct mpam_msc *msc = ris->vmsc->msc;struct device *dev = &msc->pdev->dev;struct mpam_props *props = &ris->props;
@@ -838,7 +846,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)if (FIELD_GET (MPAMF_IDR_HAS_CCAP_PART , ris->idr)) {u32 ccap_features; -mpam_read_partsel_reg (msc,CCAP_IDR , &ccap_features); + err =mpam_read_partsel_reg (msc,CCAP_IDR , &ccap_features); +if (err) +return err; props->cmax_wd =FIELD_GET (MPAMF_CCAP_IDR_CMAX_WD , ccap_features); if (props->cmax_wd &&
@@ -863,7 +873,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)if (FIELD_GET (MPAMF_IDR_HAS_CPOR_PART , ris->idr)) {u32 cpor_features; -mpam_read_partsel_reg (msc,CPOR_IDR , &cpor_features); + err =mpam_read_partsel_reg (msc,CPOR_IDR , &cpor_features); +if (err) +return err; props->cpbm_wd =FIELD_GET (MPAMF_CPOR_IDR_CPBM_WD , cpor_features);if (props->cpbm_wd)
@@ -873,8 +885,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)/* Memory bandwidth partitioning */ if (FIELD_GET (MPAMF_IDR_HAS_MBW_PART , ris->idr)) {u32 mbw_features; - -mpam_read_partsel_reg (msc,MBW_IDR , &mbw_features); + err =mpam_read_partsel_reg (msc,MBW_IDR , &mbw_features); +if (err) +return err;/* portion bitmap resolution */ props->mbw_pbm_bits =FIELD_GET (MPAMF_MBW_IDR_BWPBM_WD , mbw_features);
@@ -903,8 +916,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)/* Priority partitioning */ if (FIELD_GET (MPAMF_IDR_HAS_PRI_PART , ris->idr)) {u32 pri_features; - -mpam_read_partsel_reg (msc,PRI_IDR , &pri_features); + err =mpam_read_partsel_reg (msc,PRI_IDR , &pri_features); +if (err) +return err; props->intpri_wd =FIELD_GET (MPAMF_PRI_IDR_INTPRI_WD , pri_features);if (props->intpri_wd &&FIELD_GET (MPAMF_PRI_IDR_HAS_INTPRI , pri_features)) {
@@ -925,20 +939,24 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)if (FIELD_GET (MPAMF_IDR_HAS_MSMON , ris->idr)) {u32 msmon_features; -mpam_read_partsel_reg (msc,MSMON_IDR , &msmon_features); + err =mpam_read_partsel_reg (msc,MSMON_IDR , &msmon_features); +if (err) +return err;/* * If the firmware max-nrdy-us property is missing, the * CSU counters can't be used. Should we wait forever? */ - err =device_property_read_u32 (&msc->pdev->dev, -"arm,not-ready-us" , - &msc->nrdy_usec); + fw_has_nrdy_err =device_property_read_u32 (&msc->pdev->dev, +"arm,not-ready-us" , + &msc->nrdy_usec);if (FIELD_GET (MPAMF_MSMON_IDR_MSMON_CSU , msmon_features)) {u32 csumonidr; -mpam_read_partsel_reg (msc,CSUMON_IDR , &csumonidr); + err =mpam_read_partsel_reg (msc,CSUMON_IDR , &csumonidr); +if (err) +return err; props->num_csu_mon =FIELD_GET (MPAMF_CSUMON_IDR_NUM_MON , csumonidr);if (props->num_csu_mon) {
@@ -956,7 +974,7 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)* Accept the missing firmware property if NRDY appears * un-implemented. */ -if (err && hw_managed) +if (fw_has_nrdy_err && hw_managed)dev_err_once (dev,"Counters are not usable because not-ready timeout was not provided by firmware." ); } }
@@ -964,7 +982,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)bool has_long;u32 mbwumon_idr; -mpam_read_partsel_reg (msc,MBWUMON_IDR , &mbwumon_idr); + err =mpam_read_partsel_reg (msc,MBWUMON_IDR , &mbwumon_idr); +if (err) +return err; props->num_mbwu_mon =FIELD_GET (MPAMF_MBWUMON_IDR_NUM_MON , mbwumon_idr);if (props->num_mbwu_mon) {
@@ -997,16 +1017,22 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris)u16 partid_max;u32 nrwidr; -mpam_read_partsel_reg (msc,PARTID_NRW_IDR , &nrwidr); + err =mpam_read_partsel_reg (msc,PARTID_NRW_IDR , &nrwidr); +if (err) +return err; + partid_max =FIELD_GET (MPAMF_PARTID_NRW_IDR_INTPARTID_MAX , nrwidr);mpam_set_feature (mpam_feat_partid_nrw, props); msc->partid_max =min (msc->partid_max, partid_max); } + +return 0 ; }static int mpam_msc_hw_probe (struct mpam_msc *msc) { +int ret;u64 idr;u16 partid_max;u8 ris_idx, pmg_max;
@@ -1021,11 +1047,14 @@ static int mpam_msc_hw_probe(struct mpam_msc *msc) }/* Grab an IDR value to find out how many RIS there are */ -mutex_lock (&msc->part_sel_lock); -mpam_msc_read_idr (msc, &idr); -mpam_read_partsel_reg (msc,IIDR , &msc->iidr); - -mutex_unlock (&msc->part_sel_lock); + scoped_guard(mutex, &msc->part_sel_lock) { + ret =mpam_msc_read_idr (msc, &idr); +if (ret) +return ret; + ret =mpam_read_partsel_reg (msc,IIDR , &msc->iidr); +if (ret) +return ret; + }mpam_enable_quirks (msc);
@@ -1036,10 +1065,15 @@ static int mpam_msc_hw_probe(struct mpam_msc *msc) msc->pmg_max =FIELD_GET (MPAMF_IDR_PMG_MAX , idr);for (ris_idx =0 ; ris_idx <= msc->ris_max; ris_idx++) { -mutex_lock (&msc->part_sel_lock); -__mpam_part_sel (ris_idx,0 , msc); -mpam_msc_read_idr (msc, &idr); -mutex_unlock (&msc->part_sel_lock); + scoped_guard(mutex, &msc->part_sel_lock) { + ret =__mpam_part_sel (ris_idx,0 , msc); +if (ret) +return ret; + + ret =mpam_msc_read_idr (msc, &idr); +if (ret) +return ret; + } partid_max =FIELD_GET (MPAMF_IDR_PARTID_MAX , idr); pmg_max =FIELD_GET (MPAMF_IDR_PMG_MAX , idr);
@@ -1047,17 +1081,22 @@ static int mpam_msc_hw_probe(struct mpam_msc *msc) msc->pmg_max =min (msc->pmg_max, pmg_max); msc->has_extd_esr =FIELD_GET (MPAMF_IDR_HAS_EXTD_ESR , idr); -mutex_lock (&mpam_list_lock); - ris =mpam_get_or_create_ris (msc, ris_idx); -mutex_unlock (&mpam_list_lock); -if (IS_ERR (ris)) -return PTR_ERR (ris); + scoped_guard(mutex, &mpam_list_lock) { + ris =mpam_get_or_create_ris (msc, ris_idx); +if (IS_ERR (ris)) +return PTR_ERR (ris); + } ris->idr = idr; -mutex_lock (&msc->part_sel_lock); -__mpam_part_sel (ris_idx,0 , msc); -mpam_ris_hw_probe (ris); -mutex_unlock (&msc->part_sel_lock); + scoped_guard(mutex, &msc->part_sel_lock) { + ret =__mpam_part_sel (ris_idx,0 , msc); +if (ret) +return ret; + + ret =mpam_ris_hw_probe (ris); +if (ret) +return ret; + } }/* Clear any stale errors */
drivers/resctrl/mpam_internal.h +8 -0
@@ -162,6 +162,14 @@ static inline void mpam_mon_sel_lock_init(struct mpam_msc *msc)raw_spin_lock_init (&msc->_mon_sel_lock); } +DEFINE_GUARD (mon_sel, + struct mpam_msc *, +mpam_mon_sel_lock (_T), +mpam_mon_sel_unlock (_T)); +DEFINE_GUARD_COND (mon_sel, _lock, +mpam_mon_sel_lock (_T), + _RET); +/* Bits for mpam features bitmaps */ enum mpam_device_features { mpam_feat_cpor_part,
Signature · 1 line
2.43.0
Re: [PATCH v4 02/10] arm_mpam: propagate MSC access errors for hw_probe functions
Quoted 8 lines · On Thu, 23 Jul 2026 17:54:46 +0200 Andre Przywara <[email protected]>…
Allow the functions probing for MSC hardware and features to return an error, and propagate read and write errors from the lower level up. This uses some "scoped cleanup" functions like
scoped_guard()andACQUIRE()to avoid the complexity of error handling when a lock has been taken. Since themon_sel_lockis a bit special (even more so in an upcoming patch), we define a new GUARD type for it.Signed-off-by Andre Przywara <[email protected]>
Hi Andre
A couple of whites space comments inline.
Quoted 20 lines
--- drivers/resctrl/mpam_devices.c | 127 +++++++++++++++++++++----------- drivers/resctrl/mpam_internal.h | 8 ++ 2 files changed, 91 insertions(+), 44 deletions(-)drivers/resctrl/mpam_devices.c +1
-0static int mpam_msc_hw_probe (struct mpam_msc *msc) { +int ret;u64 idr;u16 partid_max;u8 ris_idx, pmg_max;@@ -1021,11 +1047,14 @@ static int mpam_msc_hw_probe(struct mpam_msc *msc) }
drivers/resctrl/mpam_devices.c static int mpam_msc_hw_probe(struct mpam_msc *msc) · line 1053
Quoted 10 lines
drivers/resctrl/mpam_devices.c +4
-5/* Grab an IDR value to find out how many RIS there are */ -mutex_lock (&msc->part_sel_lock); -mpam_msc_read_idr (msc, &idr); -mpam_read_partsel_reg (msc,IIDR , &msc->iidr); - -mutex_unlock (&msc->part_sel_lock); + scoped_guard(mutex, &msc->part_sel_lock) { + ret =mpam_msc_read_idr (msc, &idr); +if (ret) +return ret;
Aim for formatting consistency. I'd put a blank line here and in similar locations to keep each read / error check block separate...
Quoted 10 lines
drivers/resctrl/mpam_devices.c +4
-0+ ret =mpam_read_partsel_reg (msc,IIDR , &msc->iidr); +if (ret) +return ret; + }mpam_enable_quirks (msc);@@ -1036,10 +1065,15 @@ static int mpam_msc_hw_probe(struct mpam_msc *msc) msc->pmg_max =FIELD_GET (MPAMF_IDR_PMG_MAX , idr);
drivers/resctrl/mpam_devices.c static int mpam_msc_hw_probe(struct mpam_msc *msc) · line 1072
Quoted 10 lines
drivers/resctrl/mpam_devices.c +5
-4for (ris_idx =0 ; ris_idx <= msc->ris_max; ris_idx++) { -mutex_lock (&msc->part_sel_lock); -__mpam_part_sel (ris_idx,0 , msc); -mpam_msc_read_idr (msc, &idr); -mutex_unlock (&msc->part_sel_lock); + scoped_guard(mutex, &msc->part_sel_lock) { + ret =__mpam_part_sel (ris_idx,0 , msc); +if (ret) +return ret; +
Just like you have done here.
Quoted 10 lines
drivers/resctrl/mpam_devices.c +4
-0+ ret =mpam_msc_read_idr (msc, &idr); +if (ret) +return ret; + }drivers/resctrl/mpam_internal.h +0
-0
drivers/resctrl/mpam_internal.h static inline void mpam_mon_sel_lock_init(struct mpam_msc *msc) · line 168
Quoted 7 lines
drivers/resctrl/mpam_internal.h +4
-0@@ -162,6 +162,14 @@ static inline void mpam_mon_sel_lock_init(struct mpam_msc *msc)raw_spin_lock_init (&msc->_mon_sel_lock); } +DEFINE_GUARD (mon_sel, + struct mpam_msc *, +mpam_mon_sel_lock (_T), +mpam_mon_sel_unlock (_T));
Why wrap so much. Something like:
DEFINE_GUARD(mon_sel, struct mpam_msc *,
mpam_mon_sel_lock(_T), mpam_mon_sel_unlock(_T));
or all on one line seems fine to me.
drivers/resctrl/mpam_internal.h static inline void mpam_mon_sel_lock_init(struct mpam_msc *msc) · line 172
Quoted 4 lines
drivers/resctrl/mpam_internal.h +4
-0+DEFINE_GUARD_COND (mon_sel, _lock, +mpam_mon_sel_lock (_T), + _RET); +
Similar.
Quoted 3 lines
drivers/resctrl/mpam_internal.h +0
-0/* Bits for mpam features bitmaps */ enum mpam_device_features { mpam_feat_cpor_part,
PATCH v4 3/10 arm_mpam: propagate MSC access errors for MBWU counters
In reply to Andre PrzywaraAllow the mpam_msc_read_mbwu_l() function to return an error, and propagate errors from the lower level up. This also changes the users of this function.
--- drivers/resctrl/mpam_devices.c | 45 +++++++++++++++++++++++----------- 1 file changed, 31 insertions(+), 14 deletions(-)
drivers/resctrl/mpam_devices.c +31 -14
@@ -1127,8 +1127,9 @@ static bool mpam_ris_has_mbwu_long_counter(struct mpam_msc_ris *ris) mpam_has_feature(mpam_feat_msmon_mbwu_44counter, &ris->props)); } -static u64 mpam_msc_read_mbwu_l (struct mpam_msc *msc) +static int mpam_msc_read_mbwu_l (struct mpam_msc *msc,u64 *res) { +int ret;int retry =3 ;u32 mbwu_l_low;u32 mbwu_l_high1, mbwu_l_high2;
@@ -1138,20 +1139,30 @@ static u64 mpam_msc_read_mbwu_l(struct mpam_msc *msc)WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); -__mpam_read_reg (msc,MSMON_MBWU_L +4 , &mbwu_l_high2); + ret =__mpam_read_reg (msc,MSMON_MBWU_L +4 , &mbwu_l_high2); +if (ret) +return ret; +do { mbwu_l_high1 = mbwu_l_high2; -__mpam_read_reg (msc,MSMON_MBWU_L , &mbwu_l_low); -__mpam_read_reg (msc,MSMON_MBWU_L +4 , &mbwu_l_high2); + ret =__mpam_read_reg (msc,MSMON_MBWU_L , &mbwu_l_low); +if (ret) +return ret; + ret =__mpam_read_reg (msc,MSMON_MBWU_L +4 , &mbwu_l_high2); +if (ret) +return ret; retry--; }while (mbwu_l_high1 != mbwu_l_high2 && retry >0 ); -if (mbwu_l_high1 == mbwu_l_high2) -return ((u64 )mbwu_l_high1 <<32 ) | mbwu_l_low; +if (mbwu_l_high1 == mbwu_l_high2) { + *res = ((u64 )mbwu_l_high1 <<32 ) | mbwu_l_low; + }else { +pr_warn ("Failed to read a stable value\n" ); + *res =MSMON___L_NRDY ; + } -pr_warn ("Failed to read a stable value\n" ); -return MSMON___L_NRDY ; +return 0 ; }static void mpam_msc_zero_mbwu_l (struct mpam_msc *msc)
@@ -1310,10 +1321,12 @@ static void __ris_msmon_read(void *arg)struct mpam_msc *msc = m->ris->vmsc->msc;u32 mon_sel, ctl_val, flt_val, cur_ctl, cur_flt; -if (!mpam_mon_sel_lock (msc)) { +ACQUIRE (mon_sel_lock, guard)(msc); +if (ACQUIRE_ERR (mon_sel_lock, &guard)) { m->err = -EIO ;return ; } + mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , ctx->mon) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx);mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel);
@@ -1373,7 +1386,9 @@ static void __ris_msmon_read(void *arg)case mpam_feat_msmon_mbwu_44counter:case mpam_feat_msmon_mbwu_63counter:if (m->type != mpam_feat_msmon_mbwu_31counter) { - now =mpam_msc_read_mbwu_l (msc); + m->err =mpam_msc_read_mbwu_l (msc, &now); +if (m->err) +return ; nrdy = now &MSMON___L_NRDY ;if (m->type == mpam_feat_msmon_mbwu_63counter)
@@ -1407,7 +1422,6 @@ static void __ris_msmon_read(void *arg)default : m->err = -EINVAL ; } -mpam_mon_sel_unlock (msc);if (nrdy) m->err = -EBUSY ;
@@ -1762,6 +1776,7 @@ static int mpam_save_mbwu_state(void *arg) {int i;u64 val; +int ret;struct mon_cfg *cfg;u32 cur_flt, cur_ctl, mon_sel;struct mpam_msc_ris *ris = arg;
@@ -1772,7 +1787,8 @@ static int mpam_save_mbwu_state(void *arg) mbwu_state = &ris->mbwu_state[i]; cfg = &mbwu_state->cfg; -if (WARN_ON_ONCE (!mpam_mon_sel_lock (msc))) +ACQUIRE (mon_sel_lock, guard)(msc); +if (ACQUIRE_ERR (mon_sel_lock, &guard))return -EIO ; mon_sel = FIELD_PREP(MSMON_CFG_MON_SEL_MON_SEL, i) |
@@ -1783,7 +1799,9 @@ static int mpam_save_mbwu_state(void *arg)mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 );if (mpam_ris_has_mbwu_long_counter (ris)) { - val =mpam_msc_read_mbwu_l (msc); + ret =mpam_msc_read_mbwu_l (msc, &val); +if (ret) +return ret;mpam_msc_zero_mbwu_l (msc); }else {u32 val32;
@@ -1799,7 +1817,6 @@ static int mpam_save_mbwu_state(void *arg) cfg->partid =FIELD_GET (MSMON_CFG_x_FLT_PARTID, cur_flt); mbwu_state->correction += val; mbwu_state->enabled =FIELD_GET (MSMON_CFG_x_CTL_EN, cur_ctl); -mpam_mon_sel_unlock (msc); }return 0 ;
Signature · 1 line
2.43.0
Re: [PATCH v4 03/10] arm_mpam: propagate MSC access errors for MBWU counters
Quoted 5 lines · On Thu, 23 Jul 2026 17:54:47 +0200 Andre Przywara <[email protected]>…
Allow the
mpam_msc_read_mbwu_l()function to return an error, and propagate errors from the lower level up. This also changes the users of this function.Signed-off-by Andre Przywara <[email protected]>
PATCH v4 4/10 arm_mpam: propagate MSC access errors for msmon helpers
In reply to Andre PrzywaraAllow the helper functions for msmon accesses to return an error, and propagate read and write errors from the lower level up.
--- drivers/resctrl/mpam_devices.c | 72 ++++++++++++++++++++++------------ 1 file changed, 47 insertions(+), 25 deletions(-)
drivers/resctrl/mpam_devices.c +47 -25
@@ -1165,15 +1165,20 @@ static int mpam_msc_read_mbwu_l(struct mpam_msc *msc, u64 *res)return 0 ; } -static void mpam_msc_zero_mbwu_l (struct mpam_msc *msc) +static int mpam_msc_zero_mbwu_l (struct mpam_msc *msc) { +int ret; +mpam_mon_sel_lock_held (msc);WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); -__mpam_write_reg (msc,MSMON_MBWU_L ,0 ); -__mpam_write_reg (msc,MSMON_MBWU_L +4 ,0 ); + ret =__mpam_write_reg (msc,MSMON_MBWU_L ,0 ); +if (ret) +return ret; + +return __mpam_write_reg (msc,MSMON_MBWU_L +4 ,0 ); } static void gen_msmon_ctl_flt_vals(struct mon_read *m, u32 *ctl_val,
@@ -1216,24 +1221,28 @@ static void gen_msmon_ctl_flt_vals(struct mon_read *m, u32 *ctl_val, } } -static void read_msmon_ctl_flt_vals (struct mon_read *m,u32 *ctl_val, -u32 *flt_val) +static int read_msmon_ctl_flt_vals (struct mon_read *m,u32 *ctl_val, +u32 *flt_val) {struct mpam_msc *msc = m->ris->vmsc->msc; +int ret;switch (m->type) {case mpam_feat_msmon_csu: -mpam_read_monsel_reg (msc,CFG_CSU_CTL , ctl_val); -mpam_read_monsel_reg (msc,CFG_CSU_FLT , flt_val); -break ; + ret =mpam_read_monsel_reg (msc,CFG_CSU_CTL , ctl_val); +if (ret) +return ret; +return mpam_read_monsel_reg (msc,CFG_CSU_FLT , flt_val);case mpam_feat_msmon_mbwu_31counter:case mpam_feat_msmon_mbwu_44counter:case mpam_feat_msmon_mbwu_63counter: -mpam_read_monsel_reg (msc,CFG_MBWU_CTL , ctl_val); -mpam_read_monsel_reg (msc,CFG_MBWU_FLT , flt_val); -break ; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_CTL , ctl_val); +if (ret) +return ret; +return mpam_read_monsel_reg (msc,CFG_MBWU_FLT , flt_val);default :pr_warn ("Unexpected monitor type %d\n" , m->type); +return -EINVAL ; } }
@@ -1246,10 +1255,11 @@ static inline void clean_msmon_ctl_val(u32 *cur_ctl) *cur_ctl &= ~MSMON_CFG_MBWU_CTL_OFLOW_STATUS_L ; } -static void write_msmon_ctl_flt_vals (struct mon_read *m,u32 ctl_val, -u32 flt_val) +static int write_msmon_ctl_flt_vals (struct mon_read *m,u32 ctl_val, +u32 flt_val) {struct mpam_msc *msc = m->ris->vmsc->msc; +int ret;/* * Write the ctl_val with the enable bit cleared, reset the counter,
@@ -1257,25 +1267,37 @@ static void write_msmon_ctl_flt_vals(struct mon_read *m, u32 ctl_val,*/ switch (m->type) {case mpam_feat_msmon_csu: -mpam_write_monsel_reg (msc,CFG_CSU_FLT , flt_val); -mpam_write_monsel_reg (msc,CFG_CSU_CTL , ctl_val); -mpam_write_monsel_reg (msc,CSU ,0 ); -mpam_write_monsel_reg (msc,CFG_CSU_CTL , ctl_val | MSMON_CFG_x_CTL_EN); -break ; + ret =mpam_write_monsel_reg (msc,CFG_CSU_FLT , flt_val); +if (ret) +return ret; + ret =mpam_write_monsel_reg (msc,CFG_CSU_CTL , ctl_val); +if (ret) +return ret; + ret =mpam_write_monsel_reg (msc,CSU ,0 ); +if (ret) +return ret; +return mpam_write_monsel_reg (msc,CFG_CSU_CTL , ctl_val | MSMON_CFG_x_CTL_EN);case mpam_feat_msmon_mbwu_31counter:case mpam_feat_msmon_mbwu_44counter:case mpam_feat_msmon_mbwu_63counter: -mpam_write_monsel_reg (msc,CFG_MBWU_FLT , flt_val); -mpam_write_monsel_reg (msc,CFG_MBWU_CTL , ctl_val); -mpam_write_monsel_reg (msc,CFG_MBWU_CTL , ctl_val | MSMON_CFG_x_CTL_EN); + ret =mpam_write_monsel_reg (msc,CFG_MBWU_FLT , flt_val); +if (ret) +return ret; + ret =mpam_write_monsel_reg (msc,CFG_MBWU_CTL , ctl_val); +if (ret) +return ret; + ret =mpam_write_monsel_reg (msc,CFG_MBWU_CTL , + ctl_val | MSMON_CFG_x_CTL_EN); +if (ret) +return ret;/* Counting monitors require NRDY to be reset by software */ if (m->type == mpam_feat_msmon_mbwu_31counter) -mpam_write_monsel_reg (msc,MBWU ,0 ); -else -mpam_msc_zero_mbwu_l (m->ris->vmsc->msc); -break ; +return mpam_write_monsel_reg (msc,MBWU ,0 ); + +return mpam_msc_zero_mbwu_l (m->ris->vmsc->msc);default :pr_warn ("Unexpected monitor type %d\n" , m->type); +return -EINVAL ; } }
Signature · 1 line
2.43.0
Re: [PATCH v4 04/10] arm_mpam: propagate MSC access errors for msmon helpers
Quoted 4 lines · On Thu, 23 Jul 2026 17:54:48 +0200 Andre Przywara <[email protected]>…
Allow the helper functions for msmon accesses to return an error, and propagate read and write errors from the lower level up.
Signed-off-by Andre Przywara <[email protected]>
I'd take another look at this one and consider if a few more blank lines would help with readability. Otherwise LGTM
PATCH v4 5/10 arm_mpam: propagate MSC access errors for __ris_msmon_read()
In reply to Andre PrzywaraAllow the function for RIS accesses to return an error, and propagate read and write errors from the lower level up.
This function is a bit special, since it can be called via an SMP helper, so the prototype must stay as void. So propagate any errors using the "err" field in the struct passed in as the sole argument to that function.
--- drivers/resctrl/mpam_devices.c | 36 +++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)
drivers/resctrl/mpam_devices.c +27 -9
@@ -1328,6 +1328,10 @@ static u64 mpam_msmon_overflow_val(enum mpam_device_features type,return overflow_val; } +/* +* This function might be called via smp_call_function_any(), so propagate +* errors inside the arg struct. +*/ static void __ris_msmon_read (void *arg) {u64 now;
@@ -1351,7 +1355,9 @@ static void __ris_msmon_read(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , ctx->mon) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); + m->err =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (m->err) +return ;switch (m->type) {case mpam_feat_msmon_mbwu_31counter:
@@ -1371,7 +1377,9 @@ static void __ris_msmon_read(void *arg)* Read the existing configuration to avoid re-writing the same values. * This saves waiting for 'nrdy' on subsequent reads. */ -read_msmon_ctl_flt_vals (m, &cur_ctl, &cur_flt); + m->err =read_msmon_ctl_flt_vals (m, &cur_ctl, &cur_flt); +if (m->err) +return ;if (mpam_feat_msmon_mbwu_31counter == m->type) overflow = cur_ctl & MSMON_CFG_x_CTL_OFLOW_STATUS;
@@ -1385,18 +1393,24 @@ static void __ris_msmon_read(void *arg) cur_ctl != (ctl_val | MSMON_CFG_x_CTL_EN);if (config_mismatch || reset_on_next_read) { -write_msmon_ctl_flt_vals (m, ctl_val, flt_val); + m->err =write_msmon_ctl_flt_vals (m, ctl_val, flt_val); +if (m->err) +return ; overflow = false; }else if (overflow) { -mpam_write_monsel_reg (msc,CFG_MBWU_CTL , - cur_ctl & - ~(MSMON_CFG_x_CTL_OFLOW_STATUS | -MSMON_CFG_MBWU_CTL_OFLOW_STATUS_L )); + m->err =mpam_write_monsel_reg (msc,CFG_MBWU_CTL , + cur_ctl & + ~(MSMON_CFG_x_CTL_OFLOW_STATUS | +MSMON_CFG_MBWU_CTL_OFLOW_STATUS_L )); +if (m->err) +return ; }switch (m->type) {case mpam_feat_msmon_csu: -mpam_read_monsel_reg (msc,CSU , &now32); + m->err =mpam_read_monsel_reg (msc,CSU , &now32); +if (m->err) +return ; nrdy = now32 &MSMON___NRDY ; now =FIELD_GET (MSMON___VALUE , now32);
@@ -1418,7 +1432,9 @@ static void __ris_msmon_read(void *arg)else now =FIELD_GET (MSMON___L_VALUE , now); }else { -mpam_read_monsel_reg (msc,MBWU , &now32); + m->err =mpam_read_monsel_reg (msc,MBWU , &now32); +if (m->err) +return ; nrdy = now32 &MSMON___NRDY ; now =FIELD_GET (MSMON___VALUE , now32); }
@@ -1787,6 +1803,8 @@ static int mpam_restore_mbwu_state(void *_ris) mwbu_arg.val = &val;__ris_msmon_read (&mwbu_arg); +if (mwbu_arg.err) +return mwbu_arg.err; } }
Signature · 1 line
2.43.0
Sudeep Holla
maintainerRe: [PATCH v4 05/10] arm_mpam: propagate MSC access errors for __ris_msmon_read()
Quoted 7 lines · Andre Przywara
Allow the function for RIS accesses to return an error, and propagate read and write errors from the lower level up.
This function is a bit special, since it can be called via an SMP helper, so the prototype must stay as void. So propagate any errors using the "err" field in the struct passed in as the sole argument to that function.
Not sure if this was already discussed by build bisection regresses between 4/10 and 5/10 as you change function prototype there and caller here or am I missing to read it correctly ?
Quoted 32 lines
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 36 +++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)drivers/resctrl/mpam_devices.c +7
-1@@ -1328,6 +1328,10 @@ static u64 mpam_msmon_overflow_val(enum mpam_device_features type,return overflow_val; } +/* +* This function might be called via smp_call_function_any(), so propagate +* errors inside the arg struct. +*/ static void __ris_msmon_read (void *arg) {u64 now;@@ -1351,7 +1355,9 @@ static void __ris_msmon_read(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , ctx->mon) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); + m->err =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (m->err) +return ;switch (m->type) {case mpam_feat_msmon_mbwu_31counter:
drivers/resctrl/mpam_devices.c static void __ris_msmon_read(void *arg) · line 1383
Quoted 8 lines
drivers/resctrl/mpam_devices.c +3
-1@@ -1371,7 +1377,9 @@ static void __ris_msmon_read(void *arg)* Read the existing configuration to avoid re-writing the same values. * This saves waiting for 'nrdy' on subsequent reads. */ -read_msmon_ctl_flt_vals (m, &cur_ctl, &cur_flt); + m->err =read_msmon_ctl_flt_vals (m, &cur_ctl, &cur_flt); +if (m->err) +return ;
Atleast read_msmon_ctl_flt_vals() changed to return value in last patch.
Signature · 2 lines
Regards, Sudeep
Re: [PATCH v4 05/10] arm_mpam: propagate MSC access errors for __ris_msmon_read()
Hi,
Quoted 9 lines · Sudeep Holla
Quoted 7 lines · Andre Przywara
Allow the function for RIS accesses to return an error, and propagate read and write errors from the lower level up.
This function is a bit special, since it can be called via an SMP helper, so the prototype must stay as void. So propagate any errors using the "err" field in the struct passed in as the sole argument to that function.
Quoted 3 lines
Not sure if this was already discussed by build bisection regresses between 4/10 and 5/10 as you change function prototype there and caller here or am I missing to read it correctly ?
Does it? I build tested every patch multiple times yesterday, and just repeated it, and it comes back cleanly.
Quoted 29 lines
Quoted 29 lines
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 36 +++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)drivers/resctrl/mpam_devices.c +7
-1@@ -1328,6 +1328,10 @@ static u64 mpam_msmon_overflow_val(enum mpam_device_features type,return overflow_val; } +/* +* This function might be called via smp_call_function_any(), so propagate +* errors inside the arg struct. +*/ static void __ris_msmon_read (void *arg) {u64 now;@@ -1351,7 +1355,9 @@ static void __ris_msmon_read(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , ctx->mon) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); + m->err =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (m->err) +return ;
Quoted 13 lines
drivers/resctrl/mpam_devices.c static void __ris_msmon_read(void *arg) · line 1383
Quoted 11 lines
drivers/resctrl/mpam_devices.c +3
-1switch (m->type) {case mpam_feat_msmon_mbwu_31counter:@@ -1371,7 +1377,9 @@ static void __ris_msmon_read(void *arg)* Read the existing configuration to avoid re-writing the same values. * This saves waiting for 'nrdy' on subsequent reads. */ -read_msmon_ctl_flt_vals (m, &cur_ctl, &cur_flt); + m->err =read_msmon_ctl_flt_vals (m, &cur_ctl, &cur_flt); +if (m->err) +return ;Atleast
read_msmon_ctl_flt_vals()changed to return value in last patch.
Ah, but this is not a problem that breaks the build, as we formerly just ignored the (at this point bogus) error value, but now pass it up.
But this is one occasion of "touching the same line twice", that was criticised before. Not nice, but the alternative would be to have one giant patch, which I figured to be unreadable.
Cheers,
Andre
Sudeep Holla
maintainerRe: [PATCH v4 05/10] arm_mpam: propagate MSC access errors for __ris_msmon_read()
Quoted 59 lines · Andre Przywara
Hi,
Quoted 9 lines · Sudeep Holla
Quoted 7 lines · Andre Przywara
Allow the function for RIS accesses to return an error, and propagate read and write errors from the lower level up.
This function is a bit special, since it can be called via an SMP helper, so the prototype must stay as void. So propagate any errors using the "err" field in the struct passed in as the sole argument to that function.Quoted 3 lines
Not sure if this was already discussed by build bisection regresses between 4/10 and 5/10 as you change function prototype there and caller here or am I missing to read it correctly ?
Does it? I build tested every patch multiple times yesterday, and just repeated it, and it comes back cleanly.
Quoted 31 lines
Quoted 31 lines
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 36 +++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)drivers/resctrl/mpam_devices.c +7
-1@@ -1328,6 +1328,10 @@ static u64 mpam_msmon_overflow_val(enum mpam_device_features type,return overflow_val; } +/* +* This function might be called via smp_call_function_any(), so propagate +* errors inside the arg struct. +*/ static void __ris_msmon_read (void *arg) {u64 now;@@ -1351,7 +1355,9 @@ static void __ris_msmon_read(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , ctx->mon) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); + m->err =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (m->err) +return ;switch (m->type) {case mpam_feat_msmon_mbwu_31counter:@@ -1371,7 +1377,9 @@ static void __ris_msmon_read(void *arg)* Read the existing configuration to avoid re-writing the same values. * This saves waiting for 'nrdy' on subsequent reads. Quoted 8 lines
drivers/resctrl/mpam_devices.c static void __ris_msmon_read(void *arg) · line 1383
Quoted 6 lines
drivers/resctrl/mpam_devices.c +3
-1*/ -read_msmon_ctl_flt_vals (m, &cur_ctl, &cur_flt); + m->err =read_msmon_ctl_flt_vals (m, &cur_ctl, &cur_flt); +if (m->err) +return ;Atleast
read_msmon_ctl_flt_vals()changed to return value in last patch.
Quoted 2 lines
Ah, but this is not a problem that breaks the build, as we formerly just ignored the (at this point bogus) error value, but now pass it up.
Rather than return I was thinking about added parameter to get the read value.
Quoted 4 lines
But this is one occasion of "touching the same line twice", that was criticised before. Not nice, but the alternative would be to have one giant patch, which I figured to be unreadable.
Ah then good. May be I was overthinking.
Signature · 2 lines
Regards, Sudeep
Re: [PATCH v4 05/10] arm_mpam: propagate MSC access errors for __ris_msmon_read()
In reply to Andre PrzywaraQuoted 8 lines · On Thu, 23 Jul 2026 17:54:49 +0200 Andre Przywara <[email protected]>…
Allow the function for RIS accesses to return an error, and propagate read and write errors from the lower level up.
This function is a bit special, since it can be called via an SMP helper, so the prototype must stay as void. So propagate any errors using the "err" field in the struct passed in as the sole argument to that function.Signed-off-by Andre Przywara <[email protected]>
PATCH v4 6/10 arm_mpam: propagate MSC access errors for state saving function
In reply to Andre PrzywaraAllow the mpam_save_mbwu_state() function to return an error, and propagate read and write errors from the lower level up.
--- drivers/resctrl/mpam_devices.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)
drivers/resctrl/mpam_devices.c +22 -7
@@ -1833,22 +1833,37 @@ static int mpam_save_mbwu_state(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , i) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); -mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); -mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); -mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 ); + ret =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (ret) +return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); +if (ret) +return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); +if (ret) +return ret; + ret =mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 ); +if (ret) +return ret;if (mpam_ris_has_mbwu_long_counter (ris)) { ret =mpam_msc_read_mbwu_l (msc, &val);if (ret)return ret; -mpam_msc_zero_mbwu_l (msc); + ret =mpam_msc_zero_mbwu_l (msc); +if (ret) +return ret; }else {u32 val32; -mpam_read_monsel_reg (msc,MBWU , &val32); + ret =mpam_read_monsel_reg (msc,MBWU , &val32); +if (ret) +return ret; + val = val32; -mpam_write_monsel_reg (msc,MBWU ,0 ); + ret =mpam_write_monsel_reg (msc,MBWU ,0 ); +if (ret) +return ret; } cfg->mon = i;
Signature · 1 line
2.43.0
Sudeep Holla
maintainerRe: [PATCH v4 06/10] arm_mpam: propagate MSC access errors for state saving function
Quoted 19 lines · Andre Przywara
Allow the
mpam_save_mbwu_state()function to return an error, and propagate read and write errors from the lower level up.Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)drivers/resctrl/mpam_devices.c +0
-3@@ -1833,22 +1833,37 @@ static int mpam_save_mbwu_state(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , i) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); -mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); -mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl);
drivers/resctrl/mpam_devices.c static int mpam_save_mbwu_state(void *arg) · line 1844
Quoted 10 lines
drivers/resctrl/mpam_devices.c +9
-1-mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 ); + ret =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (ret) +return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); +if (ret) +return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); +if (ret) +return ret;
How does it work in general with PCC. Now that you can fail at any point, what happens to the write that occurs before a failed read like above one. Who will take care of erasing those new writes or it doesn't matter ? Just checking as I don't have much knowledge on MPAM intrinsics.
Signature · 2 lines
Regards, Sudeep
Re: [PATCH v4 06/10] arm_mpam: propagate MSC access errors for state saving function
Hi,
Quoted 27 lines · Sudeep Holla
Quoted 26 lines · Andre Przywara
Allow the
mpam_save_mbwu_state()function to return an error, and propagate read and write errors from the lower level up.Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)drivers/resctrl/mpam_devices.c +6
-4@@ -1833,22 +1833,37 @@ static int mpam_save_mbwu_state(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , i) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); -mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); -mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); -mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 ); + ret =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (ret) +return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); +if (ret) +return ret;
Quoted 8 lines
drivers/resctrl/mpam_devices.c static int mpam_save_mbwu_state(void *arg) · line 1844
Quoted 3 lines
drivers/resctrl/mpam_devices.c +3
-0+ ret =mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); +if (ret) +return ret;How does it work in general with PCC. Now that you can fail at any point, what happens to the write that occurs before a failed read like above one. Who will take care of erasing those new writes or it doesn't matter ? Just checking as I don't have much knowledge on MPAM intrinsics.
TBH I don't know, but I think we consider MPAM botched at this point, and just stop the driver, similar to an error IRQ? But I am not sure this is properly implemented at this point. The focus of these first six patches was merely to lay the dirty groundwork for *being able* to handle errors, and do this now rather than in the future.
Cheers,
Andre
Sudeep Holla
maintainerRe: [PATCH v4 06/10] arm_mpam: propagate MSC access errors for state saving function
Quoted 38 lines · Andre Przywara
Hi,
Quoted 26 lines · Sudeep Holla
Quoted 25 lines · Andre Przywara
Allow the
mpam_save_mbwu_state()function to return an error, and propagate read and write errors from the lower level up.Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)drivers/resctrl/mpam_devices.c +6
-4@@ -1833,22 +1833,37 @@ static int mpam_save_mbwu_state(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , i) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); -mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); -mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); -mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 ); + ret =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (ret) +return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); +if (ret) +return ret;Quoted 8 lines
drivers/resctrl/mpam_devices.c static int mpam_save_mbwu_state(void *arg) · line 1844
Quoted 3 lines
drivers/resctrl/mpam_devices.c +3
-0+ ret =mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); +if (ret) +return ret;How does it work in general with PCC. Now that you can fail at any point, what happens to the write that occurs before a failed read like above one. Who will take care of erasing those new writes or it doesn't matter ? Just checking as I don't have much knowledge on MPAM intrinsics.
Quoted 6 lines
TBH I don't know, but I think we consider MPAM botched at this point, and just stop the driver, similar to an error IRQ? But I am not sure this is properly implemented at this point. The focus of these first six patches was merely to lay the dirty groundwork for *being able* to handle errors, and do this now rather than in the future.
Well, I agree to some extent, but PCC adds that failure case before which it wasn't there. So, it is hard to claim that it was botched up before so let it be. I will let James/Ben to decide if it was already botched up or PCC addition makes it fragile in terms of error handling.
Signature · 2 lines
Regards, Sudeep
Ben Horgan
maintainerRe: [PATCH v4 06/10] arm_mpam: propagate MSC access errors for state saving function
Hi Sudeep, Andre,
Quoted 29 lines · Sudeep Holla
Quoted 28 lines · Andre Przywara
Hi,
Quoted 25 lines · Sudeep Holla
Quoted 24 lines · Andre Przywara
Allow the
mpam_save_mbwu_state()function to return an error, and propagate read and write errors from the lower level up.Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)drivers/resctrl/mpam_devices.c +5
-4@@ -1833,22 +1833,37 @@ static int mpam_save_mbwu_state(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , i) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); -mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); -mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); -mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 ); + ret =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (ret) +return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); +if (ret)
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/resctrl/mpam_devices.c static int mpam_save_mbwu_state(void *arg) · line 1844
Quoted 4 lines
drivers/resctrl/mpam_devices.c +4
-0+return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); +if (ret) +return ret;How does it work in general with PCC. Now that you can fail at any point, what happens to the write that occurs before a failed read like above one. Who will take care of erasing those new writes or it doesn't matter ?
At least in this particular case it doesn't matter. The MON_SEL just configures which instance of the monitor we are reading or writing from and we'll just write MON_SEL again next time we want to interact with a monitor.
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Just checking as I don't have much knowledge on MPAM intrinsics.
TBH I don't know, but I think we consider MPAM botched at this point, and just stop the driver, similar to an error IRQ? But I am not sure this is properly implemented at this point. The focus of these first six patches was merely to lay the dirty groundwork for *being able* to handle errors, and do this now rather than in the future.
I've just been discussing the error handling with James and trying to work out what we can do without limiting our options going forward. Ideally, if there are transient errors we'd like to retry in the kernel if it's not taking "too long" and otherwise report the error back to user space. For catastrophic errors, such as when the scp doesn't reply at all or that no MPAM commands are going to work again we should disable MPAM. As resctrl does not anticipate errors willy nilly from arch code it ends up reporting the info/last_cmd_status as "ok" even in situations when the MSC accesses have failed. I notice this in at least rdtgroup_schemata_write(). Before we can report proper failure information from resctrl and update last_cmd_status based on the error status from the architecture code it doesn't make sense to report the errors to user space. We would likely want resctrl to understand -ETIMEDOUT as well. As such, this leaves us the option of just nuking MPAM on any error reported from the MPAM firmware interface. For now, we needn't do any retries and just error out on the first error. It should be possible to just schedule mpam_broken_work from mpam_fb.c on error.
Sorry for prompting the writing of these error handling patches but they will likely come in useful in the future.
Thanks,
Ben
Quoted 5 lines
Well, I agree to some extent, but PCC adds that failure case before which it wasn't there. So, it is hard to claim that it was botched up before so let it be. I will let James/Ben to decide if it was already botched up or PCC addition makes it fragile in terms of error handling.
Re: [PATCH v4 06/10] arm_mpam: propagate MSC access errors for state saving function
Quoted 54 lines · On Fri, 24 Jul 2026 17:43:12 +0100 Ben Horgan <[email protected]> wrote:
Hi Sudeep, Andre,
Quoted 29 lines · Sudeep Holla
Quoted 28 lines · Andre Przywara
Hi,
Quoted 25 lines · Sudeep Holla
Quoted 24 lines · Andre Przywara
Allow the
mpam_save_mbwu_state()function to return an error, and propagate read and write errors from the lower level up.Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)drivers/resctrl/mpam_devices.c +5
-4@@ -1833,22 +1833,37 @@ static int mpam_save_mbwu_state(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , i) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); -mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); -mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); -mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 ); + ret =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (ret) +return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); +if (ret)Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/resctrl/mpam_devices.c static int mpam_save_mbwu_state(void *arg) · line 1844
Quoted 4 lines
drivers/resctrl/mpam_devices.c +4
-0+return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); +if (ret) +return ret;How does it work in general with PCC. Now that you can fail at any point, what happens to the write that occurs before a failed read like above one. Who will take care of erasing those new writes or it doesn't matter ?
At least in this particular case it doesn't matter. The
MON_SELjust configures which instance of the monitor we are reading or writing from and we'll just writeMON_SELagain next time we want to interact with a monitor.Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Just checking as I don't have much knowledge on MPAM intrinsics.
TBH I don't know, but I think we consider MPAM botched at this point, and just stop the driver, similar to an error IRQ? But I am not sure this is properly implemented at this point. The focus of these first six patches was merely to lay the dirty groundwork for *being able* to handle errors, and do this now rather than in the future.
Quoted 3 lines
I've just been discussing the error handling with James and trying to work out what we can do without limiting our options going forward. Ideally, if there are transient errors we'd like to retry in the kernel if it's not taking "too long" and otherwise report the error back to user space.
Agreed a retry makes sense. Perhaps moan a fair bit via a ratelimited print though as it is a pretty bad sign if retries are normally needed.
Quoted 10 lines
For catastrophic errors, such as when the scp doesn't reply at all or that no MPAM commands are going to work again we should disable MPAM. As resctrl does not anticipate errors willy nilly from arch code it ends up reporting the info/last_cmd_status as "ok" even in situations when the MSC accesses have failed. I notice this in at least
rdtgroup_schemata_write(). Before we can report proper failure information from resctrl and updatelast_cmd_statusbased on the error status from the architecture code it doesn't make sense to report the errors to user space. We would likely want resctrl to understand-ETIMEDOUTas well. As such, this leaves us the option of just nuking MPAM on any error reported from the MPAM firmware interface. For now, we needn't do any retries and just error out on the first error. It should be possible to just schedulempam_broken_workfrommpam_fb.con error.
Hmm. Inelegant but I suppose that works...
Quoted 16 lines
Sorry for prompting the writing of these error handling patches but they will likely come in useful in the future.
Thanks,
Ben
Quoted 5 lines
Well, I agree to some extent, but PCC adds that failure case before which it wasn't there. So, it is hard to claim that it was botched up before so let it be. I will let James/Ben to decide if it was already botched up or PCC addition makes it fragile in terms of error handling.
Ben Horgan
maintainerRe: [PATCH v4 06/10] arm_mpam: propagate MSC access errors for state saving function
Hi,
Quoted 75 lines · Jonathan Cameron
Quoted 54 lines · On Fri, 24 Jul 2026 17:43:12 +0100 Ben Horgan <[email protected]> wrote:
Hi Sudeep, Andre,
Quoted 29 lines · Sudeep Holla
Quoted 28 lines · Andre Przywara
Hi,
Quoted 25 lines · Sudeep Holla
Quoted 24 lines · Andre Przywara
Allow the
mpam_save_mbwu_state()function to return an error, and propagate read and write errors from the lower level up.Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)drivers/resctrl/mpam_devices.c +5
-4@@ -1833,22 +1833,37 @@ static int mpam_save_mbwu_state(void *arg) mon_sel =FIELD_PREP (MSMON_CFG_MON_SEL_MON_SEL , i) |FIELD_PREP (MSMON_CFG_MON_SEL_RIS , ris->ris_idx); -mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); -mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); -mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); -mpam_write_monsel_reg (msc,CFG_MBWU_CTL ,0 ); + ret =mpam_write_monsel_reg (msc,CFG_MON_SEL , mon_sel); +if (ret) +return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_FLT , &cur_flt); +if (ret)Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/resctrl/mpam_devices.c static int mpam_save_mbwu_state(void *arg) · line 1844
Quoted 4 lines
drivers/resctrl/mpam_devices.c +4
-0+return ret; + ret =mpam_read_monsel_reg (msc,CFG_MBWU_CTL , &cur_ctl); +if (ret) +return ret;How does it work in general with PCC. Now that you can fail at any point, what happens to the write that occurs before a failed read like above one. Who will take care of erasing those new writes or it doesn't matter ?
At least in this particular case it doesn't matter. The
MON_SELjust configures which instance of the monitor we are reading or writing from and we'll just writeMON_SELagain next time we want to interact with a monitor.Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Just checking as I don't have much knowledge on MPAM intrinsics.
TBH I don't know, but I think we consider MPAM botched at this point, and just stop the driver, similar to an error IRQ? But I am not sure this is properly implemented at this point. The focus of these first six patches was merely to lay the dirty groundwork for *being able* to handle errors, and do this now rather than in the future.
Quoted 3 lines
I've just been discussing the error handling with James and trying to work out what we can do without limiting our options going forward. Ideally, if there are transient errors we'd like to retry in the kernel if it's not taking "too long" and otherwise report the error back to user space.
Agreed a retry makes sense. Perhaps moan a fair bit via a ratelimited print though as it is a pretty bad sign if retries are normally needed.
Quoted 10 lines
For catastrophic errors, such as when the scp doesn't reply at all or that no MPAM commands are going to work again we should disable MPAM. As resctrl does not anticipate errors willy nilly from arch code it ends up reporting the info/last_cmd_status as "ok" even in situations when the MSC accesses have failed. I notice this in at least
rdtgroup_schemata_write(). Before we can report proper failure information from resctrl and updatelast_cmd_statusbased on the error status from the architecture code it doesn't make sense to report the errors to user space. We would likely want resctrl to understand-ETIMEDOUTas well. As such, this leaves us the option of just nuking MPAM on any error reported from the MPAM firmware interface. For now, we needn't do any retries and just error out on the first error. It should be possible to just schedulempam_broken_workfrommpam_fb.con error.
Quoted 5 lines
Hmm. Inelegant but I suppose that works...
Quoted 3 lines
Sorry for prompting the writing of these error handling patches but they will likely come in useful in the future.
Just to roll back on that a little bit. We will need the error handling patches even if we are scheduling mpam_broken. This is so that we can also return the error code up to resctrl. As calling mpam_broken will mean last_cmd_status is no longer present there is no file to read incorrect status from. (Andre, this is just to summarise what we discussed off list.)
Thanks,
Ben
Quoted 14 lines
Quoted 13 lines
Thanks,
Ben
Quoted 5 lines
Well, I agree to some extent, but PCC adds that failure case before which it wasn't there. So, it is hard to claim that it was botched up before so let it be. I will let James/Ben to decide if it was already botched up or PCC addition makes it fragile in terms of error handling.
Re: [PATCH v4 06/10] arm_mpam: propagate MSC access errors for state saving function
In reply to Andre PrzywaraQuoted 4 lines · On Thu, 23 Jul 2026 17:54:50 +0200 Andre Przywara <[email protected]>…
Allow the
mpam_save_mbwu_state()function to return an error, and propagate read and write errors from the lower level up.Signed-off-by Andre Przywara <[email protected]>
PATCH v4 7/10 arm_mpam: prepare mon_sel locking for MPAM-Fb
In reply to Andre PrzywaraThe MSC MON_SEL register needs to be accessed from hardirq for the overflow interrupt, and when taking an IPI to access these registers on platforms where MSCs are not accesible from every CPU. This makes an irqsave spinlock the obvious lock to protect these registers. On systems with MPAM-Fb mailbox MSC access it must be able to sleep, meaning a mutex must be used. So MPAM-Fb platforms cannot support an overflow interrupt easily. Clearly these two methods can't exist for one MSC at the same time.
Change the mon_sel locking wrapper function to only use a spinlock when the MSC is accessed directly via MMIO. In case of MPAM-Fb, we use a mutex, but only if we are in a sleepable context. If that's not the case, we return an error. This should not happen, as MPAM-Fb by design does not require an MSC access to happen from a specific CPU, so there is no need for any IPIs or preemption disabling to satisfy CPU constraints. And since overflow interrupts are not supported at the moment anyway, we also wouldn't meet the other case.
Bailing out early is already happening in rare occasions today.
--- drivers/resctrl/mpam_devices.c | 5 ++++- drivers/resctrl/mpam_internal.h | 32 ++++++++++++++++++++++++++------ 2 files changed, 30 insertions(+), 7 deletions(-)
drivers/resctrl/mpam_devices.c +4 -1
@@ -2225,7 +2225,10 @@ static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev)if (err)return ERR_PTR (err); -mpam_mon_sel_lock_init (msc); + err =mpam_mon_sel_lock_init (dev, msc); +if (err) +return ERR_PTR (err); + msc->id = pdev->id; msc->pdev = pdev;INIT_LIST_HEAD_RCU (&msc->all_msc_list);
drivers/resctrl/mpam_internal.h +26 -6
@@ -126,6 +126,7 @@ struct mpam_msc {*/ raw_spinlock_t _mon_sel_lock;unsigned long _mon_sel_flags; +struct mutex mon_sel_mutex;void __iomem *mapped_hwpage;size_t mapped_hwpage_sz;
@@ -139,27 +140,46 @@ struct mpam_msc {/* Returning false here means accesses to mon_sel must fail and report an error. */ static inline bool __must_check mpam_mon_sel_lock (struct mpam_msc *msc) { -/* Locking will require updating to support a firmware backed interface */ -if (WARN_ON_ONCE (msc->iface !=MPAM_IFACE_MMIO )) +if (msc->iface ==MPAM_IFACE_MMIO ) { +raw_spin_lock_irqsave (&msc->_mon_sel_lock, msc->_mon_sel_flags); + +return true; + } + +if (!preemptible ())return false; -raw_spin_lock_irqsave (&msc->_mon_sel_lock, msc->_mon_sel_flags); +mutex_lock (&msc->mon_sel_mutex); +return true; }static inline void mpam_mon_sel_unlock (struct mpam_msc *msc) { -raw_spin_unlock_irqrestore (&msc->_mon_sel_lock, msc->_mon_sel_flags); +if (msc->iface ==MPAM_IFACE_MMIO ) { +raw_spin_unlock_irqrestore (&msc->_mon_sel_lock, + msc->_mon_sel_flags); + +return ; + } + +mutex_unlock (&msc->mon_sel_mutex); }static inline void mpam_mon_sel_lock_held (struct mpam_msc *msc) { -lockdep_assert_held_once (&msc->_mon_sel_lock); +if (msc->iface ==MPAM_IFACE_MMIO ) +lockdep_assert_held_once (&msc->_mon_sel_lock); +else +lockdep_assert_held_once (&msc->mon_sel_mutex); } -static inline void mpam_mon_sel_lock_init (struct mpam_msc *msc) +static inline int mpam_mon_sel_lock_init (struct device *dev, +struct mpam_msc *msc) {raw_spin_lock_init (&msc->_mon_sel_lock); + +return devm_mutex_init (dev, &msc->mon_sel_mutex); } DEFINE_GUARD(mon_sel,
Signature · 1 line
2.43.0
Ben Horgan
maintainerRe: [PATCH v4 07/10] arm_mpam: prepare mon_sel locking for MPAM-Fb
Hi Andre,
Quoted 92 lines · Andre Przywara
The MSC
MON_SELregister needs to be accessed from hardirq for the overflow interrupt, and when taking an IPI to access these registers on platforms where MSCs are not accesible from every CPU. This makes an irqsave spinlock the obvious lock to protect these registers. On systems with MPAM-Fb mailbox MSC access it must be able to sleep, meaning a mutex must be used. So MPAM-Fb platforms cannot support an overflow interrupt easily. Clearly these two methods can't exist for one MSC at the same time.Change the
mon_sellocking wrapper function to only use a spinlock when the MSC is accessed directly via MMIO. In case of MPAM-Fb, we use a mutex, but only if we are in a sleepable context. If that's not the case, we return an error. This should not happen, as MPAM-Fb by design does not require an MSC access to happen from a specific CPU, so there is no need for any IPIs or preemption disabling to satisfy CPU constraints. And since overflow interrupts are not supported at the moment anyway, we also wouldn't meet the other case.
Bailing out early is already happening in rare occasions today.Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 5 ++++- drivers/resctrl/mpam_internal.h | 32 ++++++++++++++++++++++++++------ 2 files changed, 30 insertions(+), 7 deletions(-)drivers/resctrl/mpam_devices.c +4
-1@@ -2225,7 +2225,10 @@ static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev)if (err)return ERR_PTR (err); -mpam_mon_sel_lock_init (msc); + err =mpam_mon_sel_lock_init (dev, msc); +if (err) +return ERR_PTR (err); + msc->id = pdev->id; msc->pdev = pdev;INIT_LIST_HEAD_RCU (&msc->all_msc_list);drivers/resctrl/mpam_internal.h +21
-5@@ -126,6 +126,7 @@ struct mpam_msc {*/ raw_spinlock_t _mon_sel_lock;unsigned long _mon_sel_flags; +struct mutex mon_sel_mutex;void __iomem *mapped_hwpage;size_t mapped_hwpage_sz;@@ -139,27 +140,46 @@ struct mpam_msc {/* Returning false here means accesses to mon_sel must fail and report an error. */ static inline bool __must_check mpam_mon_sel_lock (struct mpam_msc *msc) { -/* Locking will require updating to support a firmware backed interface */ -if (WARN_ON_ONCE (msc->iface !=MPAM_IFACE_MMIO )) +if (msc->iface ==MPAM_IFACE_MMIO ) { +raw_spin_lock_irqsave (&msc->_mon_sel_lock, msc->_mon_sel_flags); + +return true; + } + +if (!preemptible ())return false; -raw_spin_lock_irqsave (&msc->_mon_sel_lock, msc->_mon_sel_flags); +mutex_lock (&msc->mon_sel_mutex); +return true; }static inline void mpam_mon_sel_unlock (struct mpam_msc *msc) { -raw_spin_unlock_irqrestore (&msc->_mon_sel_lock, msc->_mon_sel_flags); +if (msc->iface ==MPAM_IFACE_MMIO ) { +raw_spin_unlock_irqrestore (&msc->_mon_sel_lock, + msc->_mon_sel_flags); + +return ; + } + +mutex_unlock (&msc->mon_sel_mutex); }static inline void mpam_mon_sel_lock_held (struct mpam_msc *msc) { -lockdep_assert_held_once (&msc->_mon_sel_lock); +if (msc->iface ==MPAM_IFACE_MMIO ) +lockdep_assert_held_once (&msc->_mon_sel_lock); +else
drivers/resctrl/mpam_internal.h struct mpam_msc { · line 182
Quoted 10 lines
drivers/resctrl/mpam_internal.h +5
-1+lockdep_assert_held_once (&msc->mon_sel_mutex); } -static inline void mpam_mon_sel_lock_init (struct mpam_msc *msc) +static inline int mpam_mon_sel_lock_init (struct device *dev, +struct mpam_msc *msc) {raw_spin_lock_init (&msc->_mon_sel_lock); + +return devm_mutex_init (dev, &msc->mon_sel_mutex);
Any reason not to just init the lock that's being used?
Thanks,
Ben
Quoted 3 lines
drivers/resctrl/mpam_internal.h +0
-0} DEFINE_GUARD(mon_sel,
PATCH v4 8/10 arm_mpam: add MPAM-Fb MSC firmware access support
In reply to Andre PrzywaraThe Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
--- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 20 ++++ include/linux/arm_mpam.h | 2 +- 5 files changed, 240 insertions(+), 8 deletions(-) create mode 100644 drivers/resctrl/mpam_fb.c
drivers/resctrl/Makefile +1 -1
@@ -1,5 +1,5 @@ obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o -mpam-y += mpam_devices.o +mpam-y += mpam_devices.o mpam_fb.o mpam-$(CONFIG_ARM64_MPAM_RESCTRL_FS) += mpam_resctrl.o ccflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUG
drivers/resctrl/mpam_devices.c +21 -6
@@ -181,6 +181,9 @@ static int __mpam_read_reg(struct mpam_msc *msc, u16 reg, u32 *res) {WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); +if (msc->iface ==MPAM_IFACE_PCC ) +return mpam_fb_send_read_request (msc, reg, res); + *res =readl_relaxed (msc->mapped_hwpage + reg);return 0 ;
@@ -197,9 +200,12 @@ static inline int _mpam_read_partsel_reg(struct mpam_msc *msc, u16 reg,static int __mpam_write_reg (struct mpam_msc *msc,u16 reg,u32 val) { -WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); +if (msc->iface ==MPAM_IFACE_PCC ) +return mpam_fb_send_write_request (msc, reg, val); + +WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);writel_relaxed (val, msc->mapped_hwpage + reg);return 0 ;
@@ -1136,7 +1142,8 @@ static int mpam_msc_read_mbwu_l(struct mpam_msc *msc, u64 *res)mpam_mon_sel_lock_held (msc); -WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz); +if (msc->iface ==MPAM_IFACE_MMIO ) +WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); ret =__mpam_read_reg (msc,MSMON_MBWU_L +4 , &mbwu_l_high2);
@@ -1485,11 +1492,16 @@ static int _msmon_read(struct mpam_component *comp, struct mon_read *arg) srcu_read_lock_held(&mpam_srcu)) { arg->ris = ris; - err =smp_call_function_any (&msc->accessibility, - __ris_msmon_read, arg, - true); -if (!err && arg->err) +if (msc->iface ==MPAM_IFACE_MMIO ) { + err =smp_call_function_any (&msc->accessibility, + __ris_msmon_read, + arg, true); +if (!err) + err = arg->err; + }else { +__ris_msmon_read (arg); err = arg->err; + }/* * Save one error to be returned to the caller, but
@@ -1916,6 +1928,9 @@ static int mpam_get_msc_preferred_cpu(struct mpam_msc *msc)static int mpam_touch_msc (struct mpam_msc *msc,int (*fn)(void *a),void *arg) { +if (msc->iface !=MPAM_IFACE_MMIO ) +return fn (arg); +lockdep_assert_irqs_enabled ();lockdep_assert_cpus_held ();WARN_ON_ONCE (!srcu_read_lock_held ((&mpam_srcu)));
drivers/resctrl/mpam_fb.c new file +197 -0
197 lines not on this page · raw message
drivers/resctrl/mpam_internal.h +20 -0
@@ -11,6 +11,7 @@#include <linux/io.h> #include <linux/jump_label.h> #include <linux/llist.h> +#include <linux/mailbox_client.h> #include <linux/mutex.h> #include <linux/resctrl.h> #include <linux/spinlock.h>
@@ -57,6 +58,15 @@ struct mpam_garbage {struct platform_device *pdev; }; +struct mpam_pcc_chan { +struct list_head pcc_chans; +struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +struct mutex pcc_chan_lock;/* only one message at a time */ +struct kref refcount; +int subspace_id; +}; +struct mpam_msc {/* member of mpam_all_msc */ struct list_head all_msc_list;
@@ -66,6 +76,8 @@ struct mpam_msc {/* Not modified after mpam_is_enabled() becomes true */ enum mpam_msc_iface iface; +struct mpam_pcc_chan *pcc_chan; +int mpam_fb_msc_id;/* in its own name space */ u32 nrdy_usec;cpumask_t accessibility;bool has_extd_esr;
@@ -511,6 +523,14 @@ static inline void mpam_resctrl_offline_cpu(unsigned int cpu) { }static inline void mpam_resctrl_teardown_class (struct mpam_class *class) { }#endif /* CONFIG_RESCTRL_FS */ +/* MPAM-Fb Firmware-backed protocol wrappers */ +int mpam_fb_send_read_request (struct mpam_msc *msc,u16 reg,u32 *result); +int mpam_fb_send_write_request (struct mpam_msc *msc,u16 reg,u32 value); +int mpam_fb_get_protocol_version (struct mpam_msc *msc); + +#define MPAM_FB_PROT_HEADER_LEN sizeof(u32) +#define MPAM_FB_MAX_MSG_SIZE (MPAM_FB_PROT_HEADER_LEN + 4 * sizeof(u32)) +/* * MPAM MSCs have the following register layout. See: * Arm Memory System Resource Partitioning and Monitoring (MPAM) System
include/linux/arm_mpam.h +1 -1
@@ -12,7 +12,7 @@ struct mpam_msc;enum mpam_msc_iface {MPAM_IFACE_MMIO ,/* a real MPAM MSC */ -MPAM_IFACE_PCC ,/* a fake MPAM MSC */ +MPAM_IFACE_PCC ,/* using the MPAM-Fb firmware redirection */ };enum mpam_class_types {
Signature · 1 line
2.43.0
Sudeep Holla
maintainerRe: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
Quoted 46 lines · Andre Przywara
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 20 ++++ include/linux/arm_mpam.h | 2 +- 5 files changed, 240 insertions(+), 8 deletions(-) create mode 100644 drivers/resctrl/mpam_fb.cdrivers/resctrl/Makefile +1
-1@@ -1,5 +1,5 @@ obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o -mpam-y += mpam_devices.o +mpam-y += mpam_devices.o mpam_fb.o mpam-$(CONFIG_ARM64_MPAM_RESCTRL_FS) += mpam_resctrl.o ccflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUGdrivers/resctrl/mpam_devices.c +0
-0
drivers/resctrl/mpam_devices.c static int __mpam_read_reg(struct mpam_msc *msc, u16 reg, u32 *res) · line 183
Quoted 3 lines
drivers/resctrl/mpam_devices.c +0
-0@@ -181,6 +181,9 @@ static int __mpam_read_reg(struct mpam_msc *msc, u16 reg, u32 *res) {WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility));
The above doesn't make any sense for PCC accesses, perhaps need to be pushed down ?
drivers/resctrl/mpam_devices.c static inline int _mpam_read_partsel_reg(struct mpam_msc *msc, u16 reg, · line 204
Quoted 13 lines
drivers/resctrl/mpam_devices.c +3
-1+if (msc->iface ==MPAM_IFACE_PCC ) +return mpam_fb_send_read_request (msc, reg, res); + *res =readl_relaxed (msc->mapped_hwpage + reg);return 0 ;@@ -197,9 +200,12 @@ static inline int _mpam_read_partsel_reg(struct mpam_msc *msc, u16 reg,static int __mpam_write_reg (struct mpam_msc *msc,u16 reg,u32 val) { -WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility));
Ditto
Signature · 2 lines
Regards, Sudeep
Re: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
Hi Sudeep,
many thanks for having a look!
Quoted 41 lines · Sudeep Holla
Quoted 40 lines · Andre Przywara
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 20 ++++ include/linux/arm_mpam.h | 2 +- 5 files changed, 240 insertions(+), 8 deletions(-)create mode 100644
drivers/resctrl/mpam_fb.cdrivers/resctrl/Makefile +1
-1@@ -1,5 +1,5 @@ obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o -mpam-y += mpam_devices.o +mpam-y += mpam_devices.o mpam_fb.o mpam-$(CONFIG_ARM64_MPAM_RESCTRL_FS) += mpam_resctrl.o
Quoted 12 lines
drivers/resctrl/mpam_devices.c static int __mpam_read_reg(struct mpam_msc *msc, u16 reg, u32 *res) · line 183
Quoted 9 lines
drivers/resctrl/Makefile +0
-0ccflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUGdrivers/resctrl/mpam_devices.c +0
-0@@ -181,6 +181,9 @@ static int __mpam_read_reg(struct mpam_msc *msc, u16 reg, u32 *res) {WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility));The above doesn't make any sense for PCC accesses, perhaps need to be pushed down ?
Well, the idea was that the mask would always encompass all CPUs when using MPAM-Fb, but to consider this an implementation detail of MPAM-Fb. We can of course move this out of the way for MPAM-Fb.
Quoted 6 lines
Quoted 6 lines
drivers/resctrl/mpam_devices.c +3
-0+if (msc->iface ==MPAM_IFACE_PCC ) +return mpam_fb_send_read_request (msc, reg, res); + *res =readl_relaxed (msc->mapped_hwpage + reg);
Quoted 9 lines
drivers/resctrl/mpam_devices.c static inline int _mpam_read_partsel_reg(struct mpam_msc *msc, u16 reg, · line 204
Quoted 8 lines
drivers/resctrl/mpam_devices.c +0
-1return 0 ;@@ -197,9 +200,12 @@ static inline int _mpam_read_partsel_reg(struct mpam_msc *msc, u16 reg,static int __mpam_write_reg (struct mpam_msc *msc,u16 reg,u32 val) { -WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility));
And by the way: I think the hwpage_sz check should belong to the probe function, not be done on every access. Or are there optional registers that might not be part of the mapping on devices that don't support them?
Cheers,
Andre
Sudeep Holla
maintainerRe: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
Quoted 58 lines · Andre Przywara
Hi Sudeep,
many thanks for having a look!
Quoted 44 lines · Sudeep Holla
Quoted 43 lines · Andre Przywara
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 20 ++++ include/linux/arm_mpam.h | 2 +- 5 files changed, 240 insertions(+), 8 deletions(-)create mode 100644
drivers/resctrl/mpam_fb.cdrivers/resctrl/Makefile +1
-1@@ -1,5 +1,5 @@ obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o -mpam-y += mpam_devices.o +mpam-y += mpam_devices.o mpam_fb.o mpam-$(CONFIG_ARM64_MPAM_RESCTRL_FS) += mpam_resctrl.o ccflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUGdrivers/resctrl/mpam_devices.c +0
-1--- a/drivers/resctrl/mpam_devices.cQuoted 8 lines
drivers/resctrl/mpam_devices.c static int __mpam_read_reg(struct mpam_msc *msc, u16 reg, u32 *res) · line 183
Quoted 5 lines
+++
b/drivers/resctrl/mpam_devices.cdrivers/resctrl/mpam_devices.c +0
-0@@ -181,6 +181,9 @@ static int __mpam_read_reg(struct mpam_msc *msc, u16 reg, u32 *res) {WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility));The above doesn't make any sense for PCC accesses, perhaps need to be pushed down ?
Quoted 4 lines
Well, the idea was that the mask would always encompass all CPUs when using MPAM-Fb, but to consider this an implementation detail of MPAM-Fb. We can of course move this out of the way for MPAM-Fb.
I prefer that.
Quoted 13 lines
Quoted 12 lines
drivers/resctrl/mpam_devices.c static inline int _mpam_read_partsel_reg(struct mpam_msc *msc, u16 reg, · line 202
Quoted 11 lines
drivers/resctrl/mpam_devices.c +3
-1+if (msc->iface ==MPAM_IFACE_PCC ) +return mpam_fb_send_read_request (msc, reg, res); + *res =readl_relaxed (msc->mapped_hwpage + reg);return 0 ;@@ -197,9 +200,12 @@ static inline int _mpam_read_partsel_reg(struct mpam_msc *msc, u16 reg,static int __mpam_write_reg (struct mpam_msc *msc,u16 reg,u32 val) { -WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility));
Quoted 4 lines
And by the way: I think the
hwpage_szcheck should belong to the probe function, not be done on every access. Or are there optional registers that might not be part of the mapping on devices that don't support them?
No, I was just referring the cpumask and not hwpage_sz check.
Signature · 2 lines
Regards, Sudeep
Ben Horgan
maintainerRe: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
In reply to Andre PrzywaraHi Andre,
Quoted 122 lines · Andre Przywara
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 20 ++++ include/linux/arm_mpam.h | 2 +- 5 files changed, 240 insertions(+), 8 deletions(-) create mode 100644 drivers/resctrl/mpam_fb.cdrivers/resctrl/Makefile +1
-1@@ -1,5 +1,5 @@ obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o -mpam-y += mpam_devices.o +mpam-y += mpam_devices.o mpam_fb.o mpam-$(CONFIG_ARM64_MPAM_RESCTRL_FS) += mpam_resctrl.o ccflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUGdrivers/resctrl/mpam_devices.c +21
-6@@ -181,6 +181,9 @@ static int __mpam_read_reg(struct mpam_msc *msc, u16 reg, u32 *res) {WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); +if (msc->iface ==MPAM_IFACE_PCC ) +return mpam_fb_send_read_request (msc, reg, res); + *res =readl_relaxed (msc->mapped_hwpage + reg);return 0 ;@@ -197,9 +200,12 @@ static inline int _mpam_read_partsel_reg(struct mpam_msc *msc, u16 reg,static int __mpam_write_reg (struct mpam_msc *msc,u16 reg,u32 val) { -WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); +if (msc->iface ==MPAM_IFACE_PCC ) +return mpam_fb_send_write_request (msc, reg, val); + +WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);writel_relaxed (val, msc->mapped_hwpage + reg);return 0 ;@@ -1136,7 +1142,8 @@ static int mpam_msc_read_mbwu_l(struct mpam_msc *msc, u64 *res)mpam_mon_sel_lock_held (msc); -WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz); +if (msc->iface ==MPAM_IFACE_MMIO ) +WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); ret =__mpam_read_reg (msc,MSMON_MBWU_L +4 , &mbwu_l_high2);@@ -1485,11 +1492,16 @@ static int _msmon_read(struct mpam_component *comp, struct mon_read *arg) srcu_read_lock_held(&mpam_srcu)) { arg->ris = ris; - err =smp_call_function_any (&msc->accessibility, - __ris_msmon_read, arg, - true); -if (!err && arg->err) +if (msc->iface ==MPAM_IFACE_MMIO ) { + err =smp_call_function_any (&msc->accessibility, + __ris_msmon_read, + arg, true); +if (!err) + err = arg->err; + }else { +__ris_msmon_read (arg); err = arg->err; + }/* * Save one error to be returned to the caller, but @@ -1916,6 +1928,9 @@ static int mpam_get_msc_preferred_cpu(struct mpam_msc *msc)static int mpam_touch_msc (struct mpam_msc *msc,int (*fn)(void *a),void *arg) { +if (msc->iface !=MPAM_IFACE_MMIO ) +return fn (arg); +lockdep_assert_irqs_enabled ();lockdep_assert_cpus_held ();WARN_ON_ONCE (!srcu_read_lock_held ((&mpam_srcu)));drivers/resctrl/mpam_fb.c new file +6
-0@@ -0,0 +1,197 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2024 Arm Ltd. + +#include <linux/arm_mpam.h> +#include <linux/cleanup.h> +#include <linux/errno.h>
drivers/resctrl/mpam_fb.c line 16
Quoted 10 lines
drivers/resctrl/mpam_fb.c new file +10
-0+#include <linux/mailbox_client.h> +#include <linux/mutex.h> +#include <linux/types.h> + +#include <acpi/pcc.h> +#include <asm/mpam.h> + +#include "mpam_internal.h" + +#define MPAM_FB_PROTOCOL_ID 0x1a
A new line here would keep the commands separate.
drivers/resctrl/mpam_fb.c line 17
Quoted 1 line
drivers/resctrl/mpam_fb.c new file +1
-0+#define MPAM_PROTOCOL_VERSION 0x0
This one needs a _CMD as well.
Quoted 24 lines
drivers/resctrl/mpam_fb.c new file +24
-0+#define MPAM_MSC_ATTRIBUTES_CMD 0x3 +#define MPAM_MSC_READ_CMD 0x4 +#define MPAM_MSC_WRITE_CMD 0x5 + +#define MPAM_FB_ERR_SUCCESS 0 +#define MPAM_FB_ERR_NOT_SUPPORTED -1 +#define MPAM_FB_ERR_INVALID_PARAMETERS -2 +#define MPAM_FB_ERR_DENIED -3 +#define MPAM_FB_ERR_NOT_FOUND -4 +#define MPAM_FB_ERR_OUT_OF_RANGE -5 +#define MPAM_FB_ERR_BUSY -6 +#define MPAM_FB_ERR_COMMS_ERROR -7 +#define MPAM_FB_ERR_GENERIC_ERROR -8 +#define MPAM_FB_ERR_HW_ERROR -9 +#define MPAM_FB_ERR_PROTOCOL_ERROR -10 +#define MPAM_FB_ERR_IN_USE -11 + +#define MPAM_MSC_PROT_ID_MASK GENMASK(17, 10) +#define MPAM_MSC_TOKEN_MASK GENMASK(27, 18) + +struct mpam_fb_read_payload { +u32 msc_id; +u32 flags; +u32 reg_offset;
drivers/resctrl/mpam_fb.c line 51
Quoted 10 lines
drivers/resctrl/mpam_fb.c new file +10
-0+} __packed; + +struct mpam_fb_write_payload { +u32 msc_id; +u32 flags; +u32 reg_offset; +u32 value; +} __packed; + +#define PCC_CHAN_FLAGS_IRQ BIT(0)
Isn't this the same as PCC_CMD_COMPLETION_NOTIFY in include/acpi/pcc.h?
drivers/resctrl/mpam_fb.c line 58
Quoted 7 lines
drivers/resctrl/mpam_fb.c new file +7
-0+ +static atomic_t mpam_fb_token =ATOMIC_INIT (0 ); + +static void mpam_fb_build_version_message (unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf;
What's the deal with .signature?
drivers/resctrl/mpam_fb.c line 60
Quoted 2 lines
drivers/resctrl/mpam_fb.c new file +2
-0+ +writel_relaxed (PCC_CHAN_FLAGS_IRQ , &pcc_shmem->flags);
Are you sure it's ok to unconditionally set this irq bit? I know we found out that we do need it set sometimes. What does setting it actually based on what the platform does look like?
Thanks,
Ben
Quoted 202 lines
drivers/resctrl/mpam_fb.c new file +137
-0+writel_relaxed (MPAM_FB_PROT_HEADER_LEN , &pcc_shmem->length); +writel_relaxed (MPAM_PROTOCOL_VERSION | +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token) | +FIELD_PREP (MPAM_MSC_PROT_ID_MASK ,MPAM_FB_PROTOCOL_ID ), + &pcc_shmem->command); +} + +static void mpam_fb_build_read_message (int msc_id,int reg,unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf; +struct mpam_fb_read_payload __iomem *payload = msg_buf +sizeof (*pcc_shmem); +int msg_size =MPAM_FB_PROT_HEADER_LEN +sizeof (*payload); + +writel_relaxed (PCC_CHAN_FLAGS_IRQ , &pcc_shmem->flags); +writel_relaxed (msg_size, &pcc_shmem->length); +writel_relaxed (MPAM_MSC_READ_CMD | +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token) | +FIELD_PREP (MPAM_MSC_PROT_ID_MASK ,MPAM_FB_PROTOCOL_ID ), + &pcc_shmem->command); + +writel_relaxed (msc_id, &payload->msc_id); +writel_relaxed (0 , &payload->flags); +writel_relaxed (reg, &payload->reg_offset); +} + +static void mpam_fb_build_write_message (int msc_id,int reg,u32 val, +unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf; +struct mpam_fb_write_payload __iomem *payload = msg_buf +sizeof (*pcc_shmem); +int msg_size =MPAM_FB_PROT_HEADER_LEN +sizeof (*payload); + +writel_relaxed (PCC_CHAN_FLAGS_IRQ , &pcc_shmem->flags); +writel_relaxed (msg_size, &pcc_shmem->length); +writel_relaxed (MPAM_MSC_WRITE_CMD | +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token) | +FIELD_PREP (MPAM_MSC_PROT_ID_MASK ,MPAM_FB_PROTOCOL_ID ), + &pcc_shmem->command); + +writel_relaxed (msc_id, &payload->msc_id); +writel_relaxed (0 , &payload->flags); +writel_relaxed (reg, &payload->reg_offset); +writel_relaxed (val, &payload->value); +} + +static int mpam_fb_send_request (struct mpam_pcc_chan *pcc_chan,u32 msc_id, +u16 reg,u32 *result,int mpam_fb_command) +{ +unsigned int token =atomic_inc_return (&mpam_fb_token); +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem; +struct pcc_mbox_chan *chan; +void __iomem *payload_ofs; +u32 status; +int ret; + +if (!pcc_chan) +return -ENODEV ; + + chan = pcc_chan->pcc_chan; + +/* prune token to fit into the 10 bits inside the command register */ + token =FIELD_GET (MPAM_MSC_TOKEN_MASK , +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token)); + +guard (mutex)(&pcc_chan->pcc_chan_lock); + +switch (mpam_fb_command) { +case MPAM_MSC_WRITE_CMD : +mpam_fb_build_write_message (msc_id, reg, *result, + token, chan->shmem); +break ; +case MPAM_MSC_READ_CMD : +mpam_fb_build_read_message (msc_id, reg, token, chan->shmem); +break ; +case MPAM_PROTOCOL_VERSION : +mpam_fb_build_version_message (token, chan->shmem); +break ; + } + + ret =mbox_send_message (chan->mchan,NULL ); +if (ret <0 ) +return ret; + + pcc_shmem = chan->shmem; + payload_ofs = chan->shmem +sizeof (*pcc_shmem); + status =readl (&pcc_shmem->command); +if (FIELD_GET (MPAM_MSC_TOKEN_MASK , status) != token) +return -ETIMEDOUT ; + + ret =readl (payload_ofs +0x0 ); +if (ret <0 ) { +switch (ret) { +case MPAM_FB_ERR_NOT_SUPPORTED : +return -EOPNOTSUPP ; +case MPAM_FB_ERR_INVALID_PARAMETERS : +return -EINVAL ; +case MPAM_FB_ERR_NOT_FOUND : +return -ENOENT ; +case MPAM_FB_ERR_OUT_OF_RANGE : +return -ERANGE ; +default : +return -EINVAL ; + } + } + +if (mpam_fb_command !=MPAM_MSC_WRITE_CMD ) + *result =readl (payload_ofs +0x4 ); + +return 0 ; +} + +int mpam_fb_send_read_request (struct mpam_msc *msc,u16 reg,u32 *result) +{ +return mpam_fb_send_request (msc->pcc_chan, msc->mpam_fb_msc_id, + reg, result,MPAM_MSC_READ_CMD ); +} + +int mpam_fb_send_write_request (struct mpam_msc *msc,u16 reg,u32 value) +{ +return mpam_fb_send_request (msc->pcc_chan, msc->mpam_fb_msc_id, + reg, &value,MPAM_MSC_WRITE_CMD ); +} + +int mpam_fb_get_protocol_version (struct mpam_msc *msc) +{ +u32 version; +int ret; + + ret =mpam_fb_send_request (msc->pcc_chan,0 , +0 , &version,MPAM_PROTOCOL_VERSION ); +if (ret) +return ret; + +return version; +}drivers/resctrl/mpam_internal.h +20
-0@@ -11,6 +11,7 @@#include <linux/io.h> #include <linux/jump_label.h> #include <linux/llist.h> +#include <linux/mailbox_client.h> #include <linux/mutex.h> #include <linux/resctrl.h> #include <linux/spinlock.h> @@ -57,6 +58,15 @@ struct mpam_garbage {struct platform_device *pdev; }; +struct mpam_pcc_chan { +struct list_head pcc_chans; +struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +struct mutex pcc_chan_lock;/* only one message at a time */ +struct kref refcount; +int subspace_id; +}; +struct mpam_msc {/* member of mpam_all_msc */ struct list_head all_msc_list;@@ -66,6 +76,8 @@ struct mpam_msc {/* Not modified after mpam_is_enabled() becomes true */ enum mpam_msc_iface iface; +struct mpam_pcc_chan *pcc_chan; +int mpam_fb_msc_id;/* in its own name space */ u32 nrdy_usec;cpumask_t accessibility;bool has_extd_esr;@@ -511,6 +523,14 @@ static inline void mpam_resctrl_offline_cpu(unsigned int cpu) { }static inline void mpam_resctrl_teardown_class (struct mpam_class *class) { }#endif /* CONFIG_RESCTRL_FS */ +/* MPAM-Fb Firmware-backed protocol wrappers */ +int mpam_fb_send_read_request (struct mpam_msc *msc,u16 reg,u32 *result); +int mpam_fb_send_write_request (struct mpam_msc *msc,u16 reg,u32 value); +int mpam_fb_get_protocol_version (struct mpam_msc *msc); + +#define MPAM_FB_PROT_HEADER_LEN sizeof(u32) +#define MPAM_FB_MAX_MSG_SIZE (MPAM_FB_PROT_HEADER_LEN + 4 * sizeof(u32)) +/* * MPAM MSCs have the following register layout. See: * Arm Memory System Resource Partitioning and Monitoring (MPAM) System include/linux/arm_mpam.h +1
-1@@ -12,7 +12,7 @@ struct mpam_msc;enum mpam_msc_iface {MPAM_IFACE_MMIO ,/* a real MPAM MSC */ -MPAM_IFACE_PCC ,/* a fake MPAM MSC */ +MPAM_IFACE_PCC ,/* using the MPAM-Fb firmware redirection */ };enum mpam_class_types {
Sudeep Holla
maintainerRe: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
Quoted 33 lines · Ben Horgan
Hi Andre,
Quoted 30 lines · Andre Przywara
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 20 ++++ include/linux/arm_mpam.h | 2 +- 5 files changed, 240 insertions(+), 8 deletions(-) create mode 100644 drivers/resctrl/mpam_fb.c
[...]
Quoted 8 lines
Quoted 8 lines
drivers/resctrl/mpam_fb.c new file +8
-0+struct mpam_fb_read_payload { +u32 msc_id; +u32 flags; +u32 reg_offset; +} __packed; + +struct mpam_fb_write_payload { +u32 msc_id;
Quoted 9 lines
drivers/resctrl/mpam_fb.c line 51
Quoted 6 lines
drivers/resctrl/mpam_fb.c new file +6
-0+u32 flags; +u32 reg_offset; +u32 value; +} __packed; + +#define PCC_CHAN_FLAGS_IRQ BIT(0)Isn't this the same as
PCC_CMD_COMPLETION_NOTIFYininclude/acpi/pcc.h?
Yes, where ever possible, one must use that. I have TODO to unify the loads of duplication we have amongst the users of PCC channels. That's the reason I ignored more duplications in this driver in absence of any alternatives.
Quoted 10 lines
drivers/resctrl/mpam_fb.c line 58
Quoted 7 lines
drivers/resctrl/mpam_fb.c new file +7
-0+ +static atomic_t mpam_fb_token =ATOMIC_INIT (0 ); + +static void mpam_fb_build_version_message (unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf;What's the deal with .signature?
No, it's written by the platform firmware when the channel is ready to be used by OSPM and OSPM just needs to verify. It will be soon part of the PCC core driver and it will shout if that's not the case.
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ + writel_relaxed(PCC_CHAN_FLAGS_IRQ, &pcc_shmem->flags);Are you sure it's ok to unconditionally set this irq bit? I know we found out that we do need it set sometimes. What does setting it actually based on what the platform does look like?
You need to set it if you are not polling the PCC channels for completions.
--
Regards,
Sudeep
Re: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
Hi,
Quoted 36 lines · Sudeep Holla
Quoted 33 lines · Ben Horgan
Hi Andre,
Quoted 30 lines · Andre Przywara
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 20 ++++ include/linux/arm_mpam.h | 2 +- 5 files changed, 240 insertions(+), 8 deletions(-)create mode 100644
drivers/resctrl/mpam_fb.c[...]
[ ... ]
Quoted 8 lines
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ + writel_relaxed(PCC_CHAN_FLAGS_IRQ, &pcc_shmem->flags);Are you sure it's ok to unconditionally set this irq bit? I know we found out that we do need it set sometimes. What does setting it actually based on what the platform does look like?
You need to set it if you are not polling the PCC channels for completions.
So I was wondering about that: why does the PCC *user* need to set it here? That's probably more of a protocol issue, but to me it breaks the abstraction: MPAM-Fb just uses the mailbox provided via PCC, and doesn't care about whether it's polling or using an interrupt, because that is a PCC detail. So is there a way to figure this out cleanly? I only found that one could possibly chase down the "struct mbox_chan" controller struct, then look at txdone_method, or something similar, but that sounds very dodgy?
Do you have any ideas on this?
Cheers,
Andre
Re: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
In reply to Ben HorganHi Ben,
Quoted 33 lines · Ben Horgan
Hi Andre,
Quoted 30 lines · Andre Przywara
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 20 ++++ include/linux/arm_mpam.h | 2 +- 5 files changed, 240 insertions(+), 8 deletions(-)create mode 100644
drivers/resctrl/mpam_fb.c
[ ... ]
Quoted 16 lines
Quoted 16 lines
drivers/resctrl/mpam_fb.c new file +10
-0@@ -0,0 +1,197 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2024 Arm Ltd. + +#include <linux/arm_mpam.h> +#include <linux/cleanup.h> +#include <linux/errno.h> +#include <linux/mailbox_client.h> +#include <linux/mutex.h> +#include <linux/types.h> +
Quoted 8 lines
drivers/resctrl/mpam_fb.c line 16
Quoted 6 lines
drivers/resctrl/mpam_fb.c new file +6
-0+#include <acpi/pcc.h> +#include <asm/mpam.h> + +#include "mpam_internal.h" + +#define MPAM_FB_PROTOCOL_ID 0x1aA new line here would keep the commands separate.
Sure.
Quoted 2 lines
drivers/resctrl/mpam_fb.c line 17
Quoted 1 line
drivers/resctrl/mpam_fb.c new file +1
-0+#define MPAM_PROTOCOL_VERSION 0x0This one needs a
_CMDas well.
You got it.
Quoted 28 lines
Quoted 28 lines
drivers/resctrl/mpam_fb.c new file +28
-0+#define MPAM_MSC_ATTRIBUTES_CMD 0x3 +#define MPAM_MSC_READ_CMD 0x4 +#define MPAM_MSC_WRITE_CMD 0x5 + +#define MPAM_FB_ERR_SUCCESS 0 +#define MPAM_FB_ERR_NOT_SUPPORTED -1 +#define MPAM_FB_ERR_INVALID_PARAMETERS -2 +#define MPAM_FB_ERR_DENIED -3 +#define MPAM_FB_ERR_NOT_FOUND -4 +#define MPAM_FB_ERR_OUT_OF_RANGE -5 +#define MPAM_FB_ERR_BUSY -6 +#define MPAM_FB_ERR_COMMS_ERROR -7 +#define MPAM_FB_ERR_GENERIC_ERROR -8 +#define MPAM_FB_ERR_HW_ERROR -9 +#define MPAM_FB_ERR_PROTOCOL_ERROR -10 +#define MPAM_FB_ERR_IN_USE -11 + +#define MPAM_MSC_PROT_ID_MASK GENMASK(17, 10) +#define MPAM_MSC_TOKEN_MASK GENMASK(27, 18) + +struct mpam_fb_read_payload { +u32 msc_id; +u32 flags; +u32 reg_offset; +} __packed; + +struct mpam_fb_write_payload { +u32 msc_id;
Quoted 8 lines
drivers/resctrl/mpam_fb.c line 51
Quoted 6 lines
drivers/resctrl/mpam_fb.c new file +6
-0+u32 flags; +u32 reg_offset; +u32 value; +} __packed; + +#define PCC_CHAN_FLAGS_IRQ BIT(0)Isn't this the same as
PCC_CMD_COMPLETION_NOTIFYininclude/acpi/pcc.h?
Looks like it, nice, will use that instead.
Quoted 9 lines
drivers/resctrl/mpam_fb.c line 58
Quoted 7 lines
drivers/resctrl/mpam_fb.c new file +7
-0+ +static atomic_t mpam_fb_token =ATOMIC_INIT (0 ); + +static void mpam_fb_build_version_message (unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf;What's the deal with .signature?
As Sudeep said: it's the platform's (SCP's) responsibility to set it. Will add a comment to that affect.
Quoted 5 lines
drivers/resctrl/mpam_fb.c line 60
Quoted 2 lines
drivers/resctrl/mpam_fb.c new file +2
-0+ +writel_relaxed (PCC_CHAN_FLAGS_IRQ , &pcc_shmem->flags);Are you sure it's ok to unconditionally set this irq bit? I know we found out that we do need it set sometimes. What does setting it actually based on what the platform does look like?
I guess that depends on the SCP implementation, but I think it always needs to set the channel free bit, as it's part of the protocol. This is what the PCC code would poll for. The interrupt is just on top. So if we set the bit here, and the SCP doesn't support an interrupt, it would still work, as the PCC code would poll, regardless of that bit here, which it doesn't know about.
Cheers,
Andre.
Quoted 208 lines
Thanks,
Ben
Quoted 202 lines
drivers/resctrl/mpam_fb.c new file +137
-0+writel_relaxed (MPAM_FB_PROT_HEADER_LEN , &pcc_shmem->length); +writel_relaxed (MPAM_PROTOCOL_VERSION | +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token) | +FIELD_PREP (MPAM_MSC_PROT_ID_MASK ,MPAM_FB_PROTOCOL_ID ), + &pcc_shmem->command); +} + +static void mpam_fb_build_read_message (int msc_id,int reg,unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf; +struct mpam_fb_read_payload __iomem *payload = msg_buf +sizeof (*pcc_shmem); +int msg_size =MPAM_FB_PROT_HEADER_LEN +sizeof (*payload); + +writel_relaxed (PCC_CHAN_FLAGS_IRQ , &pcc_shmem->flags); +writel_relaxed (msg_size, &pcc_shmem->length); +writel_relaxed (MPAM_MSC_READ_CMD | +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token) | +FIELD_PREP (MPAM_MSC_PROT_ID_MASK ,MPAM_FB_PROTOCOL_ID ), + &pcc_shmem->command); + +writel_relaxed (msc_id, &payload->msc_id); +writel_relaxed (0 , &payload->flags); +writel_relaxed (reg, &payload->reg_offset); +} + +static void mpam_fb_build_write_message (int msc_id,int reg,u32 val, +unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf; +struct mpam_fb_write_payload __iomem *payload = msg_buf +sizeof (*pcc_shmem); +int msg_size =MPAM_FB_PROT_HEADER_LEN +sizeof (*payload); + +writel_relaxed (PCC_CHAN_FLAGS_IRQ , &pcc_shmem->flags); +writel_relaxed (msg_size, &pcc_shmem->length); +writel_relaxed (MPAM_MSC_WRITE_CMD | +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token) | +FIELD_PREP (MPAM_MSC_PROT_ID_MASK ,MPAM_FB_PROTOCOL_ID ), + &pcc_shmem->command); + +writel_relaxed (msc_id, &payload->msc_id); +writel_relaxed (0 , &payload->flags); +writel_relaxed (reg, &payload->reg_offset); +writel_relaxed (val, &payload->value); +} + +static int mpam_fb_send_request (struct mpam_pcc_chan *pcc_chan,u32 msc_id, +u16 reg,u32 *result,int mpam_fb_command) +{ +unsigned int token =atomic_inc_return (&mpam_fb_token); +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem; +struct pcc_mbox_chan *chan; +void __iomem *payload_ofs; +u32 status; +int ret; + +if (!pcc_chan) +return -ENODEV ; + + chan = pcc_chan->pcc_chan; + +/* prune token to fit into the 10 bits inside the command register */ + token =FIELD_GET (MPAM_MSC_TOKEN_MASK , +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token)); + +guard (mutex)(&pcc_chan->pcc_chan_lock); + +switch (mpam_fb_command) { +case MPAM_MSC_WRITE_CMD : +mpam_fb_build_write_message (msc_id, reg, *result, + token, chan->shmem); +break ; +case MPAM_MSC_READ_CMD : +mpam_fb_build_read_message (msc_id, reg, token, chan->shmem); +break ; +case MPAM_PROTOCOL_VERSION : +mpam_fb_build_version_message (token, chan->shmem); +break ; + } + + ret =mbox_send_message (chan->mchan,NULL ); +if (ret <0 ) +return ret; + + pcc_shmem = chan->shmem; + payload_ofs = chan->shmem +sizeof (*pcc_shmem); + status =readl (&pcc_shmem->command); +if (FIELD_GET (MPAM_MSC_TOKEN_MASK , status) != token) +return -ETIMEDOUT ; + + ret =readl (payload_ofs +0x0 ); +if (ret <0 ) { +switch (ret) { +case MPAM_FB_ERR_NOT_SUPPORTED : +return -EOPNOTSUPP ; +case MPAM_FB_ERR_INVALID_PARAMETERS : +return -EINVAL ; +case MPAM_FB_ERR_NOT_FOUND : +return -ENOENT ; +case MPAM_FB_ERR_OUT_OF_RANGE : +return -ERANGE ; +default : +return -EINVAL ; + } + } + +if (mpam_fb_command !=MPAM_MSC_WRITE_CMD ) + *result =readl (payload_ofs +0x4 ); + +return 0 ; +} + +int mpam_fb_send_read_request (struct mpam_msc *msc,u16 reg,u32 *result) +{ +return mpam_fb_send_request (msc->pcc_chan, msc->mpam_fb_msc_id, + reg, result,MPAM_MSC_READ_CMD ); +} + +int mpam_fb_send_write_request (struct mpam_msc *msc,u16 reg,u32 value) +{ +return mpam_fb_send_request (msc->pcc_chan, msc->mpam_fb_msc_id, + reg, &value,MPAM_MSC_WRITE_CMD ); +} + +int mpam_fb_get_protocol_version (struct mpam_msc *msc) +{ +u32 version; +int ret; + + ret =mpam_fb_send_request (msc->pcc_chan,0 , +0 , &version,MPAM_PROTOCOL_VERSION ); +if (ret) +return ret; + +return version; +}drivers/resctrl/mpam_internal.h +20
-0@@ -11,6 +11,7 @@#include <linux/io.h> #include <linux/jump_label.h> #include <linux/llist.h> +#include <linux/mailbox_client.h> #include <linux/mutex.h> #include <linux/resctrl.h> #include <linux/spinlock.h> @@ -57,6 +58,15 @@ struct mpam_garbage {struct platform_device *pdev; }; +struct mpam_pcc_chan { +struct list_head pcc_chans; +struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +struct mutex pcc_chan_lock;/* only one message at a time */ +struct kref refcount; +int subspace_id; +}; +struct mpam_msc {/* member of mpam_all_msc */ struct list_head all_msc_list;@@ -66,6 +76,8 @@ struct mpam_msc {/* Not modified after mpam_is_enabled() becomes true */ enum mpam_msc_iface iface; +struct mpam_pcc_chan *pcc_chan; +int mpam_fb_msc_id;/* in its own name space */ u32 nrdy_usec;cpumask_t accessibility;bool has_extd_esr;@@ -511,6 +523,14 @@ static inline void mpam_resctrl_offline_cpu(unsigned int cpu) { }static inline void mpam_resctrl_teardown_class (struct mpam_class *class) { }#endif /* CONFIG_RESCTRL_FS */ +/* MPAM-Fb Firmware-backed protocol wrappers */ +int mpam_fb_send_read_request (struct mpam_msc *msc,u16 reg,u32 *result); +int mpam_fb_send_write_request (struct mpam_msc *msc,u16 reg,u32 value); +int mpam_fb_get_protocol_version (struct mpam_msc *msc); + +#define MPAM_FB_PROT_HEADER_LEN sizeof(u32) +#define MPAM_FB_MAX_MSG_SIZE (MPAM_FB_PROT_HEADER_LEN + 4 * sizeof(u32)) +/* * MPAM MSCs have the following register layout. See: * Arm Memory System Resource Partitioning and Monitoring (MPAM) System include/linux/arm_mpam.h +1
-1@@ -12,7 +12,7 @@ struct mpam_msc;enum mpam_msc_iface {MPAM_IFACE_MMIO ,/* a real MPAM MSC */ -MPAM_IFACE_PCC ,/* a fake MPAM MSC */ +MPAM_IFACE_PCC ,/* using the MPAM-Fb firmware redirection */ };enum mpam_class_types {
Re: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
In reply to Andre PrzywaraQuoted 14 lines · On Thu, 23 Jul 2026 17:54:52 +0200 Andre Przywara <[email protected]>…
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Quoted 7 lines
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>
Just a couple of suggestions that you can ignore if you like
drivers/resctrl/mpam_fb.c line 2
Quoted 8 lines
drivers/resctrl/mpam_fb.c new file +2
-0@@ -0,0 +1,197 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2024 Arm Ltd.
Arm has either been working on this for rather a long time or that's a cut and paste. Even if 2024 is a good start given changes during revisions seems reasonable to include 2026 as well.
drivers/resctrl/mpam_fb.c line 44
Quoted 7 lines
drivers/resctrl/mpam_fb.c new file +7
-0+struct mpam_fb_read_payload { +u32 msc_id; +u32 flags; +u32 reg_offset; +} __packed; + +struct mpam_fb_write_payload {
Given these are only used in the specific functions that build the messages you could move the structure definitions to be local to those functions.
Alternatively you could move all these to the header and get the advantage of using a union to get the maximum message size rather than a number currently picked out of the air.
Quoted 12 lines
drivers/resctrl/mpam_fb.c new file +5
-0+u32 msc_id; +u32 flags; +u32 reg_offset; +u32 value; +} __packed;drivers/resctrl/mpam_internal.h +0
-0
...
drivers/resctrl/mpam_internal.h static inline void mpam_resctrl_offline_cpu(unsigned int cpu) { } · line 532
Quoted 2 lines
drivers/resctrl/mpam_internal.h +2
-0+#define MPAM_FB_PROT_HEADER_LEN sizeof(u32) +#define MPAM_FB_MAX_MSG_SIZE (MPAM_FB_PROT_HEADER_LEN + 4 * sizeof(u32))
Doesn't seem to be used until patch 10. Might be more sensible to bring it in there. A spec reference might also be good for this. I think longest is current the MPAM_MSC_WRITE. A note above that one option would be to make this explicit by having the message structure definitions in this header.
Quoted 1 line
+
Re: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
Hi Jonathan,
Quoted 21 lines · Jonathan Cameron
Quoted 18 lines · On Thu, 23 Jul 2026 17:54:52 +0200 Andre Przywara <[email protected]>…
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
Quoted 6 lines
Quoted 3 lines
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>Just a couple of suggestions that you can ignore if you like
Reviewed-by Jonathan Cameron <[email protected]>
Thanks!
Quoted 13 lines
drivers/resctrl/mpam_fb.c line 2
Quoted 9 lines
drivers/resctrl/mpam_fb.c new file +2
-0@@ -0,0 +1,197 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2024 Arm Ltd. Arm has either been working on this for rather a long time or that's a cut and paste. Even if 2024 is a good start given changes during revisions seems reasonable to include 2026 as well.
===================
Date: Tue May 21 11:26:16 2024 +0100
firmware: arm_scmi: add MPAM-FB SCMI protocol stub
========================
So yes, it's been in the making for a while, although in a different shape and form. But will surely add the current year to the range.
Quoted 12 lines
drivers/resctrl/mpam_fb.c line 44
Quoted 7 lines
drivers/resctrl/mpam_fb.c new file +7
-0+struct mpam_fb_read_payload { +u32 msc_id; +u32 flags; +u32 reg_offset; +} __packed; + +struct mpam_fb_write_payload {Given these are only used in the specific functions that build the messages you could move the structure definitions to be local to those functions.
Quoted 3 lines
Alternatively you could move all these to the header and get the advantage of using a union to get the maximum message size rather than a number currently picked out of the air.
A union sounds nice, but it feels a bit over the top, and I think keeping the structs local to their users trumps this. After all this is what mpam_fb.c does: deal with the protocol internals, without bothering the rest of the code.
Quoted 16 lines
Quoted 13 lines
drivers/resctrl/mpam_fb.c new file +5
-0+u32 msc_id; +u32 flags; +u32 reg_offset; +u32 value; +} __packed;drivers/resctrl/mpam_internal.h +0
-0...
Quoted 8 lines
drivers/resctrl/mpam_internal.h static inline void mpam_resctrl_offline_cpu(unsigned int cpu) { } · line 532
Quoted 2 lines
drivers/resctrl/mpam_internal.h +2
-0+#define MPAM_FB_PROT_HEADER_LEN sizeof(u32) +#define MPAM_FB_MAX_MSG_SIZE (MPAM_FB_PROT_HEADER_LEN + 4 * sizeof(u32))Doesn't seem to be used until patch 10. Might be more sensible to bring it in there. A spec reference might also be good for this. I think longest is current the
MPAM_MSC_WRITE. A note above that one option would be to make this explicit by having the message structure definitions in this header.
Yes, but I will not boil the ocean, and just add a comment.
Thanks,
Andre
Re: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
In reply to Andre PrzywaraHi Andre,
Quoted 246 lines · Andre Przywara
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 20 ++++ include/linux/arm_mpam.h | 2 +- 5 files changed, 240 insertions(+), 8 deletions(-)create mode 100644
drivers/resctrl/mpam_fb.cdrivers/resctrl/Makefile +1
-1@@ -1,5 +1,5 @@ obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o -mpam-y += mpam_devices.o +mpam-y += mpam_devices.o mpam_fb.o mpam-$(CONFIG_ARM64_MPAM_RESCTRL_FS) += mpam_resctrl.o ccflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUGdrivers/resctrl/mpam_devices.c +21
-6@@ -181,6 +181,9 @@ static int __mpam_read_reg(struct mpam_msc *msc, u16 reg, u32 *res) {WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); +if (msc->iface ==MPAM_IFACE_PCC ) +return mpam_fb_send_read_request (msc, reg, res); + *res =readl_relaxed (msc->mapped_hwpage + reg);return 0 ;@@ -197,9 +200,12 @@ static inline int _mpam_read_partsel_reg(struct mpam_msc *msc, u16 reg,static int __mpam_write_reg (struct mpam_msc *msc,u16 reg,u32 val) { -WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); +if (msc->iface ==MPAM_IFACE_PCC ) +return mpam_fb_send_write_request (msc, reg, val); + +WARN_ON_ONCE (reg +sizeof (u32) > msc->mapped_hwpage_sz);writel_relaxed (val, msc->mapped_hwpage + reg);return 0 ;@@ -1136,7 +1142,8 @@ static int mpam_msc_read_mbwu_l(struct mpam_msc *msc, u64 *res)mpam_mon_sel_lock_held (msc); -WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz); +if (msc->iface ==MPAM_IFACE_MMIO ) +WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz);WARN_ON_ONCE (!cpumask_test_cpu (smp_processor_id (), &msc->accessibility)); ret =__mpam_read_reg (msc,MSMON_MBWU_L +4 , &mbwu_l_high2);@@ -1485,11 +1492,16 @@ static int _msmon_read(struct mpam_component *comp, struct mon_read *arg) srcu_read_lock_held(&mpam_srcu)) { arg->ris = ris; - err =smp_call_function_any (&msc->accessibility, - __ris_msmon_read, arg, - true); -if (!err && arg->err) +if (msc->iface ==MPAM_IFACE_MMIO ) { + err =smp_call_function_any (&msc->accessibility, + __ris_msmon_read, + arg, true); +if (!err) + err = arg->err; + }else { +__ris_msmon_read (arg); err = arg->err; + }/* * Save one error to be returned to the caller, but @@ -1916,6 +1928,9 @@ static int mpam_get_msc_preferred_cpu(struct mpam_msc *msc)static int mpam_touch_msc (struct mpam_msc *msc,int (*fn)(void *a),void *arg) { +if (msc->iface !=MPAM_IFACE_MMIO ) +return fn (arg); +lockdep_assert_irqs_enabled ();lockdep_assert_cpus_held ();WARN_ON_ONCE (!srcu_read_lock_held ((&mpam_srcu)));drivers/resctrl/mpam_fb.c new file +130
-0@@ -0,0 +1,197 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2024 Arm Ltd. + +#include <linux/arm_mpam.h> +#include <linux/cleanup.h> +#include <linux/errno.h> +#include <linux/mailbox_client.h> +#include <linux/mutex.h> +#include <linux/types.h> + +#include <acpi/pcc.h> +#include <asm/mpam.h> + +#include "mpam_internal.h" + +#define MPAM_FB_PROTOCOL_ID 0x1a +#define MPAM_PROTOCOL_VERSION 0x0 +#define MPAM_MSC_ATTRIBUTES_CMD 0x3 +#define MPAM_MSC_READ_CMD 0x4 +#define MPAM_MSC_WRITE_CMD 0x5 + +#define MPAM_FB_ERR_SUCCESS 0 +#define MPAM_FB_ERR_NOT_SUPPORTED -1 +#define MPAM_FB_ERR_INVALID_PARAMETERS -2 +#define MPAM_FB_ERR_DENIED -3 +#define MPAM_FB_ERR_NOT_FOUND -4 +#define MPAM_FB_ERR_OUT_OF_RANGE -5 +#define MPAM_FB_ERR_BUSY -6 +#define MPAM_FB_ERR_COMMS_ERROR -7 +#define MPAM_FB_ERR_GENERIC_ERROR -8 +#define MPAM_FB_ERR_HW_ERROR -9 +#define MPAM_FB_ERR_PROTOCOL_ERROR -10 +#define MPAM_FB_ERR_IN_USE -11 + +#define MPAM_MSC_PROT_ID_MASK GENMASK(17, 10) +#define MPAM_MSC_TOKEN_MASK GENMASK(27, 18) + +struct mpam_fb_read_payload { +u32 msc_id; +u32 flags; +u32 reg_offset; +} __packed; + +struct mpam_fb_write_payload { +u32 msc_id; +u32 flags; +u32 reg_offset; +u32 value; +} __packed; + +#define PCC_CHAN_FLAGS_IRQ BIT(0) + +static atomic_t mpam_fb_token =ATOMIC_INIT (0 ); + +static void mpam_fb_build_version_message (unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf; + +writel_relaxed (PCC_CHAN_FLAGS_IRQ , &pcc_shmem->flags); +writel_relaxed (MPAM_FB_PROT_HEADER_LEN , &pcc_shmem->length); +writel_relaxed (MPAM_PROTOCOL_VERSION | +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token) | +FIELD_PREP (MPAM_MSC_PROT_ID_MASK ,MPAM_FB_PROTOCOL_ID ), + &pcc_shmem->command); +} + +static void mpam_fb_build_read_message (int msc_id,int reg,unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf; +struct mpam_fb_read_payload __iomem *payload = msg_buf +sizeof (*pcc_shmem); +int msg_size =MPAM_FB_PROT_HEADER_LEN +sizeof (*payload); + +writel_relaxed (PCC_CHAN_FLAGS_IRQ , &pcc_shmem->flags); +writel_relaxed (msg_size, &pcc_shmem->length); +writel_relaxed (MPAM_MSC_READ_CMD | +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token) | +FIELD_PREP (MPAM_MSC_PROT_ID_MASK ,MPAM_FB_PROTOCOL_ID ), + &pcc_shmem->command); + +writel_relaxed (msc_id, &payload->msc_id); +writel_relaxed (0 , &payload->flags); +writel_relaxed (reg, &payload->reg_offset); +} + +static void mpam_fb_build_write_message (int msc_id,int reg,u32 val, +unsigned int token, +void __iomem *msg_buf) +{ +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem = msg_buf; +struct mpam_fb_write_payload __iomem *payload = msg_buf +sizeof (*pcc_shmem); +int msg_size =MPAM_FB_PROT_HEADER_LEN +sizeof (*payload); + +writel_relaxed (PCC_CHAN_FLAGS_IRQ , &pcc_shmem->flags); +writel_relaxed (msg_size, &pcc_shmem->length); +writel_relaxed (MPAM_MSC_WRITE_CMD | +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token) | +FIELD_PREP (MPAM_MSC_PROT_ID_MASK ,MPAM_FB_PROTOCOL_ID ), + &pcc_shmem->command); + +writel_relaxed (msc_id, &payload->msc_id); +writel_relaxed (0 , &payload->flags); +writel_relaxed (reg, &payload->reg_offset); +writel_relaxed (val, &payload->value); +} + +static int mpam_fb_send_request (struct mpam_pcc_chan *pcc_chan,u32 msc_id, +u16 reg,u32 *result,int mpam_fb_command) +{ +unsigned int token =atomic_inc_return (&mpam_fb_token); +struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem; +struct pcc_mbox_chan *chan; +void __iomem *payload_ofs; +u32 status; +int ret; + +if (!pcc_chan) +return -ENODEV ; + + chan = pcc_chan->pcc_chan; + +/* prune token to fit into the 10 bits inside the command register */ + token =FIELD_GET (MPAM_MSC_TOKEN_MASK , +FIELD_PREP (MPAM_MSC_TOKEN_MASK , token)); + +guard (mutex)(&pcc_chan->pcc_chan_lock); + +switch (mpam_fb_command) { +case MPAM_MSC_WRITE_CMD :
drivers/resctrl/mpam_fb.c line 140
Quoted 10 lines
drivers/resctrl/mpam_fb.c new file +10
-0+mpam_fb_build_write_message (msc_id, reg, *result, + token, chan->shmem); +break ; +case MPAM_MSC_READ_CMD : +mpam_fb_build_read_message (msc_id, reg, token, chan->shmem); +break ; +case MPAM_PROTOCOL_VERSION : +mpam_fb_build_version_message (token, chan->shmem); +break ; + }
The switch here has no default case. If an unrecognised command value is passed (in future), it maybe safer and future proof to have a default case?
default:
return -EINVAL; ?> +
Quoted 113 lines
drivers/resctrl/mpam_fb.c new file +56
-0+ ret =mbox_send_message (chan->mchan,NULL ); +if (ret <0 ) +return ret; + + pcc_shmem = chan->shmem; + payload_ofs = chan->shmem +sizeof (*pcc_shmem); + status =readl (&pcc_shmem->command); +if (FIELD_GET (MPAM_MSC_TOKEN_MASK , status) != token) +return -ETIMEDOUT ; + + ret =readl (payload_ofs +0x0 ); +if (ret <0 ) { +switch (ret) { +case MPAM_FB_ERR_NOT_SUPPORTED : +return -EOPNOTSUPP ; +case MPAM_FB_ERR_INVALID_PARAMETERS : +return -EINVAL ; +case MPAM_FB_ERR_NOT_FOUND : +return -ENOENT ; +case MPAM_FB_ERR_OUT_OF_RANGE : +return -ERANGE ; +default : +return -EINVAL ; + } + } + +if (mpam_fb_command !=MPAM_MSC_WRITE_CMD ) + *result =readl (payload_ofs +0x4 ); + +return 0 ; +} + +int mpam_fb_send_read_request (struct mpam_msc *msc,u16 reg,u32 *result) +{ +return mpam_fb_send_request (msc->pcc_chan, msc->mpam_fb_msc_id, + reg, result,MPAM_MSC_READ_CMD ); +} + +int mpam_fb_send_write_request (struct mpam_msc *msc,u16 reg,u32 value) +{ +return mpam_fb_send_request (msc->pcc_chan, msc->mpam_fb_msc_id, + reg, &value,MPAM_MSC_WRITE_CMD ); +} + +int mpam_fb_get_protocol_version (struct mpam_msc *msc) +{ +u32 version; +int ret; + + ret =mpam_fb_send_request (msc->pcc_chan,0 , +0 , &version,MPAM_PROTOCOL_VERSION ); +if (ret) +return ret; + +return version; +}drivers/resctrl/mpam_internal.h +20
-0@@ -11,6 +11,7 @@#include <linux/io.h> #include <linux/jump_label.h> #include <linux/llist.h> +#include <linux/mailbox_client.h> #include <linux/mutex.h> #include <linux/resctrl.h> #include <linux/spinlock.h> @@ -57,6 +58,15 @@ struct mpam_garbage {struct platform_device *pdev; }; +struct mpam_pcc_chan { +struct list_head pcc_chans; +struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +struct mutex pcc_chan_lock;/* only one message at a time */ +struct kref refcount; +int subspace_id; +}; +struct mpam_msc {/* member of mpam_all_msc */ struct list_head all_msc_list;@@ -66,6 +76,8 @@ struct mpam_msc {/* Not modified after mpam_is_enabled() becomes true */ enum mpam_msc_iface iface; +struct mpam_pcc_chan *pcc_chan; +int mpam_fb_msc_id;/* in its own name space */ u32 nrdy_usec;cpumask_t accessibility;bool has_extd_esr;@@ -511,6 +523,14 @@ static inline void mpam_resctrl_offline_cpu(unsigned int cpu) { }static inline void mpam_resctrl_teardown_class (struct mpam_class *class) { }#endif /* CONFIG_RESCTRL_FS */ +/* MPAM-Fb Firmware-backed protocol wrappers */ +int mpam_fb_send_read_request (struct mpam_msc *msc,u16 reg,u32 *result); +int mpam_fb_send_write_request (struct mpam_msc *msc,u16 reg,u32 value); +int mpam_fb_get_protocol_version (struct mpam_msc *msc); + +#define MPAM_FB_PROT_HEADER_LEN sizeof(u32) +#define MPAM_FB_MAX_MSG_SIZE (MPAM_FB_PROT_HEADER_LEN + 4 * sizeof(u32)) +/* * MPAM MSCs have the following register layout. See: * Arm Memory System Resource Partitioning and Monitoring (MPAM) System include/linux/arm_mpam.h +0
-0
include/linux/arm_mpam.h struct mpam_msc; · line 18
Quoted 8 lines
include/linux/arm_mpam.h +1
-1@@ -12,7 +12,7 @@ struct mpam_msc;enum mpam_msc_iface {MPAM_IFACE_MMIO ,/* a real MPAM MSC */ -MPAM_IFACE_PCC ,/* a fake MPAM MSC */ +MPAM_IFACE_PCC ,/* using the MPAM-Fb firmware redirection */ };enum mpam_class_types {
Best Regards,
Srivathsa
Re: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
Hi Srivathsa,
thanks for having a look!
Quoted 32 lines · Srivathsa L Rao
Hi Andre,
Quoted 29 lines · Andre Przywara
The Arm MPAM Firmware-backed (Fb) Profile document[1] describes an alternative way of accessing the "Memory System Components" (MSC) in an MPAM enabled system.
Normally the MSCs are MMIO mapped, but in some implementations this might not be possible (MSC located outside of the local socket, MSC mapped secure-only) or desirable (direct MMIO access too slow or needs to be mediated through a control processor). MPAM-fb standardises a protocol to abstract MSC accesses, building on the SCMI protocol.
Add functions that do an MSC read or write access by redirecting the request through a firmware interface. For now this done via an ACPI PCC shared memory and mailbox combination.
Since the protocol used is only a small subset of the full SCMI spec, and the SCMI protocol has no full ACPI support anyway, open-code the (simple) SCMI message generation, for just the fields we need.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>---
drivers/resctrl/Makefile | 2 +- drivers/resctrl/mpam_devices.c | 27 ++++- drivers/resctrl/mpam_fb.c | 197 ++++++++++++++++++++++++++++++++drivers/resctrl/mpam_internal.h| 20 ++++ include/linux/arm_mpam.h | 2 +-5 files changed, 240 insertions(+), 8 deletions(-)create mode 100644
drivers/resctrl/mpam_fb.c
[ ... ]
Quoted 6 lines
Quoted 6 lines
drivers/resctrl/mpam_fb.c new file +0
-0@@ -0,0 +1,197 @@
[ ... ]
Quoted 30 lines
Quoted 30 lines
+static int mpam_fb_send_request(struct
mpam_pcc_chan*pcc_chan, u32msc_id,
+ u16 reg, u32 *result, intmpam_fb_command)drivers/resctrl/mpam_fb.c new file +14
-0+{ + unsigned int token = atomic_inc_return(&mpam_fb_token); + struct acpi_pcct_ext_pcc_shared_memory __iomem *pcc_shmem; + struct pcc_mbox_chan *chan; + void __iomem *payload_ofs; + u32 status; + int ret; + + if (!pcc_chan) + return -ENODEV; + + chan = pcc_chan->pcc_chan; + + /* prune token to fit into the 10 bits inside the commandregister */
+ token = FIELD_GET(MPAM_MSC_TOKEN_MASK,
+FIELD_PREP(MPAM_MSC_TOKEN_MASK, token));drivers/resctrl/mpam_fb.c new file +10
-0+ + guard(mutex)(&pcc_chan->pcc_chan_lock); + + switch (mpam_fb_command) { + case MPAM_MSC_WRITE_CMD: + mpam_fb_build_write_message(msc_id, reg, *result, + token, chan->shmem); + break; + case MPAM_MSC_READ_CMD: + mpam_fb_build_read_message(msc_id, reg, token, chan->shmem);
Quoted 8 lines
Quoted 5 lines
+ break;
+ caseMPAM_PROTOCOL_VERSION:
+mpam_fb_build_version_message(token, chan->shmem); + break;
+ }The switch here has no default case. If an unrecognised command value is passed (in future), it maybe safer and future proof to have a default case?
Well, mpam_fb_send_request is a static function, and all callers pass in constant values, so this wouldn't easily happen and be more a compile time error. But I can of course add a defensive default check.
Cheers,
Andre
Quoted 126 lines
default:
return-EINVAL; ?> +Quoted 116 lines
+ ret =
mbox_send_message(chan->mchan, NULL);
+ if (ret < 0)
+ return ret;drivers/resctrl/mpam_fb.c new file +30
-0+ + pcc_shmem = chan->shmem; + payload_ofs = chan->shmem + sizeof(*pcc_shmem); + status = readl(&pcc_shmem->command); + if (FIELD_GET(MPAM_MSC_TOKEN_MASK, status) != token) + return -ETIMEDOUT; + + ret = readl(payload_ofs + 0x0); + if (ret < 0) { + switch (ret) { + case MPAM_FB_ERR_NOT_SUPPORTED: + return -EOPNOTSUPP; + case MPAM_FB_ERR_INVALID_PARAMETERS: + return -EINVAL; + case MPAM_FB_ERR_NOT_FOUND: + return -ENOENT; + case MPAM_FB_ERR_OUT_OF_RANGE: + return -ERANGE; + default: + return -EINVAL; + } + } + + if (mpam_fb_command != MPAM_MSC_WRITE_CMD) + *result = readl(payload_ofs + 0x4); + + return 0; +} + +int mpam_fb_send_read_request(struct mpam_msc *msc, u16 reg, u32*result)
drivers/resctrl/mpam_fb.c new file +23
-0+{ + return mpam_fb_send_request(msc->pcc_chan, msc->mpam_fb_msc_id, + reg, result, MPAM_MSC_READ_CMD); +} + +int mpam_fb_send_write_request (struct mpam_msc *msc,u16 reg,u32 value) +{ + return mpam_fb_send_request(msc->pcc_chan, msc->mpam_fb_msc_id, + reg, &value, MPAM_MSC_WRITE_CMD); +} + +int mpam_fb_get_protocol_version (struct mpam_msc *msc) +{ + u32 version; + int ret; + + ret = mpam_fb_send_request(msc->pcc_chan, 0, + 0, &version, MPAM_PROTOCOL_VERSION); + if (ret) + return ret; + + return version; +}drivers/resctrl/mpam_internal.h +20
-0@@ -11,6 +11,7 @@ #include <linux/io.h> #include <linux/jump_label.h> #include <linux/llist.h> +#include <linux/mailbox_client.h> #include <linux/mutex.h> #include <linux/resctrl.h> #include <linux/spinlock.h>@@ -57,6 +58,15 @@ struct mpam_garbage { struct platform_device *pdev; }; +struct mpam_pcc_chan { + struct list_head pcc_chans; + struct mbox_client pcc_cl; + struct pcc_mbox_chan *pcc_chan; + struct mutex pcc_chan_lock; /* only one message at a time */ + struct kref refcount; + int subspace_id; +}; + struct mpam_msc { /* member of mpam_all_msc */ struct list_head all_msc_list;@@ -66,6 +76,8 @@ struct mpam_msc { /* Not modified after mpam_is_enabled() becomes true */ enum mpam_msc_iface iface; + struct mpam_pcc_chan *pcc_chan; + int mpam_fb_msc_id; /* in its own name space */ u32 nrdy_usec; cpumask_t accessibility; bool has_extd_esr;@@ -511,6 +523,14 @@ static inline void mpam_resctrl_offline_cpu(unsigned int cpu) { } static inline void mpam_resctrl_teardown_class(struct mpam_class *class) { } #endif /* CONFIG_RESCTRL_FS */ +/* MPAM-Fb Firmware-backed protocol wrappers */ +int mpam_fb_send_read_request (struct mpam_msc *msc,u16 reg,u32 *result); +int mpam_fb_send_write_request (struct mpam_msc *msc,u16 reg,u32 value); +int mpam_fb_get_protocol_version (struct mpam_msc *msc); + +#define MPAM_FB_PROT_HEADER_LEN sizeof(u32) +#define MPAM_FB_MAX_MSG_SIZE (MPAM_FB_PROT_HEADER_LEN + 4 * sizeof(u32)) +/*
* MPAM MSCs have the following register layout. See:
* Arm Memory System Resource Partitioning and Monitoring (MPAM) Systeminclude/linux/arm_mpam.h +0
-0@@ -12,7 +12,7 @@ struct mpam_msc; enum mpam_msc_iface {Quoted 5 lines
MPAM_IFACE_MMIO, /* a real MPAM MSC */
- MPAM_IFACE_PCC, /* a fake MPAM MSC */
+ MPAM_IFACE_PCC, /* using the MPAM-Fb firmware redirection */ };
enummpam_class_types{Best Regards,
Srivathsa
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH v4 08/10] arm_mpam: add MPAM-Fb MSC firmware access support
Hi Ritwick,
Quoted 7 lines · Ritwick Sharma
drivers/resctrl/mpam_devices.c static int mpam_msc_read_mbwu_l(struct mpam_msc *msc, u64 *res) · line 1146
Quoted 3 lines · Andre Przywara
drivers/resctrl/mpam_devices.c +2
-1-WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz); +if (msc->iface ==MPAM_IFACE_MMIO ) +WARN_ON_ONCE ((MSMON_MBWU_L +sizeof (u64)) > msc->mapped_hwpage_sz);I tested the MPAM-Fb series on an Arm AGI CPU.
Many thanks for that, and for reporting on the list!
Quoted 6 lines
I think a similar
MPAM_IFACE_MMIOcheck is needed inmpam_msc_zero_mbwu_l(). Without this check, themapped_hwpage_szbounds check is also performed for an MPAM-Fb MSC, which does not have an MMIO mapping. I observed the following warning during testing:[ 50.638246] WARNING: drivers/resctrl/mpam_devices.c:1266 at mpam_msc_zero_mbwu_l+0x98/0xa8
Ouch, you are right, I missed that one. It seems my very limited testing didn't touch that code path.
Quoted 9 lines
There also appears to be an off-by-one issue in the calculation of
mapped_hwpage_sz:msc->mapped_hwpage_sz = msc_res->end - msc_res->start;The current calculation records a size that is one byte smaller than the actual resource. I think this should use
resource_size()instead:msc->mapped_hwpage_sz = resource_size(msc_res);
Well spotted, I can make a separate patch for that, unless you beat me to it!
Many thanks,
Andre
Quoted 3 lines
Thanks,
Ritwick
PATCH v4 9/10 arm_mpam: prevent MPAM-Fb accesses inside IRQ handler
In reply to Andre PrzywaraWhen an MPAM MSC gets into an error condition, it can trigger an error IRQ. We cannot really do much about those errors, but we at least query and log the error, then disable MPAM functionality.
This error report relies on reading the MSC's error status register (ESR) in the IRQ handler, which is not possible for MPAM-Fb based MSC accesses, since they involve mailbox routines that might sleep. The same is true for clearing the interrupt at the source, which requires MSC access.
For simplicity just skip the ESR read when the MSC is not using direct MMIO accesses, and just ignore the pending interrupts. We will wrap up MPAM functionality regardless, knowing the exact error value will not change that.
--- drivers/resctrl/mpam_devices.c | 36 +++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 14 deletions(-)
drivers/resctrl/mpam_devices.c +22 -14
@@ -2650,25 +2650,33 @@ static irqreturn_t __mpam_irq_handler(int irq, struct mpam_msc *msc) &msc->accessibility)))return IRQ_NONE ; -mpam_msc_read_esr (msc, ®); +/* MPAM-Fb MSC accesses cannot be done in atomic context. */ +if (msc->iface ==MPAM_IFACE_MMIO ) { +mpam_msc_read_esr (msc, ®); - errcode =FIELD_GET (MPAMF_ESR_ERRCODE , reg); -if (!errcode) -return IRQ_NONE ; + errcode =FIELD_GET (MPAMF_ESR_ERRCODE , reg); +if (!errcode) +return IRQ_NONE ; -/* Clear level triggered irq */ -mpam_msc_clear_esr (msc); +/* Clear level triggered irq */ +mpam_msc_clear_esr (msc); - partid =FIELD_GET (MPAMF_ESR_PARTID_MON , reg); - pmg =FIELD_GET (MPAMF_ESR_PMG , reg); - ris =FIELD_GET (MPAMF_ESR_RIS , reg); + partid =FIELD_GET (MPAMF_ESR_PARTID_MON , reg); + pmg =FIELD_GET (MPAMF_ESR_PMG , reg); + ris =FIELD_GET (MPAMF_ESR_RIS , reg); -pr_err_ratelimited ("error irq from msc:%u '%s', partid:%u, pmg: %u, ris: %u\n" , - msc->id, mpam_errcode_names[errcode], partid, pmg, - ris); +pr_err_ratelimited ("error irq from msc:%u '%s', partid:%u, pmg: %u, ris: %u\n" , + msc->id, mpam_errcode_names[errcode], partid, + pmg, ris); -/* Disable this interrupt. */ -mpam_disable_msc_ecr (msc); +/* Disable this interrupt. */ +mpam_disable_msc_ecr (msc); + }else { +struct irq_data *d =irq_get_irq_data (irq); + +pr_err_ratelimited ("unknown error irq %d/%ld from msc:%u\n" , + irq, d ?irqd_to_hwirq (d) :-1 , msc->id); + }/* Are we racing with the thread disabling MPAM? */ if (!mpam_is_enabled ())
Signature · 1 line
2.43.0
Ben Horgan
maintainerRe: [PATCH v4 09/10] arm_mpam: prevent MPAM-Fb accesses inside IRQ handler
Hi Andre,
Quoted 53 lines · Andre Przywara
When an MPAM MSC gets into an error condition, it can trigger an error IRQ. We cannot really do much about those errors, but we at least query and log the error, then disable MPAM functionality.
This error report relies on reading the MSC's error status register (ESR) in the IRQ handler, which is not possible for MPAM-Fb based MSC accesses, since they involve mailbox routines that might sleep. The same is true for clearing the interrupt at the source, which requires MSC access.
For simplicity just skip the ESR read when the MSC is not using direct MMIO accesses, and just ignore the pending interrupts. We will wrap up MPAM functionality regardless, knowing the exact error value will not change that.
Signed-off-by Andre Przywara <[email protected]>--- drivers/resctrl/mpam_devices.c | 36 +++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 14 deletions(-)drivers/resctrl/mpam_devices.c +11
-10@@ -2650,25 +2650,33 @@ static irqreturn_t __mpam_irq_handler(int irq, struct mpam_msc *msc) &msc->accessibility)))return IRQ_NONE ; -mpam_msc_read_esr (msc, ®); +/* MPAM-Fb MSC accesses cannot be done in atomic context. */ +if (msc->iface ==MPAM_IFACE_MMIO ) { +mpam_msc_read_esr (msc, ®); - errcode =FIELD_GET (MPAMF_ESR_ERRCODE , reg); -if (!errcode) -return IRQ_NONE ; + errcode =FIELD_GET (MPAMF_ESR_ERRCODE , reg); +if (!errcode) +return IRQ_NONE ; -/* Clear level triggered irq */ -mpam_msc_clear_esr (msc); +/* Clear level triggered irq */ +mpam_msc_clear_esr (msc); - partid =FIELD_GET (MPAMF_ESR_PARTID_MON , reg); - pmg =FIELD_GET (MPAMF_ESR_PMG , reg); - ris =FIELD_GET (MPAMF_ESR_RIS , reg); + partid =FIELD_GET (MPAMF_ESR_PARTID_MON , reg); + pmg =FIELD_GET (MPAMF_ESR_PMG , reg); + ris =FIELD_GET (MPAMF_ESR_RIS , reg); - pr_err_ratelimited("error irq from msc:%u '%s', partid:%u, pmg: %u, ris: %u\n",
drivers/resctrl/mpam_devices.c static irqreturn_t __mpam_irq_handler(int irq, struct mpam_msc *msc) · line 2673
Quoted 10 lines
drivers/resctrl/mpam_devices.c +5
-4- msc->id, mpam_errcode_names[errcode], partid, pmg, - ris); +pr_err_ratelimited ("error irq from msc:%u '%s', partid:%u, pmg: %u, ris: %u\n" , + msc->id, mpam_errcode_names[errcode], partid, + pmg, ris); -/* Disable this interrupt. */ -mpam_disable_msc_ecr (msc); +/* Disable this interrupt. */ +mpam_disable_msc_ecr (msc);
I still think we should use a threaded interrupt with IRQF_ONESHOT and disable the interrupt and schedule the mpam_disable from there. That way it can be obvious that won't just keep on running the interrupt handler and never getting to run the scheduled work.
Thanks,
Ben
Quoted 9 lines
drivers/resctrl/mpam_devices.c +6
-0+ }else { +struct irq_data *d =irq_get_irq_data (irq); + +pr_err_ratelimited ("unknown error irq %d/%ld from msc:%u\n" , + irq, d ?irqd_to_hwirq (d) :-1 , msc->id); + }/* Are we racing with the thread disabling MPAM? */ if (!mpam_is_enabled ())
PATCH v4 10/10 arm_mpam: detect and enable MPAM-Fb PCC support
In reply to Andre PrzywaraThe Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.
[1] https://developer.arm.com/documentation/den0144/latest
--- drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/mpam_devices.c | 127 ++++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-)
drivers/acpi/arm64/mpam.c +4 -2
@@ -220,8 +220,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ struct platform_device *pdev __free(platform_device_put) =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);int next_res =0 , next_prop =0 , err; -/* pcc, nrdy, affinity and a sentinel */ -struct property_entry props[4 ] = {0 }; +/* pcc, msc-id, nrdy, affinity and a sentinel */ +struct property_entry props[5 ] = {0 };/* mmio, 2xirq, no sentinel. */ struct resource res[3 ] = {0 };struct acpi_device *companion;
@@ -256,6 +256,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ }else if (iface ==MPAM_IFACE_PCC ) { props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32 ("msc-id" , + tbl_msc->identifier); }acpi_mpam_parse_irqs (pdev, tbl_msc, res, &next_res);
drivers/resctrl/mpam_devices.c +125 -2
171 lines not on this page · raw message
Signature · 1 line
2.43.0
Sudeep Holla
maintainerRe: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
Quoted 26 lines · Andre Przywara
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/mpam_devices.c | 127 ++++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-)drivers/acpi/arm64/mpam.c +0
-0
drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 221
Quoted 2 lines
drivers/acpi/arm64/mpam.c +0
-0@@ -220,8 +220,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ struct platform_device *pdev __free(platform_device_put) =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);
Looks like tbl_msc->identifier is getting stashed as pdev->id...
Quoted 9 lines
drivers/acpi/arm64/mpam.c +2
-2int next_res =0 , next_prop =0 , err; -/* pcc, nrdy, affinity and a sentinel */ -struct property_entry props[4 ] = {0 }; +/* pcc, msc-id, nrdy, affinity and a sentinel */ +struct property_entry props[5 ] = {0 };/* mmio, 2xirq, no sentinel. */ struct resource res[3 ] = {0 };struct acpi_device *companion;
drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 260
Quoted 5 lines
drivers/acpi/arm64/mpam.c +2
-0@@ -256,6 +256,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ }else if (iface ==MPAM_IFACE_PCC ) { props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32 ("msc-id" , + tbl_msc->identifier);
Why is this needed then as you can fetch it as pdev->id ?
Signature · 2 lines
Regards, Sudeep
Lorenzo Pieralisi
maintainerRe: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
Quoted 33 lines · Sudeep Holla
Quoted 26 lines · Andre Przywara
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/mpam_devices.c | 127 ++++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-)drivers/acpi/arm64/mpam.c +0
-0drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 221
Quoted 2 lines
drivers/acpi/arm64/mpam.c +0
-0@@ -220,8 +220,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ struct platform_device *pdev __free(platform_device_put) =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);Looks like
tbl_msc->identifieris getting stashed aspdev->id...
Quoted 16 lines
Quoted 9 lines
drivers/acpi/arm64/mpam.c +2
-2int next_res =0 , next_prop =0 , err; -/* pcc, nrdy, affinity and a sentinel */ -struct property_entry props[4 ] = {0 }; +/* pcc, msc-id, nrdy, affinity and a sentinel */ +struct property_entry props[5 ] = {0 };/* mmio, 2xirq, no sentinel. */ struct resource res[3 ] = {0 };struct acpi_device *companion;drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 260
Quoted 5 lines
drivers/acpi/arm64/mpam.c +2
-0@@ -256,6 +256,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ }else if (iface ==MPAM_IFACE_PCC ) { props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32 ("msc-id" , + tbl_msc->identifier);Why is this needed then as you can fetch it as
pdev->id?
I suppose it is to have properties parity between OF and ACPI without having to change the driver property parsing path.
Lorenzo
Re: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
Quoted 51 lines · Lorenzo Pieralisi
Quoted 33 lines · Sudeep Holla
Quoted 26 lines · Andre Przywara
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/mpam_devices.c | 127 ++++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-)drivers/acpi/arm64/mpam.c +0
-0drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 221
Quoted 2 lines
drivers/acpi/arm64/mpam.c +0
-0@@ -220,8 +220,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ struct platform_device *pdev __free(platform_device_put) =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);Looks like
tbl_msc->identifieris getting stashed aspdev->id...Quoted 16 lines
Quoted 9 lines
drivers/acpi/arm64/mpam.c +2
-2int next_res =0 , next_prop =0 , err; -/* pcc, nrdy, affinity and a sentinel */ -struct property_entry props[4 ] = {0 }; +/* pcc, msc-id, nrdy, affinity and a sentinel */ +struct property_entry props[5 ] = {0 };/* mmio, 2xirq, no sentinel. */ struct resource res[3 ] = {0 };struct acpi_device *companion;drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 260
Quoted 5 lines
drivers/acpi/arm64/mpam.c +2
-0@@ -256,6 +256,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ }else if (iface ==MPAM_IFACE_PCC ) { props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32 ("msc-id" , + tbl_msc->identifier);Why is this needed then as you can fetch it as
pdev->id?
Quoted 3 lines
I suppose it is to have properties parity between OF and ACPI without having to change the driver property parsing path.
Yes, that was my initial thought as well. However, after looking into it further, I could not find any device tree bindings in the upstream tree. I am therefore unsure whether MPAM is supported on DT platforms.
The use of these device properties appears to have been based on DT bindings that may never have been upstreamed. As [1] suggests, the bindings were dropped before the initial merge.
Although we could rename these properties later to align with any DT bindings that are introduced, I think it would be preferable to avoid adding new properties unless they are strictly necessary.
Signature · 4 lines
Regards, Sudeep [1] https://lore.kernel.org/all/[email protected]/
Lorenzo Pieralisi
maintainerRe: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
Quoted 60 lines · Sudeep Holla
Quoted 51 lines · Lorenzo Pieralisi
Quoted 33 lines · Sudeep Holla
Quoted 26 lines · Andre Przywara
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/mpam_devices.c | 127 ++++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-)drivers/acpi/arm64/mpam.c +0
-0drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 221
Quoted 2 lines
drivers/acpi/arm64/mpam.c +0
-0@@ -220,8 +220,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ struct platform_device *pdev __free(platform_device_put) =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);Looks like
tbl_msc->identifieris getting stashed aspdev->id...Quoted 16 lines
Quoted 9 lines
drivers/acpi/arm64/mpam.c +2
-2int next_res =0 , next_prop =0 , err; -/* pcc, nrdy, affinity and a sentinel */ -struct property_entry props[4 ] = {0 }; +/* pcc, msc-id, nrdy, affinity and a sentinel */ +struct property_entry props[5 ] = {0 };/* mmio, 2xirq, no sentinel. */ struct resource res[3 ] = {0 };struct acpi_device *companion;drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 260
Quoted 5 lines
drivers/acpi/arm64/mpam.c +2
-0@@ -256,6 +256,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ }else if (iface ==MPAM_IFACE_PCC ) { props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32 ("msc-id" , + tbl_msc->identifier);Why is this needed then as you can fetch it as
pdev->id?Quoted 3 lines
I suppose it is to have properties parity between OF and ACPI without having to change the driver property parsing path.
Yes, that was my initial thought as well. However, after looking into it further, I could not find any device tree bindings in the upstream tree. I am therefore unsure whether MPAM is supported on DT platforms.
Quoted 7 lines
The use of these device properties appears to have been based on DT bindings that may never have been upstreamed. As [1] suggests, the bindings were dropped before the initial merge.
Although we could rename these properties later to align with any DT bindings that are introduced, I think it would be preferable to avoid adding new properties unless they are strictly necessary.
Ah. That makes sense, if DT bindings get published the driver can be reworked accordingly (if we want to improve how the code looks), this is just Linux kernel internals anyway.
As it stands the "msc-id" property is churn, that's what you are saying.
Lorenzo
Quoted 5 lines
Signature · 4 lines
Regards, Sudeep [1] https://lore.kernel.org/all/[email protected]/
Sudeep Holla
maintainerRe: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
Quoted 2 lines · Lorenzo Pieralisi
Quoted 1 line · Sudeep Holla
On Mon, Jul 27, 2026 at 02:48:47PM +0200, Lorenzo Pieralisi wrote:
[...]
Quoted 17 lines
Quoted 9 lines
Quoted 4 lines
I suppose it is to have properties parity between OF and ACPI without having to change the driver property parsing path.
Yes, that was my initial thought as well. However, after looking into it further, I could not find any device tree bindings in the upstream tree. I am therefore unsure whether MPAM is supported on DT platforms.
Quoted 7 lines
The use of these device properties appears to have been based on DT bindings that may never have been upstreamed. As [1] suggests, the bindings were dropped before the initial merge.
Although we could rename these properties later to align with any DT bindings that are introduced, I think it would be preferable to avoid adding new properties unless they are strictly necessary.
Quoted 6 lines
Ah. That makes sense, if DT bindings get published the driver can be reworked accordingly (if we want to improve how the code looks), this is just Linux kernel internals anyway.
As it stands the "msc-id" property is churn, that's what you are saying.
Indeed why add more property with names that may need to be changed with DT bindings in the future especially if we don't need it(IIUC it is already there in pdev->id unless I am missing something.
Signature · 2 lines
Regards, Sudeep
Quoted 2 lines
Quoted 2 lines
Re: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
In reply to Sudeep HollaHi,
Quoted 52 lines · Sudeep Holla
Quoted 51 lines · Lorenzo Pieralisi
Quoted 49 lines · Sudeep Holla
Quoted 26 lines · Andre Przywara
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/mpam_devices.c | 127 ++++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-)drivers/acpi/arm64/mpam.c +0
-0drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 221
Quoted 2 lines
drivers/acpi/arm64/mpam.c +0
-0@@ -220,8 +220,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ struct platform_device *pdev __free(platform_device_put) =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);Looks like
tbl_msc->identifieris getting stashed aspdev->id...Quoted 9 lines
drivers/acpi/arm64/mpam.c +2
-2int next_res =0 , next_prop =0 , err; -/* pcc, nrdy, affinity and a sentinel */ -struct property_entry props[4 ] = {0 }; +/* pcc, msc-id, nrdy, affinity and a sentinel */ +struct property_entry props[5 ] = {0 };/* mmio, 2xirq, no sentinel. */ struct resource res[3 ] = {0 };struct acpi_device *companion;drivers/acpi/arm64/mpam.c static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ · line 260
Quoted 5 lines
drivers/acpi/arm64/mpam.c +2
-0@@ -256,6 +256,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ }else if (iface ==MPAM_IFACE_PCC ) { props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32 ("msc-id" , + tbl_msc->identifier);Why is this needed then as you can fetch it as
pdev->id?
Quoted 2 lines
Quoted 2 lines
I suppose it is to have properties parity between OF and ACPI without having to change the driver property parsing path.
I could claim: "what he said ^^^^", but in truth I simply missed that the identifier is also the platform device ID.
Quoted 3 lines
Yes, that was my initial thought as well. However, after looking into it further, I could not find any device tree bindings in the upstream tree. I am therefore unsure whether MPAM is supported on DT platforms.
No, at the moment MPAM is ACPI only. As you mention, DT bindings have been proposed at some point, but apparently there was no real user, so this effort was shelved until someone complains, IIUC.
Quoted 3 lines
The use of these device properties appears to have been based on DT bindings that may never have been upstreamed. As [1] suggests, the bindings were dropped before the initial merge.
Yes, indeed these names already cater for the proposed DT bindings, short of inventing some new names, I guess. But I was wondering how much of a problem it is to rename them, for ACPI those names are internal only, right? So any potentially deviating DT property name would just be a patch to rename the property name here and in the MPAM driver, right?
Quoted 3 lines
Although we could rename these properties later to align with any DT bindings that are introduced, I think it would be preferable to avoid adding new properties unless they are strictly necessary.
I don't really mind, so can do either way, but the MSC identifier is clearly defined in the MPAM-Fb protocol, and connected to the number used in the ACPI table, so kind of assuming that the platform device ID (a more Linux internal thing?) is the same, is a bit fragile.
But as mentioned: whatever you prefer, it looks like we can change it anytime.
Thanks,
Andre.
Re: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
In reply to Andre PrzywaraQuoted 115 lines · Andre Przywara
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/mpam_devices.c | 127 ++++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-)drivers/acpi/arm64/mpam.c +4
-2@@ -220,8 +220,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ struct platform_device *pdev __free(platform_device_put) =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);int next_res =0 , next_prop =0 , err; -/* pcc, nrdy, affinity and a sentinel */ -struct property_entry props[4 ] = {0 }; +/* pcc, msc-id, nrdy, affinity and a sentinel */ +struct property_entry props[5 ] = {0 };/* mmio, 2xirq, no sentinel. */ struct resource res[3 ] = {0 };struct acpi_device *companion;@@ -256,6 +256,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ }else if (iface ==MPAM_IFACE_PCC ) { props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32 ("msc-id" , + tbl_msc->identifier); }acpi_mpam_parse_irqs (pdev, tbl_msc, res, &next_res);drivers/resctrl/mpam_devices.c +47
-0@@ -19,14 +19,19 @@#include <linux/irqdesc.h> #include <linux/list.h> #include <linux/lockdep.h> +#include <linux/mailbox_client.h> #include <linux/mutex.h> #include <linux/platform_device.h> #include <linux/printk.h> +#include <linux/property.h> #include <linux/srcu.h> #include <linux/spinlock.h> #include <linux/types.h> #include <linux/workqueue.h> +#include <acpi/pcc.h> +#include <acpi/acpi_io.h> +#include "mpam_internal.h" /* Values for the T241 errata workaround */ @@ -49,6 +54,86 @@ static LIST_HEAD(mpam_all_msc);struct srcu_struct mpam_srcu; +/* PCC channels might be serving multiple MSCs, so keep a refcounted list. */ +static DEFINE_MUTEX (pcc_chan_list_lock); +static LIST_HEAD (pcc_chan_list); + +static void mpam_pcc_chan_release (struct kref *ref) +{ + struct mpam_pcc_chan *cur = container_of(ref, struct mpam_pcc_chan, + refcount); + +pcc_mbox_free_channel (cur->pcc_chan); +list_del (&cur->pcc_chans); +kfree (cur); +} + +static struct mpam_pcc_chan *mpam_pcc_chan_get (struct device *dev, +int subspace_id) +{ +struct mpam_pcc_chan *cur; +int ret; + +guard (mutex)(&pcc_chan_list_lock); + +list_for_each_entry (cur, &pcc_chan_list, pcc_chans) { +if (cur->subspace_id == subspace_id) { +kref_get (&cur->refcount); + +return cur; + } + } + + cur =kzalloc_obj (*cur); +if (!cur) +return ERR_PTR (-ENOMEM ); + + cur->pcc_cl.dev = dev; + cur->pcc_cl.tx_block = true; + + cur->pcc_chan =pcc_mbox_request_channel (&cur->pcc_cl, subspace_id); +if (IS_ERR (cur->pcc_chan)) { +long err =PTR_ERR (cur->pcc_chan); + +kfree (cur);
drivers/resctrl/mpam_devices.c static LIST_HEAD(mpam_all_msc); · line 108
Quoted 10 lines
drivers/resctrl/mpam_devices.c +10
-0+return ERR_PTR (err); + } + +/* Timeout based on the "nominal latency" from the PCC ACPI table. */ + cur->pcc_cl.tx_tout = cur->pcc_chan->latency *5 ; + + ret =devm_mutex_init (dev, &cur->pcc_chan_lock); +if (ret) +return ERR_PTR (ret); +
Here, devm_mutex_init() ties the lifetime of pcc_chan_lock to dev, which is &pdev->dev of the first MSC to call mpam_pcc_chan_get(). But mpam_pcc_chan is shared across multiple MSCs via kref, so it can
outlive that first device.
If a second MSC has incremented the refcount and is still active when the first MSC's device is removed, the devm cleanup on the first device would call mutex_destroy() on pcc_chan_lock while the second MSC may still be inside:
guard(mutex)(&pcc_chan->pcc_chan_lock);
I am not 100% sure this is reachable in practice — it may depend on the order in which platform devices are unbound, and whether that can happen with MSCs sharing a channel. I also note that devm_mutex_init() is a no-op on non-debug kernels, so this would only be observable with CONFIG_DEBUG_MUTEXES=y. But it felt worth raising in case it is a real path.
If my understanding is correct, would it make sense to use a plain mutex_init() here instead, and move mutex_destroy() into mpam_pcc_chan_release() where the kref guarantees no concurrent users remain?
Quoted 91 lines
drivers/resctrl/mpam_devices.c +61
-2+ cur->subspace_id = subspace_id; +kref_init (&cur->refcount); + +list_add_tail (&cur->pcc_chans, &pcc_chan_list); + +return cur; +} + +static int mpam_pcc_chan_put (struct mpam_pcc_chan *pcc_chan) +{ +struct mpam_pcc_chan *cur, *tmp; + +if (!pcc_chan) +return 0 ; + +guard (mutex)(&pcc_chan_list_lock); + +list_for_each_entry_safe (cur, tmp, &pcc_chan_list, pcc_chans) { +if (cur == pcc_chan) { +kref_put (&cur->refcount, mpam_pcc_chan_release); + +return 0 ; + } + } + +return -ENOENT ; +} +/* * Number of MSCs that have been probed. Once all MSCs have been probed MPAM * can be enabled. @@ -2201,6 +2286,8 @@ static void mpam_msc_drv_remove(struct platform_device *pdev) {struct mpam_msc *msc =platform_get_drvdata (pdev); +mpam_pcc_chan_put (msc->pcc_chan); +mutex_lock (&mpam_list_lock);mpam_msc_destroy (msc);mutex_unlock (&mpam_list_lock);@@ -2211,7 +2298,7 @@ static void mpam_msc_drv_remove(struct platform_device *pdev)static struct mpam_msc *do_mpam_msc_drv_probe (struct platform_device *pdev) {int err; -u32 tmp; +u32 pcc_subspace_id;struct mpam_msc *msc;struct resource *msc_res;struct device *dev = &pdev->dev;@@ -2259,7 +2346,8 @@ static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev)if (err)return ERR_PTR (err); -if (device_property_read_u32 (&pdev->dev,"pcc-channel" , &tmp)) +if (device_property_read_u32 (&pdev->dev,"pcc-channel" , + &pcc_subspace_id)) msc->iface =MPAM_IFACE_MMIO ;else msc->iface =MPAM_IFACE_PCC ;@@ -2275,6 +2363,41 @@ static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) } msc->mapped_hwpage_sz = msc_res->end - msc_res->start; msc->mapped_hwpage = io; + }else if (msc->iface ==MPAM_IFACE_PCC ) { +u32 msc_id; +int ret; + +if (device_property_read_u32 (&pdev->dev,"msc-id" , &msc_id)) { +pr_err ("missing MPAM-Fb MSC identifier\n" ); +return ERR_PTR (-EINVAL ); + } + msc->mpam_fb_msc_id = msc_id; + + msc->pcc_chan =mpam_pcc_chan_get (&pdev->dev, pcc_subspace_id); +if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); +return ERR_CAST (msc->pcc_chan); + } + +if (msc->pcc_chan->pcc_chan->shmem_size <MPAM_FB_MAX_MSG_SIZE ) { +pr_err ("MPAM-Fb PCC channel size too small.\n" ); +mpam_pcc_chan_put (msc->pcc_chan); +return ERR_PTR (-ENOMEM ); + } + ret =mpam_fb_get_protocol_version (msc); +if (ret <0 ) { +pr_err ("Cannot query MPAM-Fb protocol version.\n" ); +mpam_pcc_chan_put (msc->pcc_chan); + +/* ret is an MPAM-Fb error code, return a Linux one. */ +return ERR_PTR (-EIO );
drivers/resctrl/mpam_devices.c static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) · line 2403
Quoted 10 lines
drivers/resctrl/mpam_devices.c +7
-0+ } +if ((ret >>16 ) !=1 ) { +pr_err ("Incompatible MPAM-Fb protocol version %d.%d\n" , + ret >>16 , ret &0xffff ); +mpam_pcc_chan_put (msc->pcc_chan); +return ERR_PTR (-EINVAL ); + } }else {return ERR_PTR (-EINVAL ); }
Best Regards,
Srivathsa
Re: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
Quoted 7 lines · On Mon, 27 Jul 2026 21:57:15 +0530 Srivathsa L Rao…
Quoted 6 lines · Andre Przywara
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Quoted 5 lines
Quoted 5 lines
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
Blank line for consistency
Quoted 79 lines
Quoted 49 lines
This will be used by the MPAM-Fb access wrapper code.
[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>drivers/resctrl/mpam_devices.c +38
-0+static void mpam_pcc_chan_release (struct kref *ref) +{ + struct mpam_pcc_chan *cur = container_of(ref, struct mpam_pcc_chan, + refcount); + +pcc_mbox_free_channel (cur->pcc_chan); +list_del (&cur->pcc_chans); +kfree (cur); +} + +static struct mpam_pcc_chan *mpam_pcc_chan_get (struct device *dev, +int subspace_id) +{ +struct mpam_pcc_chan *cur; +int ret; + +guard (mutex)(&pcc_chan_list_lock); + +list_for_each_entry (cur, &pcc_chan_list, pcc_chans) { +if (cur->subspace_id == subspace_id) { +kref_get (&cur->refcount); + +return cur; + } + } + + cur =kzalloc_obj (*cur); +if (!cur) +return ERR_PTR (-ENOMEM ); + + cur->pcc_cl.dev = dev; + cur->pcc_cl.tx_block = true; + + cur->pcc_chan =pcc_mbox_request_channel (&cur->pcc_cl, subspace_id); +if (IS_ERR (cur->pcc_chan)) { +long err =PTR_ERR (cur->pcc_chan); + +kfree (cur);drivers/resctrl/mpam_devices.c static LIST_HEAD(mpam_all_msc); · line 108
Quoted 10 lines
drivers/resctrl/mpam_devices.c +10
-0+return ERR_PTR (err); + } + +/* Timeout based on the "nominal latency" from the PCC ACPI table. */ + cur->pcc_cl.tx_tout = cur->pcc_chan->latency *5 ; + + ret =devm_mutex_init (dev, &cur->pcc_chan_lock); +if (ret) +return ERR_PTR (ret); +Here,
devm_mutex_init()ties the lifetime ofpcc_chan_lockto dev, which is&pdev->devof the first MSC to callmpam_pcc_chan_get(). Butmpam_pcc_chanis shared across multiple MSCs via kref, so it can outlive that first device.If a second MSC has incremented the refcount and is still active when the first MSC's device is removed, the devm cleanup on the first device would call
mutex_destroy()onpcc_chan_lockwhile the second MSC may still be inside:guard(mutex)(&pcc_chan->pcc_chan_lock);I am not 100% sure this is reachable in practice — it may depend on the order in which platform devices are unbound, and whether that can happen with MSCs sharing a channel. I also note that
devm_mutex_init()is a no-op on non-debug kernels, so this would only be observable withCONFIG_DEBUG_MUTEXES=y. But it felt worth raising in case it is a real path.
Quoted 4 lines
If my understanding is correct, would it make sense to use a plain
mutex_init()here instead, and movemutex_destroy()intompam_pcc_chan_release()where the kref guarantees no concurrent users remain?
Good spot and fwiw Sashiko shouted about this one as well!
https://sashiko.dev/#/patchset/20260723155454.1760823-1-andre.przywara%40arm.com
It also called out the lack of error handling should this fail, but that becomes irrelevant if the code is updated as you suggest.
Quoted 9 lines
drivers/resctrl/mpam_devices.c static LIST_HEAD(mpam_all_msc); · line 116
Quoted 9 lines
drivers/resctrl/mpam_devices.c +8
-0+ cur->subspace_id = subspace_id; +kref_init (&cur->refcount); + +list_add_tail (&cur->pcc_chans, &pcc_chan_list); + +return cur; +} +
Thanks,
Jonathan
Re: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
In reply to Srivathsa L RaoHi Srivathsa,
Quoted 123 lines · Srivathsa L Rao
Quoted 120 lines · Andre Przywara
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>---
drivers/acpi/arm64/mpam.c | 6 +-drivers/resctrl/mpam_devices.c| 127 ++++++++++++++++++++++++++++++++-2 files changed, 129 insertions(+), 4 deletions(-)drivers/acpi/arm64/mpam.c +0
-0@@ -220,8 +220,8 @@ static struct platform_device * __initacpi_mpam_parse_msc(struct
acpi_mpam_msc_
structplatform_device*pdev__free(platform_device_put)=platform_device_alloc("mpam_msc", tbl_msc->identifier); intnext_res= 0,next_prop= 0, err;
- /* pcc, nrdy, affinity and a sentinel */
- structproperty_entryprops[4] = { 0 };
+ /* pcc, msc-id, nrdy, affinity and a sentinel */
+ structproperty_entryprops[5] = { 0 };
/* mmio, 2xirq, no sentinel. */
struct resource res[3] = { 0 };
structacpi_device*companion;drivers/acpi/arm64/mpam.c +0
-0@@ -256,6 +256,8 @@ static struct platform_device * __initacpi_mpam_parse_msc(struct
acpi_mpam_msc_
} else if (iface ==MPAM_IFACE_PCC) { props[next_prop++] =PROPERTY_ENTRY_U32("pcc-channel", tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32("msc-id", + tbl_msc->identifier);
}
acpi_mpam_parse_irqs(pdev, tbl_msc, res, &next_res);drivers/resctrl/mpam_devices.c +6
-0@@ -19,14 +19,19 @@ #include <linux/irqdesc.h> #include <linux/list.h> #include <linux/lockdep.h> +#include <linux/mailbox_client.h> #include <linux/mutex.h> #include <linux/platform_device.h> #include <linux/printk.h> +#include <linux/property.h> #include <linux/srcu.h> #include <linux/spinlock.h> #include <linux/types.h> #include <linux/workqueue.h> +#include <acpi/pcc.h> +#include <acpi/acpi_io.h> + #include "mpam_internal.h" /* Values for the T241 errata workaround */@@ -49,6 +54,86 @@ static LIST_HEAD(mpam_all_msc); struct srcu_struct mpam_srcu; +/* PCC channels might be serving multiple MSCs, so keep a refcounted list. */
drivers/resctrl/mpam_devices.c +45
-0+static DEFINE_MUTEX (pcc_chan_list_lock); +static LIST_HEAD (pcc_chan_list); + +static void mpam_pcc_chan_release (struct kref *ref) +{ + struct mpam_pcc_chan *cur = container_of(ref, struct mpam_pcc_chan, + refcount); + + pcc_mbox_free_channel(cur->pcc_chan); + list_del(&cur->pcc_chans); + kfree(cur); +} + +static struct mpam_pcc_chan *mpam_pcc_chan_get (struct device *dev, + int subspace_id) +{ + struct mpam_pcc_chan *cur; + int ret; + + guard(mutex)(&pcc_chan_list_lock); + + list_for_each_entry(cur, &pcc_chan_list, pcc_chans) { + if (cur->subspace_id == subspace_id) { + kref_get(&cur->refcount); + + return cur; + } + } + + cur = kzalloc_obj(*cur); + if (!cur) + return ERR_PTR(-ENOMEM); + + cur->pcc_cl.dev = dev; + cur->pcc_cl.tx_block = true; + + cur->pcc_chan = pcc_mbox_request_channel(&cur->pcc_cl, subspace_id); + if (IS_ERR(cur->pcc_chan)) { + long err = PTR_ERR(cur->pcc_chan); + + kfree(cur); + return ERR_PTR(err); + } + + /* Timeout based on the "nominal latency" from the PCC ACPItable. */
+cur->pcc_cl.tx_tout=cur->pcc_chan->latency* 5; +
Quoted 8 lines
Quoted 4 lines
+ ret =
devm_mutex_init(dev, &cur->pcc_chan_lock); + if (ret)
+ returnERR_PTR(ret);
+Here,
devm_mutex_init()ties the lifetime ofpcc_chan_lockto dev, which is&pdev->devof the first MSC to callmpam_pcc_chan_get(). Butmpam_pcc_chanis shared across multiple MSCs via kref, so it can
outlive that first device.
Ah, that's a good point, thanks for catching that. It was too tempting to just replace the original mutex_init() with the devm_ version, but indeed the lifetime is not the same.
Quoted 14 lines
If a second MSC has incremented the refcount and is still active when the first MSC's device is removed, the devm cleanup on the first device would call
mutex_destroy()onpcc_chan_lockwhile the second MSC may still be inside:guard(mutex)(&pcc_chan->pcc_chan_lock);
I am not 100% sure this is reachable in practice — it may depend on the order in which platform devices are unbound, and whether that can happen with MSCs sharing a channel. I also note that
devm_mutex_init()is a no-op on non-debug kernels, so this would only be observable withCONFIG_DEBUG_MUTEXES=y. But it felt worth raising in case it is a real path.
Quoted 4 lines
If my understanding is correct, would it make sense to use a plain
mutex_init()here instead, and movemutex_destroy()intompam_pcc_chan_release()where the kref guarantees no concurrent users remain?
Yes, that's a good idea, since we free the data structure in the release function anyway, we can surely destroy the mutex there. And thanks for the heads up about this being a no-op mostly!
Cheers,
Andre
Ben Horgan
maintainerRe: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
In reply to Andre PrzywaraHi Andre,
Just a couple of nits I spotted just now.
Quoted 185 lines · Andre Przywara
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/mpam_devices.c | 127 ++++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-)drivers/acpi/arm64/mpam.c +4
-2@@ -220,8 +220,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ struct platform_device *pdev __free(platform_device_put) =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);int next_res =0 , next_prop =0 , err; -/* pcc, nrdy, affinity and a sentinel */ -struct property_entry props[4 ] = {0 }; +/* pcc, msc-id, nrdy, affinity and a sentinel */ +struct property_entry props[5 ] = {0 };/* mmio, 2xirq, no sentinel. */ struct resource res[3 ] = {0 };struct acpi_device *companion;@@ -256,6 +256,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ }else if (iface ==MPAM_IFACE_PCC ) { props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32 ("msc-id" , + tbl_msc->identifier); }acpi_mpam_parse_irqs (pdev, tbl_msc, res, &next_res);drivers/resctrl/mpam_devices.c +90
-2@@ -19,14 +19,19 @@#include <linux/irqdesc.h> #include <linux/list.h> #include <linux/lockdep.h> +#include <linux/mailbox_client.h> #include <linux/mutex.h> #include <linux/platform_device.h> #include <linux/printk.h> +#include <linux/property.h> #include <linux/srcu.h> #include <linux/spinlock.h> #include <linux/types.h> #include <linux/workqueue.h> +#include <acpi/pcc.h> +#include <acpi/acpi_io.h> +#include "mpam_internal.h" /* Values for the T241 errata workaround */ @@ -49,6 +54,86 @@ static LIST_HEAD(mpam_all_msc);struct srcu_struct mpam_srcu; +/* PCC channels might be serving multiple MSCs, so keep a refcounted list. */ +static DEFINE_MUTEX (pcc_chan_list_lock); +static LIST_HEAD (pcc_chan_list); + +static void mpam_pcc_chan_release (struct kref *ref) +{ + struct mpam_pcc_chan *cur = container_of(ref, struct mpam_pcc_chan, + refcount); + +pcc_mbox_free_channel (cur->pcc_chan); +list_del (&cur->pcc_chans); +kfree (cur); +} + +static struct mpam_pcc_chan *mpam_pcc_chan_get (struct device *dev, +int subspace_id) +{ +struct mpam_pcc_chan *cur; +int ret; + +guard (mutex)(&pcc_chan_list_lock); + +list_for_each_entry (cur, &pcc_chan_list, pcc_chans) { +if (cur->subspace_id == subspace_id) { +kref_get (&cur->refcount); + +return cur; + } + } + + cur =kzalloc_obj (*cur); +if (!cur) +return ERR_PTR (-ENOMEM ); + + cur->pcc_cl.dev = dev; + cur->pcc_cl.tx_block = true; + + cur->pcc_chan =pcc_mbox_request_channel (&cur->pcc_cl, subspace_id); +if (IS_ERR (cur->pcc_chan)) { +long err =PTR_ERR (cur->pcc_chan); + +kfree (cur); +return ERR_PTR (err); + } + +/* Timeout based on the "nominal latency" from the PCC ACPI table. */ + cur->pcc_cl.tx_tout = cur->pcc_chan->latency *5 ; + + ret =devm_mutex_init (dev, &cur->pcc_chan_lock); +if (ret) +return ERR_PTR (ret); + + cur->subspace_id = subspace_id; +kref_init (&cur->refcount); + +list_add_tail (&cur->pcc_chans, &pcc_chan_list); + +return cur; +} + +static int mpam_pcc_chan_put (struct mpam_pcc_chan *pcc_chan) +{ +struct mpam_pcc_chan *cur, *tmp; + +if (!pcc_chan) +return 0 ; + +guard (mutex)(&pcc_chan_list_lock); + +list_for_each_entry_safe (cur, tmp, &pcc_chan_list, pcc_chans) { +if (cur == pcc_chan) { +kref_put (&cur->refcount, mpam_pcc_chan_release); + +return 0 ; + } + } + +return -ENOENT ; +} +/* * Number of MSCs that have been probed. Once all MSCs have been probed MPAM * can be enabled. @@ -2201,6 +2286,8 @@ static void mpam_msc_drv_remove(struct platform_device *pdev) {struct mpam_msc *msc =platform_get_drvdata (pdev); +mpam_pcc_chan_put (msc->pcc_chan); +mutex_lock (&mpam_list_lock);mpam_msc_destroy (msc);mutex_unlock (&mpam_list_lock);@@ -2211,7 +2298,7 @@ static void mpam_msc_drv_remove(struct platform_device *pdev)static struct mpam_msc *do_mpam_msc_drv_probe (struct platform_device *pdev) {int err; -u32 tmp; +u32 pcc_subspace_id;struct mpam_msc *msc;struct resource *msc_res;struct device *dev = &pdev->dev;@@ -2259,7 +2346,8 @@ static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev)if (err)return ERR_PTR (err); -if (device_property_read_u32 (&pdev->dev,"pcc-channel" , &tmp)) +if (device_property_read_u32 (&pdev->dev,"pcc-channel" , + &pcc_subspace_id)) msc->iface =MPAM_IFACE_MMIO ;else msc->iface =MPAM_IFACE_PCC ;
drivers/resctrl/mpam_devices.c static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) · line 2370
Quoted 8 lines
drivers/resctrl/mpam_devices.c +5
-0@@ -2275,6 +2363,41 @@ static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) } msc->mapped_hwpage_sz = msc_res->end - msc_res->start; msc->mapped_hwpage = io; + }else if (msc->iface ==MPAM_IFACE_PCC ) { +u32 msc_id; +int ret; + +if (device_property_read_u32 (&pdev->dev,"msc-id" , &msc_id)) {
We have an existing local variable 'dev' we can use here...
drivers/resctrl/mpam_devices.c static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) · line 2376
Quoted 6 lines
drivers/resctrl/mpam_devices.c +6
-0+pr_err ("missing MPAM-Fb MSC identifier\n" ); +return ERR_PTR (-EINVAL ); + } + msc->mpam_fb_msc_id = msc_id; + + msc->pcc_chan =mpam_pcc_chan_get (&pdev->dev, pcc_subspace_id);
and here.
Quoted 9 lines
drivers/resctrl/mpam_devices.c +9
-0+if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); +return ERR_CAST (msc->pcc_chan); + } + +if (msc->pcc_chan->pcc_chan->shmem_size <MPAM_FB_MAX_MSG_SIZE ) { +pr_err ("MPAM-Fb PCC channel size too small.\n" ); +mpam_pcc_chan_put (msc->pcc_chan); +return ERR_PTR (-ENOMEM );
drivers/resctrl/mpam_devices.c static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) · line 2395
Quoted 10 lines
drivers/resctrl/mpam_devices.c +10
-0+ } + ret =mpam_fb_get_protocol_version (msc); +if (ret <0 ) { +pr_err ("Cannot query MPAM-Fb protocol version.\n" ); +mpam_pcc_chan_put (msc->pcc_chan); + +/* ret is an MPAM-Fb error code, return a Linux one. */ +return ERR_PTR (-EIO ); + } +if ((ret >>16 ) !=1 ) {
We can use FIELD_GET() to make this check clearer and to populate the error message just below.
drivers/resctrl/mpam_devices.c static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) · line 2403
Quoted 8 lines
drivers/resctrl/mpam_devices.c +5
-0+pr_err ("Incompatible MPAM-Fb protocol version %d.%d\n" , + ret >>16 , ret &0xffff ); +mpam_pcc_chan_put (msc->pcc_chan); +return ERR_PTR (-EINVAL ); + } }else {return ERR_PTR (-EINVAL ); }
Thanks,
Ben
Re: [PATCH v4 10/10] arm_mpam: detect and enable MPAM-Fb PCC support
Hi Ben,
Quoted 3 lines · Ben Horgan
Hi Andre,
Just a couple of nits I spotted just now.
Ok, thanks, both fixed.
Cheers,
Andre
Quoted 242 lines · Andre Przywara
Quoted 185 lines
The Arm MPAM-Fb specification [1] describes a protocol to access MSC registers through a firmware interface. This requires a shared memory region to hold the message, and a mailbox to trigger the access. For ACPI this is wrapped as a PCC channel, described using existing ACPI abstractions.
Add code to parse those PCC table descriptions associated with an MSC, and store the parsed information in the MSC struct.
There can be multiple PCC channels, and each channel can serve multiple MSCs, so we need to keep track of the channel usage, using a list and a refcount.
This will be used by the MPAM-Fb access wrapper code.[1] https://developer.arm.com/documentation/den0144/latest
Signed-off-by Andre Przywara <[email protected]>--- drivers/acpi/arm64/mpam.c | 6 +- drivers/resctrl/mpam_devices.c | 127 ++++++++++++++++++++++++++++++++- 2 files changed, 129 insertions(+), 4 deletions(-)drivers/acpi/arm64/mpam.c +4
-2@@ -220,8 +220,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ struct platform_device *pdev __free(platform_device_put) =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);int next_res =0 , next_prop =0 , err; -/* pcc, nrdy, affinity and a sentinel */ -struct property_entry props[4 ] = {0 }; +/* pcc, msc-id, nrdy, affinity and a sentinel */ +struct property_entry props[5 ] = {0 };/* mmio, 2xirq, no sentinel. */ struct resource res[3 ] = {0 };struct acpi_device *companion;@@ -256,6 +256,8 @@ static struct platform_device * __init acpi_mpam_parse_msc(struct acpi_mpam_msc_ }else if (iface ==MPAM_IFACE_PCC ) { props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , tbl_msc->base_address); + props[next_prop++] =PROPERTY_ENTRY_U32 ("msc-id" , + tbl_msc->identifier); }acpi_mpam_parse_irqs (pdev, tbl_msc, res, &next_res);drivers/resctrl/mpam_devices.c +90
-2@@ -19,14 +19,19 @@#include <linux/irqdesc.h> #include <linux/list.h> #include <linux/lockdep.h> +#include <linux/mailbox_client.h> #include <linux/mutex.h> #include <linux/platform_device.h> #include <linux/printk.h> +#include <linux/property.h> #include <linux/srcu.h> #include <linux/spinlock.h> #include <linux/types.h> #include <linux/workqueue.h> +#include <acpi/pcc.h> +#include <acpi/acpi_io.h> +#include "mpam_internal.h" /* Values for the T241 errata workaround */ @@ -49,6 +54,86 @@ static LIST_HEAD(mpam_all_msc);struct srcu_struct mpam_srcu; +/* PCC channels might be serving multiple MSCs, so keep a refcounted list. */ +static DEFINE_MUTEX (pcc_chan_list_lock); +static LIST_HEAD (pcc_chan_list); + +static void mpam_pcc_chan_release (struct kref *ref) +{ + struct mpam_pcc_chan *cur = container_of(ref, struct mpam_pcc_chan, + refcount); + +pcc_mbox_free_channel (cur->pcc_chan); +list_del (&cur->pcc_chans); +kfree (cur); +} + +static struct mpam_pcc_chan *mpam_pcc_chan_get (struct device *dev, +int subspace_id) +{ +struct mpam_pcc_chan *cur; +int ret; + +guard (mutex)(&pcc_chan_list_lock); + +list_for_each_entry (cur, &pcc_chan_list, pcc_chans) { +if (cur->subspace_id == subspace_id) { +kref_get (&cur->refcount); + +return cur; + } + } + + cur =kzalloc_obj (*cur); +if (!cur) +return ERR_PTR (-ENOMEM ); + + cur->pcc_cl.dev = dev; + cur->pcc_cl.tx_block = true; + + cur->pcc_chan =pcc_mbox_request_channel (&cur->pcc_cl, subspace_id); +if (IS_ERR (cur->pcc_chan)) { +long err =PTR_ERR (cur->pcc_chan); + +kfree (cur); +return ERR_PTR (err); + } + +/* Timeout based on the "nominal latency" from the PCC ACPI table. */ + cur->pcc_cl.tx_tout = cur->pcc_chan->latency *5 ; + + ret =devm_mutex_init (dev, &cur->pcc_chan_lock); +if (ret) +return ERR_PTR (ret); + + cur->subspace_id = subspace_id; +kref_init (&cur->refcount); + +list_add_tail (&cur->pcc_chans, &pcc_chan_list); + +return cur; +} + +static int mpam_pcc_chan_put (struct mpam_pcc_chan *pcc_chan) +{ +struct mpam_pcc_chan *cur, *tmp; + +if (!pcc_chan) +return 0 ; + +guard (mutex)(&pcc_chan_list_lock); + +list_for_each_entry_safe (cur, tmp, &pcc_chan_list, pcc_chans) { +if (cur == pcc_chan) { +kref_put (&cur->refcount, mpam_pcc_chan_release); + +return 0 ; + } + } + +return -ENOENT ; +} +/* * Number of MSCs that have been probed. Once all MSCs have been probed MPAM * can be enabled. @@ -2201,6 +2286,8 @@ static void mpam_msc_drv_remove(struct platform_device *pdev) {struct mpam_msc *msc =platform_get_drvdata (pdev); +mpam_pcc_chan_put (msc->pcc_chan); +mutex_lock (&mpam_list_lock);mpam_msc_destroy (msc);mutex_unlock (&mpam_list_lock);@@ -2211,7 +2298,7 @@ static void mpam_msc_drv_remove(struct platform_device *pdev)static struct mpam_msc *do_mpam_msc_drv_probe (struct platform_device *pdev) {int err; -u32 tmp; +u32 pcc_subspace_id;struct mpam_msc *msc;struct resource *msc_res;struct device *dev = &pdev->dev;@@ -2259,7 +2346,8 @@ static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev)if (err)return ERR_PTR (err); -if (device_property_read_u32 (&pdev->dev,"pcc-channel" , &tmp)) +if (device_property_read_u32 (&pdev->dev,"pcc-channel" , + &pcc_subspace_id)) msc->iface =MPAM_IFACE_MMIO ;else msc->iface =MPAM_IFACE_PCC ;drivers/resctrl/mpam_devices.c static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) · line 2370
Quoted 8 lines
drivers/resctrl/mpam_devices.c +5
-0@@ -2275,6 +2363,41 @@ static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) } msc->mapped_hwpage_sz = msc_res->end - msc_res->start; msc->mapped_hwpage = io; + }else if (msc->iface ==MPAM_IFACE_PCC ) { +u32 msc_id; +int ret; + +if (device_property_read_u32 (&pdev->dev,"msc-id" , &msc_id)) {We have an existing local variable 'dev' we can use here...
drivers/resctrl/mpam_devices.c static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) · line 2376
Quoted 6 lines
drivers/resctrl/mpam_devices.c +6
-0+pr_err ("missing MPAM-Fb MSC identifier\n" ); +return ERR_PTR (-EINVAL ); + } + msc->mpam_fb_msc_id = msc_id; + + msc->pcc_chan =mpam_pcc_chan_get (&pdev->dev, pcc_subspace_id);and here.
Quoted 9 lines
drivers/resctrl/mpam_devices.c +9
-0+if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); +return ERR_CAST (msc->pcc_chan); + } + +if (msc->pcc_chan->pcc_chan->shmem_size <MPAM_FB_MAX_MSG_SIZE ) { +pr_err ("MPAM-Fb PCC channel size too small.\n" ); +mpam_pcc_chan_put (msc->pcc_chan); +return ERR_PTR (-ENOMEM );drivers/resctrl/mpam_devices.c static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) · line 2395
Quoted 10 lines
drivers/resctrl/mpam_devices.c +10
-0+ } + ret =mpam_fb_get_protocol_version (msc); +if (ret <0 ) { +pr_err ("Cannot query MPAM-Fb protocol version.\n" ); +mpam_pcc_chan_put (msc->pcc_chan); + +/* ret is an MPAM-Fb error code, return a Linux one. */ +return ERR_PTR (-EIO ); + } +if ((ret >>16 ) !=1 ) {We can use
FIELD_GET()to make this check clearer and to populate the error message just below.drivers/resctrl/mpam_devices.c static struct mpam_msc *do_mpam_msc_drv_probe(struct platform_device *pdev) · line 2403
Quoted 8 lines
drivers/resctrl/mpam_devices.c +5
-0+pr_err ("Incompatible MPAM-Fb protocol version %d.%d\n" , + ret >>16 , ret &0xffff ); +mpam_pcc_chan_put (msc->pcc_chan); +return ERR_PTR (-EINVAL ); + } }else {return ERR_PTR (-EINVAL ); }Thanks,
Ben