arm_mpam: Add basic mpam driver
Subsystems (6)
James Morse
maintainercover arm_mpam: Add basic mpam driver
Hello,
This is just enough MPAM driver for the ACPI and DT pre-requisites. It doesn't contain any of the resctrl code, meaning you can't actually drive it from user-space yet. Becuase of that, its hidden behind CONFIG_EXPERT. This will change once the user interface is connected up.
This is the initial group of patches that allows the resctrl code to be built on top. Including that will increase the number of trees that may need to coordinate, so breaking it up make sense.
The locking looks very strange - but is influenced by the 'mpam-fb' firmware interface specification that is still alpha. That thing needs to wait for an interrupt after every system register write, which significantly impacts the driver. Some features just won't work, e.g. reading the monitor registers via perf.
The aim is to not have to make invasive changes to the locking to support the firmware interface, hence it looks strange from day-1.
I've not found a platform that can test all the behaviours around the monitors, so this is where I'd expect the most bugs.
The MPAM spec that describes all the system and MMIO registers can be found here:
https://developer.arm.com/documentation/ddi0598/db/?lang=en
(Ignored the 'RETIRED' warning - that is just arm moving the documentation around.
This document has the best overview)
The expectation is this will go via the arm64 tree.
This series is based on v6.17-rc2, and can be retrieved from:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/driver/rv1
The rest of the driver can be found here:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/snapshot/v6.17-rc2
What is MPAM? Set your time-machine to 2020:
https://lore.kernel.org/lkml/[email protected]/
This series was previously posted here:
[RFC] lore.kernel.org/r/[email protected]
Bugs welcome,
Thanks,
James Morse (29):
cacheinfo: Expose the code to generate a cache-id from a device_node
drivers: base: cacheinfo: Add helper to find the cache size from
cpu+level
ACPI / PPTT: Add a helper to fill a cpumask from a processor container ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear
levels
ACPI / PPTT: Find cache level by cache-id
ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
arm64: kconfig: Add Kconfig entry for MPAM
ACPI / MPAM: Parse the MPAM table
arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
arm_mpam: Add the class and component structures for ris firmware
described
arm_mpam: Add MPAM MSC register layout definitions
arm_mpam: Add cpuhp callbacks to probe MSC hardware
arm_mpam: Probe MSCs to find the supported partid/pmg values
arm_mpam: Add helpers for managing the locking around the mon_sel
registers
arm_mpam: Probe the hardware features resctrl supports
arm_mpam: Merge supported features during mpam_enable() into
mpam_class
arm_mpam: Reset MSC controls from cpu hp callbacks
arm_mpam: Add a helper to touch an MSC from any CPU
arm_mpam: Extend reset logic to allow devices to be reset any time
arm_mpam: Register and enable IRQs
arm_mpam: Use a static key to indicate when mpam is enabled
arm_mpam: Allow configuration to be applied and restored during cpu
online
arm_mpam: Probe and reset the rest of the features
arm_mpam: Add helpers to allocate monitors
arm_mpam: Add mpam_msmon_read() to read monitor value
arm_mpam: Track bandwidth counter state for overflow and power
management
arm_mpam: Add helper to reset saved mbwu state
arm_mpam: Add kunit test for bitmap reset
arm_mpam: Add kunit tests for props_mismatch()
Rob Herring (1):
dt-bindings: arm: Add MPAM MSC binding
Rohit Mathew (2):
arm_mpam: Probe for long/lwd mbwu counters
arm_mpam: Use long MBWU counters if supported
Shanker Donthineni (1):
arm_mpam: Add support for memory controller MSC on DT platforms
.../devicetree/bindings/arm/arm,mpam-msc.yaml | 200 ++ arch/arm64/Kconfig | 19 + drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/acpi/arm64/Kconfig | 3 + drivers/acpi/arm64/Makefile | 1 + drivers/acpi/arm64/mpam.c | 331 ++ drivers/acpi/pptt.c | 230 +- drivers/acpi/tables.c | 2 +- drivers/base/cacheinfo.c | 19 +- drivers/resctrl/Kconfig | 24 + drivers/resctrl/Makefile | 4 + drivers/resctrl/mpam_devices.c | 2909 +++++++++++++++++ drivers/resctrl/mpam_internal.h | 692 ++++ drivers/resctrl/test_mpam_devices.c | 390 +++ include/linux/acpi.h | 26 + include/linux/arm_mpam.h | 56 + include/linux/cacheinfo.h | 16 + 18 files changed, 4911 insertions(+), 14 deletions(-) create mode 100644 Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml create mode 100644 drivers/acpi/arm64/mpam.c create mode 100644 drivers/resctrl/Kconfig create mode 100644 drivers/resctrl/Makefile create mode 100644 drivers/resctrl/mpam_devices.c create mode 100644 drivers/resctrl/mpam_internal.h create mode 100644 drivers/resctrl/test_mpam_devices.c create mode 100644 include/linux/arm_mpam.h
Signature · 1 line
2.20.1
James Morse
maintainerPATCH 1/33 cacheinfo: Expose the code to generate a cache-id from a device_node
The MPAM driver identifies caches by id for use with resctrl. It needs to know the cache-id when probe-ing, but the value isn't set in cacheinfo until device_initcall().
Expose the code that generates the cache-id. The parts of the MPAM driver that run early can use this to set up the resctrl structures before cacheinfo is ready in device_initcall().
---
Changes since v1:
* Renamed cache_of_get_id() cache_of_calculate_id().
--- drivers/base/cacheinfo.c | 19 +++++++++++++------ include/linux/cacheinfo.h | 1 + 2 files changed, 14 insertions(+), 6 deletions(-)
drivers/base/cacheinfo.c +13 -6
@@ -207,11 +207,10 @@ static bool match_cache_node(struct device_node *cpu,#define arch_compact_of_hwid (_x) (_x)#endif -static void cache_of_set_id (struct cacheinfo *this_leaf, -struct device_node *cache_node) +unsigned long cache_of_calculate_id (struct device_node *cache_node) {struct device_node *cpu; -u32 min_id = ~0 ; +unsigned long min_id = ~0UL ;for_each_of_cpu_node (cpu) {u64 id =of_get_cpu_hwid (cpu,0 );
@@ -219,15 +218,23 @@ static void cache_of_set_id(struct cacheinfo *this_leaf, id =arch_compact_of_hwid (id);if (FIELD_GET (GENMASK_ULL (63 ,32 ), id)) {of_node_put (cpu); -return ; +return ~0UL ; }if (match_cache_node (cpu, cache_node)) min_id =min (min_id, id); } -if (min_id != ~0 ) { - this_leaf->id = min_id; +return min_id; +} + +static void cache_of_set_id (struct cacheinfo *this_leaf, +struct device_node *cache_node) +{ +unsigned long id =cache_of_calculate_id (cache_node); + +if (id != ~0UL ) { + this_leaf->id = id; this_leaf->attributes |=CACHE_ID ; } }
include/linux/cacheinfo.h +1 -0
@@ -112,6 +112,7 @@ int acpi_get_cache_info(unsigned int cpu,#endif const struct attribute_group *cache_get_priv_group (struct cacheinfo *this_leaf); +unsigned long cache_of_calculate_id (struct device_node *np);/* * Get the cacheinfo structure for the cache associated with @cpu at
Signature · 1 line
2.20.1
Re: [PATCH 01/33] cacheinfo: Expose the code to generate a cache-id from a device_node
Hi James,
Quoted 7 lines · James Morse
The MPAM driver identifies caches by id for use with resctrl. It needs to know the cache-id when probe-ing, but the value isn't set in cacheinfo until
device_initcall().Expose the code that generates the cache-id. The parts of the MPAM driver that run early can use this to set up the resctrl structures before cacheinfo is ready in
device_initcall().
Why can't the MPAM driver just consume the precomputed cache-id information?
Possible reasons are that the MPAM driver probes too early, or that it must parse the PPTT directly (which is true) and needs to label caches consistently with the way the kernel does it.
Quoted 15 lines
Signed-off-by James Morse <[email protected]>---
Changes since v1:
* Renamedcache_of_get_id()cache_of_calculate_id().--- drivers/base/cacheinfo.c | 19 +++++++++++++------ include/linux/cacheinfo.h | 1 + 2 files changed, 14 insertions(+), 6 deletions(-)drivers/base/cacheinfo.c +0
-0
drivers/base/cacheinfo.c static bool match_cache_node(struct device_node *cpu, · line 213
Quoted 10 lines
drivers/base/cacheinfo.c +2
-3@@ -207,11 +207,10 @@ static bool match_cache_node(struct device_node *cpu,#define arch_compact_of_hwid (_x) (_x)#endif -static void cache_of_set_id (struct cacheinfo *this_leaf, -struct device_node *cache_node) +unsigned long cache_of_calculate_id (struct device_node *cache_node) {struct device_node *cpu; -u32 min_id = ~0 ; +unsigned long min_id = ~0UL ;
Why the change of type here?
This does mean that 0xffffffff can now be generated as a valid cache-id, but if that is necessary then this patch is also fixing a bug in the code -- but the commit message doesn't say anything about that.
For a patch that is just exposing an internal result, it may be better to keep the original type. ~(u32)0 is already used as an exceptional value.
[...]
Otherwise, this looks reasonable to me.
Cheers
---Dave
James Morse
maintainerRe: [PATCH 01/33] cacheinfo: Expose the code to generate a cache-id from a device_node
Hi Dave,
Quoted 11 lines · Dave Martin
Quoted 7 lines · James Morse
The MPAM driver identifies caches by id for use with resctrl. It needs to know the cache-id when probe-ing, but the value isn't set in cacheinfo until
device_initcall().Expose the code that generates the cache-id. The parts of the MPAM driver that run early can use this to set up the resctrl structures before cacheinfo is ready in
device_initcall().Why can't the MPAM driver just consume the precomputed cache-id information?
Because it would need to wait until cacheinfo was ready, and it would still need a way of getting the cache-id for caches where all the CPUs are offline.
The resctrl glue code has a waitqueue to wait for device_initcall_sync(), but that is asynchronous to driver probing, its triggered by the schedule_work() from the cpuhp callbacks. This bit is about the driver's use, which just gets probed whenever the core code feels like it.
I toyed with always using cacheinfo for everything, and just waiting - but the MPAM driver already has to parse the PPTT to find the information it needs on ACPI platforms, so the wait would only happen on DT.
It seemed simpler to grab what the value would be, instead of waiting (or probe defer) - especially as this is also needed for caches where all the CPUs are offline.
(I'll add the offline-cpus angle to the commit message)
Quoted 1 line
Possible reasons are that the MPAM driver probes too early,
yup,
Quoted 3 lines
or that it
must parse the PPTT directly (which is true) and needs to label caches consistently with the way the kernel does it.
It needs to match what will be exposed to user-space from cacheinfo.
This isn't about the PPTT, its the value that is generated for DT systems. The driver has to know if its ACPI or DT to call the appropriate thing to get cache-ids before cacheinfo is ready.
Quoted 8 lines
Quoted 8 lines
drivers/base/cacheinfo.c +0
-0@@ -207,11 +207,10 @@ static bool match_cache_node(struct device_node *cpu,#define arch_compact_of_hwid (_x) (_x)#endif
Quoted 9 lines
drivers/base/cacheinfo.c static bool match_cache_node(struct device_node *cpu, · line 213
Quoted 7 lines
drivers/base/cacheinfo.c +2
-3-static void cache_of_set_id (struct cacheinfo *this_leaf, -struct device_node *cache_node) +unsigned long cache_of_calculate_id (struct device_node *cache_node) {struct device_node *cpu; -u32 min_id = ~0 ; +unsigned long min_id = ~0UL ;Why the change of type here?
This is a hang over from Rob's approach of making the cache-id 64 bit.
Quoted 7 lines
This does mean that
0xffffffffcan now be generated as a valid cache-id, but if that is necessary then this patch is also fixing a bug in the code -- but the commit message doesn't say anything about that.For a patch that is just exposing an internal result, it may be better to keep the original type. ~(u32)0 is already used as an exceptional value.
Yup, I'll fix that.
Thanks!
James
Re: [PATCH 01/33] cacheinfo: Expose the code to generate a cache-id from a device_node
Hi James,
Quoted 23 lines · James Morse
Hi Dave,
Quoted 11 lines · Dave Martin
Quoted 7 lines · James Morse
The MPAM driver identifies caches by id for use with resctrl. It needs to know the cache-id when probe-ing, but the value isn't set in cacheinfo until
device_initcall().Expose the code that generates the cache-id. The parts of the MPAM driver that run early can use this to set up the resctrl structures before cacheinfo is ready in
device_initcall().Why can't the MPAM driver just consume the precomputed cache-id information?
Because it would need to wait until cacheinfo was ready, and it would still need a way of getting the cache-id for caches where all the CPUs are offline.
The resctrl glue code has a waitqueue to wait for
device_initcall_sync(), but that is asynchronous to driver probing, its triggered by theschedule_work()from the cpuhp callbacks. This bit is about the driver's use, which just gets probed whenever the core code feels like it.
Quoted 8 lines
I toyed with always using cacheinfo for everything, and just waiting - but the MPAM driver already has to parse the PPTT to find the information it needs on ACPI platforms, so the wait would only happen on DT.
It seemed simpler to grab what the value would be, instead of waiting (or probe defer) - especially as this is also needed for caches where all the CPUs are offline.
(I'll add the offline-cpus angle to the commit message)
Ack
Quoted 10 lines
Quoted 1 line
Possible reasons are that the MPAM driver probes too early,
yup,
Quoted 3 lines
or that it
must parse the PPTT directly (which is true) and needs to label caches consistently with the way the kernel does it.It needs to match what will be exposed to user-space from cacheinfo. This isn't about the PPTT, its the value that is generated for DT systems.
Right -- confused myself there. From the point of view of this series, the usage scenario isn't clear at this point.
Quoted 2 lines
The driver has to know if its ACPI or DT to call the appropriate thing to get cache-ids before cacheinfo is ready.
I see. This might be worth stating in the commit message.
Quoted 18 lines
Quoted 8 lines
Quoted 8 lines
drivers/base/cacheinfo.c +0
-0@@ -207,11 +207,10 @@ static bool match_cache_node(struct device_node *cpu,#define arch_compact_of_hwid (_x) (_x)#endif Quoted 9 lines
drivers/base/cacheinfo.c static bool match_cache_node(struct device_node *cpu, · line 213
Quoted 7 lines
drivers/base/cacheinfo.c +2
-3-static void cache_of_set_id (struct cacheinfo *this_leaf, -struct device_node *cache_node) +unsigned long cache_of_calculate_id (struct device_node *cache_node) {struct device_node *cpu; -u32 min_id = ~0 ; +unsigned long min_id = ~0UL ;Why the change of type here?
Quoted 1 line
This is a hang over from Rob's approach of making the cache-id 64 bit.
Ah, right.
(I have assumed that 0xffffffff is never going to clash with a valid value.)
Quoted 9 lines
Quoted 7 lines
This does mean that
0xffffffffcan now be generated as a valid cache-id, but if that is necessary then this patch is also fixing a bug in the code -- but the commit message doesn't say anything about that.For a patch that is just exposing an internal result, it may be better to keep the original type. ~(u32)0 is already used as an exceptional value.
Yup, I'll fix that.
OK -- it works either way, of course, but this should make the patch a little less noisy.
Cheers
---Dave
James Morse
maintainerPATCH 1/33 cacheinfo: Expose the code to generate a cache-id from a device_node
In reply to James MorseThe MPAM driver identifies caches by id for use with resctrl. It needs to know the cache-id when probe-ing, but the value isn't set in cacheinfo until device_initcall().
Expose the code that generates the cache-id. The parts of the MPAM driver that run early can use this to set up the resctrl structures before cacheinfo is ready in device_initcall().
---
Changes since v1:
* Renamed cache_of_get_id() cache_of_calculate_id().
--- drivers/base/cacheinfo.c | 19 +++++++++++++------ include/linux/cacheinfo.h | 1 + 2 files changed, 14 insertions(+), 6 deletions(-)
drivers/base/cacheinfo.c +13 -6
@@ -207,11 +207,10 @@ static bool match_cache_node(struct device_node *cpu,#define arch_compact_of_hwid (_x) (_x)#endif -static void cache_of_set_id (struct cacheinfo *this_leaf, -struct device_node *cache_node) +unsigned long cache_of_calculate_id (struct device_node *cache_node) {struct device_node *cpu; -u32 min_id = ~0 ; +unsigned long min_id = ~0UL ;for_each_of_cpu_node (cpu) {u64 id =of_get_cpu_hwid (cpu,0 );
@@ -219,15 +218,23 @@ static void cache_of_set_id(struct cacheinfo *this_leaf, id =arch_compact_of_hwid (id);if (FIELD_GET (GENMASK_ULL (63 ,32 ), id)) {of_node_put (cpu); -return ; +return ~0UL ; }if (match_cache_node (cpu, cache_node)) min_id =min (min_id, id); } -if (min_id != ~0 ) { - this_leaf->id = min_id; +return min_id; +} + +static void cache_of_set_id (struct cacheinfo *this_leaf, +struct device_node *cache_node) +{ +unsigned long id =cache_of_calculate_id (cache_node); + +if (id != ~0UL ) { + this_leaf->id = id; this_leaf->attributes |=CACHE_ID ; } }
include/linux/cacheinfo.h +1 -0
@@ -112,6 +112,7 @@ int acpi_get_cache_info(unsigned int cpu,#endif const struct attribute_group *cache_get_priv_group (struct cacheinfo *this_leaf); +unsigned long cache_of_calculate_id (struct device_node *np);/* * Get the cacheinfo structure for the cache associated with @cpu at
Signature · 1 line
2.20.1
James Morse
maintainerPATCH 2/33 drivers: base: cacheinfo: Add helper to find the cache size from cpu+level
In reply to James MorseMPAM needs to know the size of a cache associated with a particular CPU. The DT/ACPI agnostic way of doing this is to ask cacheinfo.
Add a helper to do this.
---
Changes since v1:
* Converted to kdoc.
* Simplified helper to use get_cpu_cacheinfo_level().
--- include/linux/cacheinfo.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
include/linux/cacheinfo.h +15 -0
@@ -148,6 +148,21 @@ static inline int get_cpu_cacheinfo_id(int cpu, int level)return ci ? ci->id :-1 ; } +/** +* get_cpu_cacheinfo_size() - Get the size of the cache. +* @cpu: The cpu that is associated with the cache. +* @level: The level of the cache as seen by @cpu. +* +* Callers must hold the cpuhp lock. +* Returns the cache-size on success, or 0 for an error. +*/ +static inline unsigned int get_cpu_cacheinfo_size (int cpu,int level) +{ +struct cacheinfo *ci =get_cpu_cacheinfo_level (cpu, level); + +return ci ? ci->size :0 ; +} +#if defined(CONFIG_ARM64 ) || defined(CONFIG_ARM )#define use_arch_cache_info () (true)#else
Signature · 1 line
2.20.1
Re: [PATCH 02/33] drivers: base: cacheinfo: Add helper to find the cache size from cpu+level
Quoted 9 lines · James Morse
MPAM needs to know the size of a cache associated with a particular CPU. The DT/ACPI agnostic way of doing this is to ask cacheinfo.
Add a helper to do this.
Signed-off-by James Morse <[email protected]>---
Changes since v1:
You marked this as v1.
Quoted 2 lines
* Converted to kdoc.
* Simplified helper to useget_cpu_cacheinfo_level().
Please use consistent subject prefixes. Look at previous patch subject prefix.
Best regards,
Krzysztof
James Morse
maintainerRe: [PATCH 02/33] drivers: base: cacheinfo: Add helper to find the cache size from cpu+level
Hi Krzysztof,
Quoted 10 lines · Krzysztof Kozlowski
Quoted 7 lines · James Morse
MPAM needs to know the size of a cache associated with a particular CPU. The DT/ACPI agnostic way of doing this is to ask cacheinfo.
Add a helper to do this.
---
Changes since v1:You marked this as v1.
Oops - that should say RFC. I'll fix all those.
Quoted 5 lines
Quoted 2 lines
* Converted to kdoc.
* Simplified helper to useget_cpu_cacheinfo_level().Please use consistent subject prefixes. Look at previous patch subject prefix.
Presumably the previous patch in my series - this is a side effect of multiple branches that were written at different times getting combined! I'll change it to 'cacheinfo:' as that seems to be the most popular recently.
Thanks,
James
Re: [PATCH 02/33] drivers: base: cacheinfo: Add helper to find the cache size from cpu+level
In reply to James MorseHi,
Quoted 21 lines · James Morse
MPAM needs to know the size of a cache associated with a particular CPU. The DT/ACPI agnostic way of doing this is to ask cacheinfo.
Add a helper to do this.
Signed-off-by James Morse <[email protected]>---
Changes since v1:
* Converted to kdoc.
* Simplified helper to useget_cpu_cacheinfo_level().--- include/linux/cacheinfo.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)include/linux/cacheinfo.h +0
-0@@ -148,6 +148,21 @@ static inline int get_cpu_cacheinfo_id(int cpu, int level)return ci ? ci->id :-1 ;
include/linux/cacheinfo.h static inline int get_cpu_cacheinfo_id(int cpu, int level) · line 158
Quoted 10 lines
include/linux/cacheinfo.h +8
-0} +/** +* get_cpu_cacheinfo_size() - Get the size of the cache. +* @cpu: The cpu that is associated with the cache. +* @level: The level of the cache as seen by @cpu. +* +* Callers must hold the cpuhp lock. +* Returns the cache-size on success, or 0 for an error. +*/
Nit: Maybe use the wording
cpuhp lock must be held.
in the kerneldoc here, to match the other helpers it sits alongside.
Otherwise, looks reasonable.
include/linux/cacheinfo.h static inline int get_cpu_cacheinfo_id(int cpu, int level) · line 165
Quoted 7 lines
include/linux/cacheinfo.h +7
-0+static inline unsigned int get_cpu_cacheinfo_size (int cpu,int level) +{ +struct cacheinfo *ci =get_cpu_cacheinfo_level (cpu, level); + +return ci ? ci->size :0 ; +} +
Orphaned function?
Can fs/resctrl/rdtgroup.c:rdtgroup_cbm_to_size() be ported to use this? If so, this wouldn't just be dead code in this series.
Cheers
---Dave
James Morse
maintainerRe: [PATCH 02/33] drivers: base: cacheinfo: Add helper to find the cache size from cpu+level
Hi Dave,
Quoted 27 lines · Dave Martin
Hi,
Quoted 11 lines · James Morse
MPAM needs to know the size of a cache associated with a particular CPU. The DT/ACPI agnostic way of doing this is to ask cacheinfo.
Add a helper to do this.
include/linux/cacheinfo.h +0
-0@@ -148,6 +148,21 @@ static inline int get_cpu_cacheinfo_id(int cpu, int level)return ci ? ci->id :-1 ;include/linux/cacheinfo.h static inline int get_cpu_cacheinfo_id(int cpu, int level) · line 158
Quoted 10 lines
include/linux/cacheinfo.h +8
-0} +/** +* get_cpu_cacheinfo_size() - Get the size of the cache. +* @cpu: The cpu that is associated with the cache. +* @level: The level of the cache as seen by @cpu. +* +* Callers must hold the cpuhp lock. +* Returns the cache-size on success, or 0 for an error. +*/ Nit: Maybe use the wording
Quoted 5 lines
cpuhp lock must be held.
in the kerneldoc here, to match the other helpers it sits alongside.
Otherwise, looks reasonable.
Sure,
Quoted 10 lines
include/linux/cacheinfo.h static inline int get_cpu_cacheinfo_id(int cpu, int level) · line 165
Quoted 7 lines
include/linux/cacheinfo.h +7
-0+static inline unsigned int get_cpu_cacheinfo_size (int cpu,int level) +{ +struct cacheinfo *ci =get_cpu_cacheinfo_level (cpu, level); + +return ci ? ci->size :0 ; +} +Orphaned function?
Quoted 2 lines
Can fs/resctrl/rdtgroup.c:rdtgroup_cbm_to_size() be ported to use this? If so, this wouldn't just be dead code in this series.
Ah - I thought the MPAM driver was pulling this value in, but its the resctrl glue code. I was trying to reduce the number of trees this touches - its probably best to kick this into the next series that adds the resctrl code as its pretty trivial.
Thanks,
James
Re: [PATCH 02/33] drivers: base: cacheinfo: Add helper to find the cache size from cpu+level
Hi James,
Quoted 52 lines · James Morse
Hi Dave,
Quoted 27 lines · Dave Martin
Hi,
Quoted 11 lines · James Morse
MPAM needs to know the size of a cache associated with a particular CPU. The DT/ACPI agnostic way of doing this is to ask cacheinfo.
Add a helper to do this.
include/linux/cacheinfo.h +0
-0@@ -148,6 +148,21 @@ static inline int get_cpu_cacheinfo_id(int cpu, int level)return ci ? ci->id :-1 ;include/linux/cacheinfo.h static inline int get_cpu_cacheinfo_id(int cpu, int level) · line 158
Quoted 10 lines
include/linux/cacheinfo.h +8
-0} +/** +* get_cpu_cacheinfo_size() - Get the size of the cache. +* @cpu: The cpu that is associated with the cache. +* @level: The level of the cache as seen by @cpu. +* +* Callers must hold the cpuhp lock. +* Returns the cache-size on success, or 0 for an error. +*/ Nit: Maybe use the wording
Quoted 5 lines
cpuhp lock must be held.
in the kerneldoc here, to match the other helpers it sits alongside.
Otherwise, looks reasonable.
Sure,
Quoted 10 lines
include/linux/cacheinfo.h static inline int get_cpu_cacheinfo_id(int cpu, int level) · line 165
Quoted 7 lines
include/linux/cacheinfo.h +7
-0+static inline unsigned int get_cpu_cacheinfo_size (int cpu,int level) +{ +struct cacheinfo *ci =get_cpu_cacheinfo_level (cpu, level); + +return ci ? ci->size :0 ; +} +Orphaned function?
Quoted 2 lines
Can fs/resctrl/rdtgroup.c:rdtgroup_cbm_to_size() be ported to use this? If so, this wouldn't just be dead code in this series.
Quoted 8 lines
Ah - I thought the MPAM driver was pulling this value in, but its the resctrl glue code. I was trying to reduce the number of trees this touches - its probably best to kick this into the next series that adds the resctrl code as its pretty trivial.
Thanks,
James
Sure, that also works.
Cheers
---Dave
Re: [PATCH 02/33] drivers: base: cacheinfo: Add helper to find the cache size from cpu+level
In reply to James MorseHi,
Quoted 12 lines · James Morse
Hi Dave,
Quoted 9 lines · Dave Martin
Hi,
Quoted 6 lines · James Morse
MPAM needs to know the size of a cache associated with a particular CPU. The DT/ACPI agnostic way of doing this is to ask cacheinfo.
Add a helper to do this.
include/linux/cacheinfo.h +0
-0
[...]
Quoted 13 lines
Quoted 10 lines
include/linux/cacheinfo.h static inline int get_cpu_cacheinfo_id(int cpu, int level) · line 165
Quoted 7 lines
include/linux/cacheinfo.h +7
-0+static inline unsigned int get_cpu_cacheinfo_size (int cpu,int level) +{ +struct cacheinfo *ci =get_cpu_cacheinfo_level (cpu, level); + +return ci ? ci->size :0 ; +} +Orphaned function?
Quoted 2 lines
Can fs/resctrl/rdtgroup.c:rdtgroup_cbm_to_size() be ported to use this? If so, this wouldn't just be dead code in this series.
Quoted 9 lines
Ah - I thought the MPAM driver was pulling this value in, but its the resctrl glue code. I was trying to reduce the number of trees this touches - its probably best to kick this into the next series that adds the resctrl code as its pretty trivial.
Thanks,
James
Fair enough.
Cheers
---Dave
James Morse
maintainerPATCH 2/33 drivers: base: cacheinfo: Add helper to find the cache size from cpu+level
In reply to James MorseMPAM needs to know the size of a cache associated with a particular CPU. The DT/ACPI agnostic way of doing this is to ask cacheinfo.
Add a helper to do this.
---
Changes since v1:
* Converted to kdoc.
* Simplified helper to use get_cpu_cacheinfo_level().
--- include/linux/cacheinfo.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
include/linux/cacheinfo.h +15 -0
@@ -148,6 +148,21 @@ static inline int get_cpu_cacheinfo_id(int cpu, int level)return ci ? ci->id :-1 ; } +/** +* get_cpu_cacheinfo_size() - Get the size of the cache. +* @cpu: The cpu that is associated with the cache. +* @level: The level of the cache as seen by @cpu. +* +* Callers must hold the cpuhp lock. +* Returns the cache-size on success, or 0 for an error. +*/ +static inline unsigned int get_cpu_cacheinfo_size (int cpu,int level) +{ +struct cacheinfo *ci =get_cpu_cacheinfo_level (cpu, level); + +return ci ? ci->size :0 ; +} +#if defined(CONFIG_ARM64 ) || defined(CONFIG_ARM )#define use_arch_cache_info () (true)#else
Signature · 1 line
2.20.1
James Morse
maintainerPATCH 3/33 ACPI / PPTT: Add a helper to fill a cpumask from a processor container
In reply to James MorseThe PPTT describes CPUs and caches, as well as processor containers. The ACPI table for MPAM describes the set of CPUs that can access an MSC with the UID of a processor container.
Add a helper to find the processor container by its id, then walk the possible CPUs to fill a cpumask with the CPUs that have this processor container as a parent.
---
Changes since RFC:
* Dropped has_leaf_flag dodging of acpi_pptt_leaf_node()
* Added missing : in kernel-doc
* Made helper return void as this never actually returns an error.
--- drivers/acpi/pptt.c | 86 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 3 ++ 2 files changed, 89 insertions(+)
drivers/acpi/pptt.c +86 -0
@@ -298,6 +298,92 @@ static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_hereturn NULL ; } +/** +* acpi_pptt_get_child_cpus() - Find all the CPUs below a PPTT processor node +* @table_hdr: A reference to the PPTT table. +* @parent_node: A pointer to the processor node in the @table_hdr. +* @cpus: A cpumask to fill with the CPUs below @parent_node. +* +* Walks up the PPTT from every possible CPU to find if the provided +* @parent_node is a parent of this CPU. +*/ +static void acpi_pptt_get_child_cpus (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *parent_node, +cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +u32 acpi_id; +int cpu; + +cpumask_clear (cpus); + +for_each_possible_cpu (cpu) { + acpi_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table_hdr, acpi_id); + +while (cpu_node) { +if (cpu_node == parent_node) { +cpumask_set_cpu (cpu, cpus); +break ; + } + cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); + } + } +} + +/** +* acpi_pptt_get_cpus_from_container() - Populate a cpumask with all CPUs in a +* processor containers +* @acpi_cpu_id: The UID of the processor container. +* @cpus: The resulting CPU mask. +* +* Find the specified Processor Container, and fill @cpus with all the cpus +* below it. +* +* Not all 'Processor' entries in the PPTT are either a CPU or a Processor +* Container, they may exist purely to describe a Private resource. CPUs +* have to be leaves, so a Processor Container is a non-leaf that has the +* 'ACPI Processor ID valid' flag set. +* +* Return: 0 for a complete walk, or an error if the mask is incomplete. +*/ +void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +struct acpi_table_header *table_hdr; +struct acpi_subtable_header *entry; +unsigned long table_end; +acpi_status status; +bool leaf_flag; +u32 proc_sz; + +cpumask_clear (cpus); + + status =acpi_get_table (ACPI_SIG_PPTT ,0 , &table_hdr); +if (ACPI_FAILURE (status)) +return ; + + table_end = (unsigned long )table_hdr + table_hdr->length; + entry = ACPI_ADD_PTR(struct acpi_subtable_header, table_hdr, +sizeof (struct acpi_table_pptt )); + proc_sz =sizeof (struct acpi_pptt_processor ); +while ((unsigned long )entry + proc_sz <= table_end) { + cpu_node = (struct acpi_pptt_processor *)entry; +if (entry->type ==ACPI_PPTT_TYPE_PROCESSOR && + cpu_node->flags &ACPI_PPTT_ACPI_PROCESSOR_ID_VALID ) { + leaf_flag =acpi_pptt_leaf_node (table_hdr, cpu_node); +if (!leaf_flag) { +if (cpu_node->acpi_processor_id == acpi_cpu_id) +acpi_pptt_get_child_cpus (table_hdr, cpu_node, cpus); + } + } + entry = ACPI_ADD_PTR(struct acpi_subtable_header, entry, + entry->length); + } + +acpi_put_table (table_hdr); +} +static u8 acpi_cache_type (enum cache_type type) {switch (type) {
include/linux/acpi.h +3 -0
@@ -1541,6 +1541,7 @@ int find_acpi_cpu_topology(unsigned int cpu, int level);int find_acpi_cpu_topology_cluster (unsigned int cpu);int find_acpi_cpu_topology_package (unsigned int cpu);int find_acpi_cpu_topology_hetero_id (unsigned int cpu); +void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus);#else static inline int acpi_pptt_cpu_is_thread (unsigned int cpu) {
@@ -1562,6 +1563,8 @@ static inline int find_acpi_cpu_topology_hetero_id(unsigned int cpu) {return -EINVAL ; } +static inline void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id, +cpumask_t *cpus) { }#endif void acpi_arch_init (void );
Signature · 1 line
2.20.1
Ben Horgan
maintainerRe: [PATCH 03/33] ACPI / PPTT: Add a helper to fill a cpumask from a processor container
Hi James,
The patch logic update makes sense to me. Just a nit.
Quoted 93 lines · James Morse
The PPTT describes CPUs and caches, as well as processor containers. The ACPI table for MPAM describes the set of CPUs that can access an MSC with the UID of a processor container.
Add a helper to find the processor container by its id, then walk the possible CPUs to fill a cpumask with the CPUs that have this processor container as a parent.
CC Dave Martin <[email protected]>Reviewed-by Sudeep Holla <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* Droppedhas_leaf_flagdodging ofacpi_pptt_leaf_node()
* Added missing : in kernel-doc
* Made helper return void as this never actually returns an error.--- drivers/acpi/pptt.c | 86 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 3 ++ 2 files changed, 89 insertions(+)drivers/acpi/pptt.c +64
-0@@ -298,6 +298,92 @@ static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_hereturn NULL ; } +/** +* acpi_pptt_get_child_cpus() - Find all the CPUs below a PPTT processor node +* @table_hdr: A reference to the PPTT table. +* @parent_node: A pointer to the processor node in the @table_hdr. +* @cpus: A cpumask to fill with the CPUs below @parent_node. +* +* Walks up the PPTT from every possible CPU to find if the provided +* @parent_node is a parent of this CPU. +*/ +static void acpi_pptt_get_child_cpus (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *parent_node, +cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +u32 acpi_id; +int cpu; + +cpumask_clear (cpus); + +for_each_possible_cpu (cpu) { + acpi_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table_hdr, acpi_id); + +while (cpu_node) { +if (cpu_node == parent_node) { +cpumask_set_cpu (cpu, cpus); +break ; + } + cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); + } + } +} + +/** +* acpi_pptt_get_cpus_from_container() - Populate a cpumask with all CPUs in a +* processor containers +* @acpi_cpu_id: The UID of the processor container. +* @cpus: The resulting CPU mask. +* +* Find the specified Processor Container, and fill @cpus with all the cpus +* below it. +* +* Not all 'Processor' entries in the PPTT are either a CPU or a Processor +* Container, they may exist purely to describe a Private resource. CPUs +* have to be leaves, so a Processor Container is a non-leaf that has the +* 'ACPI Processor ID valid' flag set. +* +* Return: 0 for a complete walk, or an error if the mask is incomplete. +*/ +void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +struct acpi_table_header *table_hdr; +struct acpi_subtable_header *entry; +unsigned long table_end; +acpi_status status; +bool leaf_flag; +u32 proc_sz; + +cpumask_clear (cpus); + + status =acpi_get_table (ACPI_SIG_PPTT ,0 , &table_hdr); +if (ACPI_FAILURE (status)) +return ;
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 374
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+ + table_end = (unsigned long )table_hdr + table_hdr->length; + entry = ACPI_ADD_PTR(struct acpi_subtable_header, table_hdr, +sizeof (struct acpi_table_pptt )); + proc_sz =sizeof (struct acpi_pptt_processor ); +while ((unsigned long )entry + proc_sz <= table_end) { + cpu_node = (struct acpi_pptt_processor *)entry; +if (entry->type ==ACPI_PPTT_TYPE_PROCESSOR && + cpu_node->flags &ACPI_PPTT_ACPI_PROCESSOR_ID_VALID ) { + leaf_flag =acpi_pptt_leaf_node (table_hdr, cpu_node);
nit: Consider dropping the boolean leaf_flag and just using acpi_pptt_leaf_node() in the condition. The name leaf_flag is slightly overloaded to include the case when the acpi leaf flag is not supported and dropping it would make the code more succinct.
Quoted 28 lines
drivers/acpi/pptt.c +12
-0+if (!leaf_flag) { +if (cpu_node->acpi_processor_id == acpi_cpu_id) +acpi_pptt_get_child_cpus (table_hdr, cpu_node, cpus); + } + } + entry = ACPI_ADD_PTR(struct acpi_subtable_header, entry, + entry->length); + } + +acpi_put_table (table_hdr); +} +static u8 acpi_cache_type (enum cache_type type) {switch (type) {include/linux/acpi.h +1
-0@@ -1541,6 +1541,7 @@ int find_acpi_cpu_topology(unsigned int cpu, int level);int find_acpi_cpu_topology_cluster (unsigned int cpu);int find_acpi_cpu_topology_package (unsigned int cpu);int find_acpi_cpu_topology_hetero_id (unsigned int cpu); +void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus);#else static inline int acpi_pptt_cpu_is_thread (unsigned int cpu) {
include/linux/acpi.h static inline int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 1570
Quoted 8 lines
include/linux/acpi.h +2
-0@@ -1562,6 +1563,8 @@ static inline int find_acpi_cpu_topology_hetero_id(unsigned int cpu) {return -EINVAL ; } +static inline void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id, +cpumask_t *cpus) { }#endif void acpi_arch_init (void );
Thanks,
Ben
James Morse
maintainerRe: [PATCH 03/33] ACPI / PPTT: Add a helper to fill a cpumask from a processor container
Hi Ben,
Quoted 55 lines · Ben Horgan
The patch logic update makes sense to me. Just a nit.
Quoted 52 lines · James Morse
The PPTT describes CPUs and caches, as well as processor containers. The ACPI table for MPAM describes the set of CPUs that can access an MSC with the UID of a processor container.
Add a helper to find the processor container by its id, then walk the possible CPUs to fill a cpumask with the CPUs that have this processor container as a parent.
drivers/acpi/pptt.c +38
-0@@ -298,6 +298,92 @@ static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he +/** +* acpi_pptt_get_cpus_from_container() - Populate a cpumask with all CPUs in a +* processor containers +* @acpi_cpu_id: The UID of the processor container. +* @cpus: The resulting CPU mask. +* +* Find the specified Processor Container, and fill @cpus with all the cpus +* below it. +* +* Not all 'Processor' entries in the PPTT are either a CPU or a Processor +* Container, they may exist purely to describe a Private resource. CPUs +* have to be leaves, so a Processor Container is a non-leaf that has the +* 'ACPI Processor ID valid' flag set. +* +* Return: 0 for a complete walk, or an error if the mask is incomplete. +*/ +void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +struct acpi_table_header *table_hdr; +struct acpi_subtable_header *entry; +unsigned long table_end; +acpi_status status; +bool leaf_flag; +u32 proc_sz; + +cpumask_clear (cpus); + + status =acpi_get_table (ACPI_SIG_PPTT ,0 , &table_hdr); +if (ACPI_FAILURE (status)) +return ; + + table_end = (unsigned long )table_hdr + table_hdr->length; + entry = ACPI_ADD_PTR(struct acpi_subtable_header, table_hdr, +sizeof (struct acpi_table_pptt )); + proc_sz =sizeof (struct acpi_pptt_processor ); +while ((unsigned long )entry + proc_sz <= table_end) { + cpu_node = (struct acpi_pptt_processor *)entry;
Quoted 8 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 374
Quoted 3 lines
drivers/acpi/pptt.c +3
-0+if (entry->type ==ACPI_PPTT_TYPE_PROCESSOR && + cpu_node->flags &ACPI_PPTT_ACPI_PROCESSOR_ID_VALID ) { + leaf_flag =acpi_pptt_leaf_node (table_hdr, cpu_node);nit: Consider dropping the boolean
leaf_flagand just usingacpi_pptt_leaf_node()in the condition. The nameleaf_flagis slightly overloaded to include the case when the acpi leaf flag is not supported and dropping it would make the code more succinct.
Sure, this is a hangover from the earlier cleanup you suggested. It's readable enough without giving the result a name.
Thanks,
James
Re: [PATCH 03/33] ACPI / PPTT: Add a helper to fill a cpumask from a processor container
In reply to James MorseHi,
Quoted 7 lines · James Morse
The PPTT describes CPUs and caches, as well as processor containers. The ACPI table for MPAM describes the set of CPUs that can access an MSC with the UID of a processor container.
Add a helper to find the processor container by its id, then walk the possible CPUs to fill a cpumask with the CPUs that have this processor container as a parent.
Nit: The motivation for the change is not clear here.
I guess this boils down to the need to map the MSC topology information in the the ACPI MPAM table to a cpumask for each MSC.
If so, a possible rearrangement and rewording might be, say:
--8<--
The ACPI MPAM table uses the UID of a processor container specified in the PPTT, to indicate the subset of CPUs and upstream cache topology that can access each MPAM Memory System Component (MSC).
This information is not directly useful to the kernel. The equivalent cpumask is needed instead.
Add a helper to find the processor container by its id, then [...]
-->8--
Quoted 33 lines
CC Dave Martin <[email protected]>Reviewed-by Sudeep Holla <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* Droppedhas_leaf_flagdodging ofacpi_pptt_leaf_node()
* Added missing : in kernel-doc
* Made helper return void as this never actually returns an error.--- drivers/acpi/pptt.c | 86 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 3 ++ 2 files changed, 89 insertions(+)drivers/acpi/pptt.c +11
-0@@ -298,6 +298,92 @@ static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_hereturn NULL ; } +/** +* acpi_pptt_get_child_cpus() - Find all the CPUs below a PPTT processor node +* @table_hdr: A reference to the PPTT table. +* @parent_node: A pointer to the processor node in the @table_hdr. +* @cpus: A cpumask to fill with the CPUs below @parent_node. +* +* Walks up the PPTT from every possible CPU to find if the provided +* @parent_node is a parent of this CPU. +*/ +static void acpi_pptt_get_child_cpus(struct acpi_table_header *table_hdr, + struct acpi_pptt_processor *parent_node,
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 321
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+ cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +u32 acpi_id; +int cpu; + +cpumask_clear (cpus); + +for_each_possible_cpu (cpu) { + acpi_id =get_acpi_id_for_cpu (cpu);
^ Presumably this can't fail?
Quoted 5 lines
drivers/acpi/pptt.c +5
-0+ cpu_node =acpi_find_processor_node (table_hdr, acpi_id); + +while (cpu_node) { +if (cpu_node == parent_node) { +cpumask_set_cpu (cpu, cpus);
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 336
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+break ; + } + cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); + } + } +} + +/** +* acpi_pptt_get_cpus_from_container() - Populate a cpumask with all CPUs in a +* processor containers
Nit: "containers" -> "container" ?
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 346
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+* @acpi_cpu_id: The UID of the processor container. +* @cpus: The resulting CPU mask. +* +* Find the specified Processor Container, and fill @cpus with all the cpus +* below it. +* +* Not all 'Processor' entries in the PPTT are either a CPU or a Processor +* Container, they may exist purely to describe a Private resource. CPUs +* have to be leaves, so a Processor Container is a non-leaf that has the +* 'ACPI Processor ID valid' flag set.
(Revise this if dropping the leaf/non-leaf distinction -- see below.)
Quoted 8 lines
drivers/acpi/pptt.c +8
-0+* +* Return: 0 for a complete walk, or an error if the mask is incomplete. +*/ +void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +struct acpi_table_header *table_hdr; +struct acpi_subtable_header *entry;
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 364
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+unsigned long table_end; +acpi_status status; +bool leaf_flag; +u32 proc_sz; + +cpumask_clear (cpus); + + status =acpi_get_table (ACPI_SIG_PPTT ,0 , &table_hdr); +if (ACPI_FAILURE (status)) +return ;
Is acpi_get_pptt() applicable here?
(That function is not thread-safe, but then, perhaps most/all of these functions are not thread safe. If we are still on the boot CPU at this point (?) then this wouldn't be a concern.)
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 370
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+ + table_end = (unsigned long )table_hdr + table_hdr->length; + entry = ACPI_ADD_PTR(struct acpi_subtable_header, table_hdr, +sizeof (struct acpi_table_pptt )); + proc_sz =sizeof (struct acpi_pptt_processor ); +while ((unsigned long )entry + proc_sz <= table_end) {
Ack that this matches the bounds check in functions that are already present.
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 376
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+ cpu_node = (struct acpi_pptt_processor *)entry; +if (entry->type ==ACPI_PPTT_TYPE_PROCESSOR && + cpu_node->flags &ACPI_PPTT_ACPI_PROCESSOR_ID_VALID ) { + leaf_flag =acpi_pptt_leaf_node (table_hdr, cpu_node); +if (!leaf_flag) { +if (cpu_node->acpi_processor_id == acpi_cpu_id)
Is there any need to distinguish processor containers from (leaf) CPU nodes, here? If not, dropping the distinction might simplify the code here (even if callers do not care).
Otherwise, maybe eliminate leaf_flag and collapse these into a single if(), as suggested by Ben [1].
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 377
Quoted 1 line
drivers/acpi/pptt.c +1
-0+acpi_pptt_get_child_cpus (table_hdr, cpu_node, cpus);
Can there ever be multiple matches?
The possibility of duplicate processor IDs in the PPTT sounds weird to me, but then I'm not an ACPI expert.
If there can only be a single match, though, then we may as well break out of the loop here, unless we want to be paranoid and report duplicates as an error -- but that would require extra implementation, so I'm not sure that would be worth it.
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 385
Quoted 8 lines
drivers/acpi/pptt.c +8
-0+ } + } + entry = ACPI_ADD_PTR(struct acpi_subtable_header, entry, + entry->length); + } + +acpi_put_table (table_hdr); +}
[...]
[1] Ben Horgan, Re: [PATCH 03/33] ACPI / PPTT: Add a helper to fill a cpumask from a processor container
https://lore.kernel.org/lkml/[email protected]/
Cheers
---Dave
James Morse
maintainerRe: [PATCH 03/33] ACPI / PPTT: Add a helper to fill a cpumask from a processor container
Hi Dave,
Quoted 22 lines · Dave Martin
Quoted 7 lines · James Morse
The PPTT describes CPUs and caches, as well as processor containers. The ACPI table for MPAM describes the set of CPUs that can access an MSC with the UID of a processor container.
Add a helper to find the processor container by its id, then walk the possible CPUs to fill a cpumask with the CPUs that have this processor container as a parent.
Nit: The motivation for the change is not clear here.
I guess this boils down to the need to map the MSC topology information in the the ACPI MPAM table to a cpumask for each MSC.
If so, a possible rearrangement and rewording might be, say:
--8<--
The ACPI MPAM table uses the UID of a processor container specified in the PPTT, to indicate the subset of CPUs and upstream cache topology that can access each MPAM Memory System Component (MSC).
Quoted 6 lines
This information is not directly useful to the kernel. The equivalent cpumask is needed instead.
Add a helper to find the processor container by its id, then [...]
-->8--
Thanks, that is clearer!
Quoted 23 lines
Quoted 23 lines
drivers/acpi/pptt.c +15
-0@@ -298,6 +298,92 @@ static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_hereturn NULL ; } +/** +* acpi_pptt_get_child_cpus() - Find all the CPUs below a PPTT processor node +* @table_hdr: A reference to the PPTT table. +* @parent_node: A pointer to the processor node in the @table_hdr. +* @cpus: A cpumask to fill with the CPUs below @parent_node. +* +* Walks up the PPTT from every possible CPU to find if the provided +* @parent_node is a parent of this CPU. +*/ +static void acpi_pptt_get_child_cpus (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *parent_node, +cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +u32 acpi_id;
Quoted 8 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 321
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+int cpu; + +cpumask_clear (cpus); + +for_each_possible_cpu (cpu) { + acpi_id =get_acpi_id_for_cpu (cpu);^ Presumably this can't fail?
It'll return something! This could only be a problem if this raced with a CPU becoming impossible, and there is no mechanism to do that.
Quoted 9 lines
Quoted 9 lines
drivers/acpi/pptt.c +9
-0+ cpu_node =acpi_find_processor_node (table_hdr, acpi_id); + +while (cpu_node) { +if (cpu_node == parent_node) { +cpumask_set_cpu (cpu, cpus); +break ; + } + cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); + }
Quoted 8 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 336
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+ } +} + +/** +* acpi_pptt_get_cpus_from_container() - Populate a cpumask with all CPUs in a +* processor containers Nit: "containers" -> "container" ?
Fixed,
Quoted 25 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 346
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+* @acpi_cpu_id: The UID of the processor container. +* @cpus: The resulting CPU mask. +* +* Find the specified Processor Container, and fill @cpus with all the cpus +* below it. +* +* Not all 'Processor' entries in the PPTT are either a CPU or a Processor +* Container, they may exist purely to describe a Private resource. CPUs +* have to be leaves, so a Processor Container is a non-leaf that has the +* 'ACPI Processor ID valid' flag set. (Revise this if dropping the leaf/non-leaf distinction -- see below.)
Quoted 12 lines
drivers/acpi/pptt.c +12
-0+* +* Return: 0 for a complete walk, or an error if the mask is incomplete. +*/ +void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +struct acpi_table_header *table_hdr; +struct acpi_subtable_header *entry; +unsigned long table_end; +acpi_status status; +bool leaf_flag; +u32 proc_sz;
Quoted 8 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 364
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+ +cpumask_clear (cpus); + + status =acpi_get_table (ACPI_SIG_PPTT ,0 , &table_hdr); +if (ACPI_FAILURE (status)) +return ;Is
acpi_get_pptt()applicable here?
Oh, that is new, and would let me chuck the reference counting. I guess this replaces Jonthan's magic table free'ing cleanup thing!
Quoted 3 lines
(That function is not thread-safe, but then, perhaps most/all of these functions are not thread safe. If we are still on the boot CPU at this point (?) then this wouldn't be a concern.)
I think that relies on the first caller being from somewhere that can't race. In this case its the architecture's smp_prepare_cpus() call to setup the acpi topology. That is sufficiently early its not a concern.
Quoted 10 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 370
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+ + table_end = (unsigned long )table_hdr + table_hdr->length; + entry = ACPI_ADD_PTR(struct acpi_subtable_header, table_hdr, +sizeof (struct acpi_table_pptt )); + proc_sz =sizeof (struct acpi_pptt_processor ); +while ((unsigned long )entry + proc_sz <= table_end) {Ack that this matches the bounds check in functions that are already present.
Quoted 11 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 376
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+ cpu_node = (struct acpi_pptt_processor *)entry; +if (entry->type ==ACPI_PPTT_TYPE_PROCESSOR && + cpu_node->flags &ACPI_PPTT_ACPI_PROCESSOR_ID_VALID ) { + leaf_flag =acpi_pptt_leaf_node (table_hdr, cpu_node); +if (!leaf_flag) { +if (cpu_node->acpi_processor_id == acpi_cpu_id)Is there any need to distinguish processor containers from (leaf) CPU nodes, here? If not, dropping the distinction might simplify the code here (even if callers do not care).
In the namespace the object types are different, so I assumed they have their own UID space. The PPTT holds both - hence the check for which kind of thing it is. The risk is looking for processor-container-4 and finding CPU-4 instead...
The relevant ACPI bit is "8.4.2.1 Processor Container Device", its says:
| A processor container declaration must supply a _UID method returning an ID that is | unique in the processor container hierarchy.
Which doesn't quite let me combine them here.
Quoted 9 lines
Otherwise, maybe eliminate
leaf_flagand collapse these into a singleif(), as suggested by Ben [1].drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 377
Quoted 1 line
drivers/acpi/pptt.c +1
-0+acpi_pptt_get_child_cpus (table_hdr, cpu_node, cpus);Can there ever be multiple matches?
The possibility of duplicate processor IDs in the PPTT sounds weird to me, but then I'm not an ACPI expert.
Multiple processor-containers with the same ID? That would be a corrupt table. acpi_pptt_get_child_cpus() then walks the tree again to find the CPUs below this processor-container - those have a different kind of id.
Quoted 4 lines
If there can only be a single match, though, then we may as well break out of the loop here, unless we want to be paranoid and report duplicates as an error -- but that would require extra implementation, so I'm not sure that would be worth it.
Hmmm, the PPTT node should map to only one processor or processor-container. I'll chuck the break in.
Thanks,
James
Re: [PATCH 03/33] ACPI / PPTT: Add a helper to fill a cpumask from a processor container
Hi James,
Quoted 25 lines · James Morse
Hi Dave,
Quoted 22 lines · Dave Martin
Quoted 7 lines · James Morse
The PPTT describes CPUs and caches, as well as processor containers. The ACPI table for MPAM describes the set of CPUs that can access an MSC with the UID of a processor container.
Add a helper to find the processor container by its id, then walk the possible CPUs to fill a cpumask with the CPUs that have this processor container as a parent.
Nit: The motivation for the change is not clear here.
I guess this boils down to the need to map the MSC topology information in the the ACPI MPAM table to a cpumask for each MSC.
If so, a possible rearrangement and rewording might be, say:
--8<--
The ACPI MPAM table uses the UID of a processor container specified in the PPTT, to indicate the subset of CPUs and upstream cache topology that can access each MPAM Memory System Component (MSC).
Quoted 8 lines
Quoted 6 lines
This information is not directly useful to the kernel. The equivalent cpumask is needed instead.
Add a helper to find the processor container by its id, then [...]
-->8--
Thanks, that is clearer!
Thanks
Quoted 1 line
Quoted 1 line
Quoted 1 line
drivers/acpi/pptt.c +0
-0
[...]
Quoted 1 line
Quoted 1 line
Quoted 1 line
drivers/acpi/pptt.c +0
-0@@ -298,6 +298,92 @@ static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he
[...]
Quoted 15 lines
Quoted 6 lines
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+static void acpi_pptt_get_child_cpus (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *parent_node, +cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +u32 acpi_id;Quoted 8 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 321
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+int cpu; + +cpumask_clear (cpus); + +for_each_possible_cpu (cpu) { + acpi_id =get_acpi_id_for_cpu (cpu);^ Presumably this can't fail?
Quoted 2 lines
It'll return something! This could only be a problem if this raced with a CPU becoming impossible, and there is no mechanism to do that.
Yep, now I go and look more closely at that function, my question looks misguided.
[...]
Quoted 18 lines
Quoted 9 lines
Quoted 9 lines
drivers/acpi/pptt.c +9
-0+void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +struct acpi_table_header *table_hdr; +struct acpi_subtable_header *entry; +unsigned long table_end; +acpi_status status; +bool leaf_flag; +u32 proc_sz;Quoted 8 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 364
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+ +cpumask_clear (cpus); + + status =acpi_get_table (ACPI_SIG_PPTT ,0 , &table_hdr); +if (ACPI_FAILURE (status)) +return ;Is
acpi_get_pptt()applicable here?
Quoted 2 lines
Oh, that is new, and would let me chuck the reference counting. I guess this replaces Jonthan's magic table free'ing cleanup thing!
Ah, rightho.
Quoted 7 lines
Quoted 3 lines
(That function is not thread-safe, but then, perhaps most/all of these functions are not thread safe. If we are still on the boot CPU at this point (?) then this wouldn't be a concern.)
I think that relies on the first caller being from somewhere that can't race. In this case its the architecture's
smp_prepare_cpus()call to setup the acpi topology. That is sufficiently early its not a concern.
I guess so.
[...]
Quoted 16 lines
Quoted 11 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 376
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+ cpu_node = (struct acpi_pptt_processor *)entry; +if (entry->type ==ACPI_PPTT_TYPE_PROCESSOR && + cpu_node->flags &ACPI_PPTT_ACPI_PROCESSOR_ID_VALID ) { + leaf_flag =acpi_pptt_leaf_node (table_hdr, cpu_node); +if (!leaf_flag) { +if (cpu_node->acpi_processor_id == acpi_cpu_id)Is there any need to distinguish processor containers from (leaf) CPU nodes, here? If not, dropping the distinction might simplify the code here (even if callers do not care).
In the namespace the object types are different, so I assumed they have their own UID space. The PPTT holds both - hence the check for which kind of thing it is. The risk is looking for processor-container-4 and finding CPU-4 instead...
Quoted 5 lines
The relevant ACPI bit is "8.4.2.1 Processor Container Device", its says:
| A processor container declaration must supply a_UIDmethod returning an ID that is | unique in the processor container hierarchy.Which doesn't quite let me combine them here.
I was going by the PPTT spec, where the types are not distinct -- you're probably right, though.
According to that, isn't it the "ACPI Processor ID valid" flag, not the "Node is a Leaf" flag, that says whether this field is meaningful?
It's reasonable not to bother to try to enumerate the children of a node that claims to be a leaf (even if there actually are children), but I wonder what happens if acpi_processor_id is not declared to be valid and matches by accident. That's probably not a valid table (?) but does anything bad happen on the kernel side?
Quoted 7 lines
Quoted 7 lines
Otherwise, maybe eliminate
leaf_flagand collapse these into a singleif(), as suggested by Ben [1].drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 377
Quoted 1 line
drivers/acpi/pptt.c +1
-0+acpi_pptt_get_child_cpus (table_hdr, cpu_node, cpus);Can there ever be multiple matches?
Quoted 6 lines
Quoted 2 lines
The possibility of duplicate processor IDs in the PPTT sounds weird to me, but then I'm not an ACPI expert.
Multiple processor-containers with the same ID? That would be a corrupt table.
acpi_pptt_get_child_cpus()then walks the tree again to find the CPUs below this processor-container - those have a different kind of id.
Does anything bad happen if we encounter duplicates?
(Other then the MPAM driver never getting enabled, or not working as advertised, that is.)
I haven't tried to think through all the implications, here.
Quoted 7 lines
Quoted 4 lines
If there can only be a single match, though, then we may as well break out of the loop here, unless we want to be paranoid and report duplicates as an error -- but that would require extra implementation, so I'm not sure that would be worth it.
Hmmm, the PPTT node should map to only one processor or processor-container. I'll chuck the break in.
Ack
Cheers
---Dave
James Morse
maintainerRe: [PATCH 03/33] ACPI / PPTT: Add a helper to fill a cpumask from a processor container
Hi Dave,
Quoted 29 lines · Dave Martin
Quoted 28 lines · James Morse
Quoted 11 lines · Dave Martin
Quoted 9 lines · James Morse
The PPTT describes CPUs and caches, as well as processor containers. The ACPI table for MPAM describes the set of CPUs that can access an MSC with the UID of a processor container.
drivers/acpi/pptt.c +0
-0@@ -298,6 +298,92 @@ static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he +void acpi_pptt_get_cpus_from_container(u32 acpi_cpu_id, cpumask_t *cpus) +{Quoted 11 lines
drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 376
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+ cpu_node = (struct acpi_pptt_processor *)entry; +if (entry->type ==ACPI_PPTT_TYPE_PROCESSOR && + cpu_node->flags &ACPI_PPTT_ACPI_PROCESSOR_ID_VALID ) { + leaf_flag =acpi_pptt_leaf_node (table_hdr, cpu_node); +if (!leaf_flag) { +if (cpu_node->acpi_processor_id == acpi_cpu_id)Is there any need to distinguish processor containers from (leaf) CPU nodes, here? If not, dropping the distinction might simplify the code here (even if callers do not care).
In the namespace the object types are different, so I assumed they have their own UID space. The PPTT holds both - hence the check for which kind of thing it is. The risk is looking for processor-container-4 and finding CPU-4 instead...
Quoted 8 lines
Quoted 5 lines
The relevant ACPI bit is "8.4.2.1 Processor Container Device", its says:
| A processor container declaration must supply a_UIDmethod returning an ID that is | unique in the processor container hierarchy.Which doesn't quite let me combine them here.
I was going by the PPTT spec, where the types are not distinct -- you're probably right, though.
This way round is at least robust to this happening.
Quoted 2 lines
According to that, isn't it the "ACPI Processor ID valid" flag, not the "Node is a Leaf" flag, that says whether this field is meaningful?
ACPI_PPTT_ACPI_PROCESSOR_ID_VALID was checked a few lines earlier. We're looking for processors, hence also checking the leaf.
Quoted 5 lines
It's reasonable not to bother to try to enumerate the children of a node that claims to be a leaf (even if there actually are children), but I wonder what happens if
acpi_processor_idis not declared to be valid and matches by accident. That's probably not a valid table (?) but does anything bad happen on the kernel side?
The type and flag are both checked earlier, so this can't happen.
You could certainly but junk nodes in the table that would be skipped over, and those could point to a parent that is a leaf - I can't spot anything in the table parsing code that would care about that.
Quoted 14 lines
Quoted 7 lines
Quoted 7 lines
Otherwise, maybe eliminate
leaf_flagand collapse these into a singleif(), as suggested by Ben [1].drivers/acpi/pptt.c static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_he · line 377
Quoted 1 line
drivers/acpi/pptt.c +1
-0+acpi_pptt_get_child_cpus (table_hdr, cpu_node, cpus);Can there ever be multiple matches?
Quoted 6 lines
Quoted 2 lines
The possibility of duplicate processor IDs in the PPTT sounds weird to me, but then I'm not an ACPI expert.
Multiple processor-containers with the same ID? That would be a corrupt table.
acpi_pptt_get_child_cpus()then walks the tree again to find the CPUs below this processor-container - those have a different kind of id.
Quoted 6 lines
Does anything bad happen if we encounter duplicates?
(Other then the MPAM driver never getting enabled, or not working as advertised, that is.)
I haven't tried to think through all the implications, here.
It would be unpredictable which node linux finds when it goes looking for CPUs. I don't think anything would notice. Messing up the cache hierarchy is a different story!
Thanks,
James
James Morse
maintainerPATCH 3/33 ACPI / PPTT: Add a helper to fill a cpumask from a processor container
In reply to James MorseThe PPTT describes CPUs and caches, as well as processor containers. The ACPI table for MPAM describes the set of CPUs that can access an MSC with the UID of a processor container.
Add a helper to find the processor container by its id, then walk the possible CPUs to fill a cpumask with the CPUs that have this processor container as a parent.
---
Changes since RFC:
* Dropped has_leaf_flag dodging of acpi_pptt_leaf_node()
* Added missing : in kernel-doc
* Made helper return void as this never actually returns an error.
--- drivers/acpi/pptt.c | 86 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 3 ++ 2 files changed, 89 insertions(+)
drivers/acpi/pptt.c +86 -0
@@ -298,6 +298,92 @@ static struct acpi_pptt_processor *acpi_find_processor_node(struct acpi_table_hereturn NULL ; } +/** +* acpi_pptt_get_child_cpus() - Find all the CPUs below a PPTT processor node +* @table_hdr: A reference to the PPTT table. +* @parent_node: A pointer to the processor node in the @table_hdr. +* @cpus: A cpumask to fill with the CPUs below @parent_node. +* +* Walks up the PPTT from every possible CPU to find if the provided +* @parent_node is a parent of this CPU. +*/ +static void acpi_pptt_get_child_cpus (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *parent_node, +cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +u32 acpi_id; +int cpu; + +cpumask_clear (cpus); + +for_each_possible_cpu (cpu) { + acpi_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table_hdr, acpi_id); + +while (cpu_node) { +if (cpu_node == parent_node) { +cpumask_set_cpu (cpu, cpus); +break ; + } + cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); + } + } +} + +/** +* acpi_pptt_get_cpus_from_container() - Populate a cpumask with all CPUs in a +* processor containers +* @acpi_cpu_id: The UID of the processor container. +* @cpus: The resulting CPU mask. +* +* Find the specified Processor Container, and fill @cpus with all the cpus +* below it. +* +* Not all 'Processor' entries in the PPTT are either a CPU or a Processor +* Container, they may exist purely to describe a Private resource. CPUs +* have to be leaves, so a Processor Container is a non-leaf that has the +* 'ACPI Processor ID valid' flag set. +* +* Return: 0 for a complete walk, or an error if the mask is incomplete. +*/ +void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) +{ +struct acpi_pptt_processor *cpu_node; +struct acpi_table_header *table_hdr; +struct acpi_subtable_header *entry; +unsigned long table_end; +acpi_status status; +bool leaf_flag; +u32 proc_sz; + +cpumask_clear (cpus); + + status =acpi_get_table (ACPI_SIG_PPTT ,0 , &table_hdr); +if (ACPI_FAILURE (status)) +return ; + + table_end = (unsigned long )table_hdr + table_hdr->length; + entry = ACPI_ADD_PTR(struct acpi_subtable_header, table_hdr, +sizeof (struct acpi_table_pptt )); + proc_sz =sizeof (struct acpi_pptt_processor ); +while ((unsigned long )entry + proc_sz <= table_end) { + cpu_node = (struct acpi_pptt_processor *)entry; +if (entry->type ==ACPI_PPTT_TYPE_PROCESSOR && + cpu_node->flags &ACPI_PPTT_ACPI_PROCESSOR_ID_VALID ) { + leaf_flag =acpi_pptt_leaf_node (table_hdr, cpu_node); +if (!leaf_flag) { +if (cpu_node->acpi_processor_id == acpi_cpu_id) +acpi_pptt_get_child_cpus (table_hdr, cpu_node, cpus); + } + } + entry = ACPI_ADD_PTR(struct acpi_subtable_header, entry, + entry->length); + } + +acpi_put_table (table_hdr); +} +static u8 acpi_cache_type (enum cache_type type) {switch (type) {
include/linux/acpi.h +3 -0
@@ -1541,6 +1541,7 @@ int find_acpi_cpu_topology(unsigned int cpu, int level);int find_acpi_cpu_topology_cluster (unsigned int cpu);int find_acpi_cpu_topology_package (unsigned int cpu);int find_acpi_cpu_topology_hetero_id (unsigned int cpu); +void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus);#else static inline int acpi_pptt_cpu_is_thread (unsigned int cpu) {
@@ -1562,6 +1563,8 @@ static inline int find_acpi_cpu_topology_hetero_id(unsigned int cpu) {return -EINVAL ; } +static inline void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id, +cpumask_t *cpus) { }#endif void acpi_arch_init (void );
Signature · 1 line
2.20.1
James Morse
maintainerPATCH 4/33 ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear levels
In reply to James Morseacpi_count_levels() passes the number of levels back via a pointer argument. It also passes this to acpi_find_cache_level() as the starting_level, and preserves this value as it walks up the cpu_node tree counting the levels.
This means the caller must initialise 'levels' due to acpi_count_levels() internals. The only caller acpi_get_cache_info() happens to have already initialised levels to zero, which acpi_count_levels() depends on to get the correct result.
Two results are passed back from acpi_count_levels(), unlike split_levels, levels is not optional.
Split these two results up. The mandatory 'levels' is always returned, which hides the internal details from the caller, and avoids having duplicated initialisation in all callers. split_levels remains an optional argument passed back.
---
Changes since RFC:
* Made acpi_count_levels() return the levels value.
--- drivers/acpi/pptt.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
drivers/acpi/pptt.c +11 -7
@@ -181,10 +181,10 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* levels and split cache levels (data/instruction). * @table_hdr: Pointer to the head of the PPTT table * @cpu_node: processor node we wish to count caches for -* @levels: Number of levels if success. * @split_levels: Number of split cache levels (data/instruction) if -* success. Can by NULL. +* success. Can be NULL. * +* Returns number of levels. * Given a processor node containing a processing unit, walk into it and count * how many levels exist solely for it, and then walk up each level until we hit * the root node (ignore the package level because it may be possible to have
@@ -192,14 +192,18 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* split cache levels (data/instruction) that exist at each level on the way * up. */ -static void acpi_count_levels (struct acpi_table_header *table_hdr, -struct acpi_pptt_processor *cpu_node, -unsigned int *levels,unsigned int *split_levels) +static int acpi_count_levels (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *cpu_node, +unsigned int *split_levels) { +int starting_level =0 ; +do { -acpi_find_cache_level (table_hdr, cpu_node, levels, split_levels,0 ,0 ); +acpi_find_cache_level (table_hdr, cpu_node, &starting_level, split_levels,0 ,0 ); cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); }while (cpu_node); + +return starting_level; }/**
@@ -731,7 +735,7 @@ int acpi_get_cache_info(unsigned int cpu, unsigned int *levels,if (!cpu_node)return -ENOENT ; -acpi_count_levels (table, cpu_node, levels, split_levels); + *levels =acpi_count_levels (table, cpu_node, split_levels);pr_debug ("Cache Setup: last_level=%d split_levels=%d\n" , *levels, split_levels ? *split_levels :-1 );
Signature · 1 line
2.20.1
Re: [PATCH 04/33] ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear levels
Hi,
Quoted 9 lines · James Morse
acpi_count_levels()passes the number of levels back via a pointer argument. It also passes this toacpi_find_cache_level()as thestarting_level, and preserves this value as it walks up thecpu_nodetree counting the levels.This means the caller must initialise 'levels' due to
acpi_count_levels()internals. The only calleracpi_get_cache_info()happens to have already initialised levels to zero, whichacpi_count_levels()depends on to get the correct result.
Quoted 7 lines
Two results are passed back from
acpi_count_levels(), unlikesplit_levels, levels is not optional.Split these two results up. The mandatory 'levels' is always returned, which hides the internal details from the caller, and avoids having duplicated initialisation in all callers.
split_levelsremains an optional argument passed back.
Nit: I found all this a bit hard to follow.
This seems to boil down to:
--8<--
In acpi_count_levels(), the initial value of *levels passed by the caller is really an implementation detail of acpi_count_levels(), so it is unreasonable to expect the callers of this function to know what to pass in for this parameter. The only sensible initial value is 0, which is what the only upstream caller (acpi_get_cache_info()) passes.
Use a local variable for the starting cache level in acpi_count_levels(), and pass the result back to the caller via the function return value.
Gid rid of the levels parameter, which has no remaining purpose.
Fix acpi_get_cache_info() to match.
-->8--
split_levels is orthogonal to this refactoring (as evinced by the diff). I think mentioning it in the commit message at all may just add to the confusion...
Quoted 15 lines
Suggested-by Jonathan Cameron <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* Madeacpi_count_levels()return the levels value.--- drivers/acpi/pptt.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)drivers/acpi/pptt.c +0
-0
drivers/acpi/pptt.c acpi_find_cache_level(struct acpi_table_header *table_hdr, · line 187
Quoted 9 lines
drivers/acpi/pptt.c +2
-2@@ -181,10 +181,10 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* levels and split cache levels (data/instruction). * @table_hdr: Pointer to the head of the PPTT table * @cpu_node: processor node we wish to count caches for -* @levels: Number of levels if success. * @split_levels: Number of split cache levels (data/instruction) if -* success. Can by NULL. +* success. Can be NULL. * +* Returns number of levels.
Nit: the prevailing convention in this file would be
Return: number of levels
(I don't know whether kerneldoc cares.)
Maybe also say "total number of levels" in place of "level", to make it clearer that the split levels (if any) are included in this count.
Quoted 27 lines
drivers/acpi/pptt.c +8
-4* Given a processor node containing a processing unit, walk into it and count * how many levels exist solely for it, and then walk up each level until we hit * the root node (ignore the package level because it may be possible to have @@ -192,14 +192,18 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* split cache levels (data/instruction) that exist at each level on the way * up. */ -static void acpi_count_levels (struct acpi_table_header *table_hdr, -struct acpi_pptt_processor *cpu_node, -unsigned int *levels,unsigned int *split_levels) +static int acpi_count_levels (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *cpu_node, +unsigned int *split_levels) { +int starting_level =0 ; +do { -acpi_find_cache_level (table_hdr, cpu_node, levels, split_levels,0 ,0 ); +acpi_find_cache_level (table_hdr, cpu_node, &starting_level, split_levels,0 ,0 ); cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); }while (cpu_node); + +return starting_level; }/**
drivers/acpi/pptt.c int acpi_get_cache_info(unsigned int cpu, unsigned int *levels, · line 741
Quoted 8 lines
drivers/acpi/pptt.c +1
-1@@ -731,7 +735,7 @@ int acpi_get_cache_info(unsigned int cpu, unsigned int *levels,if (!cpu_node)return -ENOENT ; -acpi_count_levels (table, cpu_node, levels, split_levels); + *levels =acpi_count_levels (table, cpu_node, split_levels);pr_debug ("Cache Setup: last_level=%d split_levels=%d\n" , *levels, split_levels ? *split_levels :-1 );
Otherwise, looks reasonable to me.
(But see my comments on the next patches re whether we really need this.)
Cheers
---Dave
James Morse
maintainerRe: [PATCH 04/33] ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear levels
Hi Dave,
Quoted 30 lines · Dave Martin
Quoted 9 lines · James Morse
acpi_count_levels()passes the number of levels back via a pointer argument. It also passes this toacpi_find_cache_level()as thestarting_level, and preserves this value as it walks up thecpu_nodetree counting the levels.This means the caller must initialise 'levels' due to
acpi_count_levels()internals. The only calleracpi_get_cache_info()happens to have already initialised levels to zero, whichacpi_count_levels()depends on to get the correct result.Quoted 7 lines
Two results are passed back from
acpi_count_levels(), unlikesplit_levels, levels is not optional.Split these two results up. The mandatory 'levels' is always returned, which hides the internal details from the caller, and avoids having duplicated initialisation in all callers.
split_levelsremains an optional argument passed back.Nit: I found all this a bit hard to follow.
This seems to boil down to:
--8<--
In
acpi_count_levels(), the initial value of *levels passed by the caller is really an implementation detail ofacpi_count_levels(), so it is unreasonable to expect the callers of this function to know what to pass in for this parameter. The only sensible initial value is 0, which is what the only upstream caller (acpi_get_cache_info()) passes.
Quoted 8 lines
Use a local variable for the starting cache level in
acpi_count_levels(), and pass the result back to the caller via the function return value.Gid rid of the levels parameter, which has no remaining purpose.
Fix
acpi_get_cache_info()to match.-->8--
I've taken this instead,
Quoted 21 lines
split_levelsis orthogonal to this refactoring (as evinced by the diff). I think mentioning it in the commit message at all may just add to the confusion...Quoted 5 lines
drivers/acpi/pptt.c +0
-0drivers/acpi/pptt.c acpi_find_cache_level(struct acpi_table_header *table_hdr, · line 187
Quoted 9 lines
drivers/acpi/pptt.c +2
-2@@ -181,10 +181,10 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* levels and split cache levels (data/instruction). * @table_hdr: Pointer to the head of the PPTT table * @cpu_node: processor node we wish to count caches for -* @levels: Number of levels if success. * @split_levels: Number of split cache levels (data/instruction) if -* success. Can by NULL. +* success. Can be NULL. * +* Returns number of levels. Nit: the prevailing convention in this file would be
Quoted 6 lines
Return: number of levels
(I don't know whether kerneldoc cares.)
Maybe also say "total number of levels" in place of "level", to make it clearer that the split levels (if any) are included in this count.
Sure,
Quoted 7 lines
Quoted 7 lines
drivers/acpi/pptt.c +1
-1@@ -731,7 +735,7 @@ int acpi_get_cache_info(unsigned int cpu, unsigned int *levels,if (!cpu_node)return -ENOENT ; -acpi_count_levels (table, cpu_node, levels, split_levels); + *levels =acpi_count_levels (table, cpu_node, split_levels);
Quoted 6 lines
drivers/acpi/pptt.c int acpi_get_cache_info(unsigned int cpu, unsigned int *levels, · line 741
Quoted 2 lines
drivers/acpi/pptt.c +0
-0pr_debug ("Cache Setup: last_level=%d split_levels=%d\n" , *levels, split_levels ? *split_levels :-1 );Otherwise, looks reasonable to me.
(But see my comments on the next patches re whether we really need this.)
It was enough fun to debug that I'd like to save anyone else the trouble!
Thanks,
James
Re: [PATCH 04/33] ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear levels
Hi James,
Quoted 36 lines · James Morse
Hi Dave,
Quoted 33 lines · Dave Martin
Quoted 9 lines · James Morse
acpi_count_levels()passes the number of levels back via a pointer argument. It also passes this toacpi_find_cache_level()as thestarting_level, and preserves this value as it walks up thecpu_nodetree counting the levels.This means the caller must initialise 'levels' due to
acpi_count_levels()internals. The only calleracpi_get_cache_info()happens to have already initialised levels to zero, whichacpi_count_levels()depends on to get the correct result.Quoted 7 lines
Two results are passed back from
acpi_count_levels(), unlikesplit_levels, levels is not optional.Split these two results up. The mandatory 'levels' is always returned, which hides the internal details from the caller, and avoids having duplicated initialisation in all callers.
split_levelsremains an optional argument passed back.Nit: I found all this a bit hard to follow.
This seems to boil down to:
--8<--
In
acpi_count_levels(), the initial value of *levels passed by the caller is really an implementation detail ofacpi_count_levels(), so it is unreasonable to expect the callers of this function to know what to pass in for this parameter. The only sensible initial value is 0, which is what the only upstream caller (acpi_get_cache_info()) passes.Use a local variable for the starting cache level in
acpi_count_levels(), and pass the result back to the caller via the function return value.
Quoted 7 lines
Quoted 5 lines
Gid rid of the levels parameter, which has no remaining purpose.
Fix
acpi_get_cache_info()to match.-->8--
I've taken this instead,
OK
[...]
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/acpi/pptt.c +1
-1@@ -731,7 +735,7 @@ int acpi_get_cache_info(unsigned int cpu, unsigned int *levels,if (!cpu_node)return -ENOENT ; -acpi_count_levels (table, cpu_node, levels, split_levels); + *levels =acpi_count_levels (table, cpu_node, split_levels);
Quoted 8 lines
Quoted 6 lines
drivers/acpi/pptt.c int acpi_get_cache_info(unsigned int cpu, unsigned int *levels, · line 741
Quoted 2 lines
drivers/acpi/pptt.c +0
-0pr_debug ("Cache Setup: last_level=%d split_levels=%d\n" , *levels, split_levels ? *split_levels :-1 );Otherwise, looks reasonable to me.
(But see my comments on the next patches re whether we really need this.)
It was enough fun to debug that I'd like to save anyone else the trouble!
Fair enough.
Cheers
---Dave
James Morse
maintainerPATCH 4/33 ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear levels
In reply to James Morseacpi_count_levels() passes the number of levels back via a pointer argument. It also passes this to acpi_find_cache_level() as the starting_level, and preserves this value as it walks up the cpu_node tree counting the levels.
This means the caller must initialise 'levels' due to acpi_count_levels() internals. The only caller acpi_get_cache_info() happens to have already initialised levels to zero, which acpi_count_levels() depends on to get the correct result.
Two results are passed back from acpi_count_levels(), unlike split_levels, levels is not optional.
Split these two results up. The mandatory 'levels' is always returned, which hides the internal details from the caller, and avoids having duplicated initialisation in all callers. split_levels remains an optional argument passed back.
---
Changes since RFC:
* Made acpi_count_levels() return the levels value.
--- drivers/acpi/pptt.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
drivers/acpi/pptt.c +11 -7
@@ -181,10 +181,10 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* levels and split cache levels (data/instruction). * @table_hdr: Pointer to the head of the PPTT table * @cpu_node: processor node we wish to count caches for -* @levels: Number of levels if success. * @split_levels: Number of split cache levels (data/instruction) if -* success. Can by NULL. +* success. Can be NULL. * +* Returns number of levels. * Given a processor node containing a processing unit, walk into it and count * how many levels exist solely for it, and then walk up each level until we hit * the root node (ignore the package level because it may be possible to have
@@ -192,14 +192,18 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* split cache levels (data/instruction) that exist at each level on the way * up. */ -static void acpi_count_levels (struct acpi_table_header *table_hdr, -struct acpi_pptt_processor *cpu_node, -unsigned int *levels,unsigned int *split_levels) +static int acpi_count_levels (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *cpu_node, +unsigned int *split_levels) { +int starting_level =0 ; +do { -acpi_find_cache_level (table_hdr, cpu_node, levels, split_levels,0 ,0 ); +acpi_find_cache_level (table_hdr, cpu_node, &starting_level, split_levels,0 ,0 ); cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); }while (cpu_node); + +return starting_level; }/**
@@ -731,7 +735,7 @@ int acpi_get_cache_info(unsigned int cpu, unsigned int *levels,if (!cpu_node)return -ENOENT ; -acpi_count_levels (table, cpu_node, levels, split_levels); + *levels =acpi_count_levels (table, cpu_node, split_levels);pr_debug ("Cache Setup: last_level=%d split_levels=%d\n" , *levels, split_levels ? *split_levels :-1 );
Signature · 1 line
2.20.1
Re: [PATCH 04/33] ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear levels
Quoted 32 lines · James Morse
acpi_count_levels()passes the number of levels back via a pointer argument. It also passes this toacpi_find_cache_level()as thestarting_level, and preserves this value as it walks up thecpu_nodetree counting the levels.This means the caller must initialise 'levels' due to
acpi_count_levels()internals. The only calleracpi_get_cache_info()happens to have already initialised levels to zero, whichacpi_count_levels()depends on to get the correct result.Two results are passed back from
acpi_count_levels(), unlikesplit_levels, levels is not optional.Split these two results up. The mandatory 'levels' is always returned, which hides the internal details from the caller, and avoids having duplicated initialisation in all callers.
split_levelsremains an optional argument passed back.Suggested-by Jonathan Cameron <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* Madeacpi_count_levels()return the levels value.--- drivers/acpi/pptt.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)drivers/acpi/pptt.c +0
-0
drivers/acpi/pptt.c acpi_find_cache_level(struct acpi_table_header *table_hdr, · line 185
Quoted 7 lines
drivers/acpi/pptt.c +1
-2@@ -181,10 +181,10 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* levels and split cache levels (data/instruction). * @table_hdr: Pointer to the head of the PPTT table * @cpu_node: processor node we wish to count caches for -* @levels: Number of levels if success. * @split_levels: Number of split cache levels (data/instruction) if -* success. Can by NULL. +* success. Can be NULL.
Nit: tempting but this change does not belong here.
Quoted 29 lines
drivers/acpi/pptt.c +9
-4* +* Returns number of levels. * Given a processor node containing a processing unit, walk into it and count * how many levels exist solely for it, and then walk up each level until we hit * the root node (ignore the package level because it may be possible to have @@ -192,14 +192,18 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* split cache levels (data/instruction) that exist at each level on the way * up. */ -static void acpi_count_levels (struct acpi_table_header *table_hdr, -struct acpi_pptt_processor *cpu_node, -unsigned int *levels,unsigned int *split_levels) +static int acpi_count_levels (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *cpu_node, +unsigned int *split_levels) { +int starting_level =0 ; +do { -acpi_find_cache_level (table_hdr, cpu_node, levels, split_levels,0 ,0 ); +acpi_find_cache_level (table_hdr, cpu_node, &starting_level, split_levels,0 ,0 ); cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); }while (cpu_node); + +return starting_level; }/**
drivers/acpi/pptt.c int acpi_get_cache_info(unsigned int cpu, unsigned int *levels, · line 738
Quoted 5 lines
drivers/acpi/pptt.c +1
-1@@ -731,7 +735,7 @@ int acpi_get_cache_info(unsigned int cpu, unsigned int *levels,if (!cpu_node)return -ENOENT ; -acpi_count_levels (table, cpu_node, levels, split_levels); + *levels =acpi_count_levels (table, cpu_node, split_levels);
Looks fine to me - though initializing
*levels = 0
upper in the function now becomes superfluous (?) (well, it initializes *levels to 0 if an error path is hit but on that case the caller should not expect *levels to be initialized to anything IIUC).
Apart from these (very) minor things:
Quoted 5 lines
drivers/acpi/pptt.c +0
-0pr_debug ("Cache Setup: last_level=%d split_levels=%d\n" , *levels, split_levels ? *split_levels :-1 );Signature · 1 line
2.20.1
James Morse
maintainerRe: [PATCH 04/33] ACPI / PPTT: Stop acpi_count_levels() expecting callers to clear levels
Hi Lorenzo,
Quoted 18 lines · Lorenzo Pieralisi
Quoted 16 lines · James Morse
acpi_count_levels()passes the number of levels back via a pointer argument. It also passes this toacpi_find_cache_level()as thestarting_level, and preserves this value as it walks up thecpu_nodetree counting the levels.This means the caller must initialise 'levels' due to
acpi_count_levels()internals. The only calleracpi_get_cache_info()happens to have already initialised levels to zero, whichacpi_count_levels()depends on to get the correct result.Two results are passed back from
acpi_count_levels(), unlikesplit_levels, levels is not optional.Split these two results up. The mandatory 'levels' is always returned, which hides the internal details from the caller, and avoids having duplicated initialisation in all callers.
split_levelsremains an optional argument passed back.
Quoted 14 lines
drivers/acpi/pptt.c acpi_find_cache_level(struct acpi_table_header *table_hdr, · line 185
Quoted 12 lines
drivers/acpi/pptt.c +1
-2@@ -181,10 +181,10 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* levels and split cache levels (data/instruction). * @table_hdr: Pointer to the head of the PPTT table * @cpu_node: processor node we wish to count caches for -* @levels: Number of levels if success. * @split_levels: Number of split cache levels (data/instruction) if -* success. Can by NULL. +* success. Can be NULL. Nit: tempting but this change does not belong here.
Clearly a much loved typo!
Quoted 32 lines
Quoted 24 lines
drivers/acpi/pptt.c +8
-4@@ -192,14 +192,18 @@ acpi_find_cache_level(struct acpi_table_header *table_hdr,* split cache levels (data/instruction) that exist at each level on the way * up. */ -static void acpi_count_levels (struct acpi_table_header *table_hdr, -struct acpi_pptt_processor *cpu_node, -unsigned int *levels,unsigned int *split_levels) +static int acpi_count_levels (struct acpi_table_header *table_hdr, +struct acpi_pptt_processor *cpu_node, +unsigned int *split_levels) { +int starting_level =0 ; +do { -acpi_find_cache_level (table_hdr, cpu_node, levels, split_levels,0 ,0 ); +acpi_find_cache_level (table_hdr, cpu_node, &starting_level, split_levels,0 ,0 ); cpu_node =fetch_pptt_node (table_hdr, cpu_node->parent); }while (cpu_node); + +return starting_level; }/** drivers/acpi/pptt.c int acpi_get_cache_info(unsigned int cpu, unsigned int *levels, · line 738
Quoted 5 lines
drivers/acpi/pptt.c +1
-1@@ -731,7 +735,7 @@ int acpi_get_cache_info(unsigned int cpu, unsigned int *levels,if (!cpu_node)return -ENOENT ; -acpi_count_levels (table, cpu_node, levels, split_levels); + *levels =acpi_count_levels (table, cpu_node, split_levels);Looks fine to me - though initializing
Quoted 5 lines
*levels = 0
upper in the function now becomes superfluous (?) (well, it initializes *levels to 0 if an error path is hit but on that case the caller should not expect *levels to be initialized to anything IIUC).
Maybe, but its the least surprising thing to do - hence the existing early clobber.
Quoted 3 lines
Apart from these (very) minor things:
Reviewed-by Lorenzo Pieralisi <[email protected]>
Thanks!
James
James Morse
maintainerPATCH 5/33 ACPI / PPTT: Find cache level by cache-id
In reply to James MorseThe MPAM table identifies caches by id. The MPAM driver also wants to know the cache level to determine if the platform is of the shape that can be managed via resctrl. Cacheinfo has this information, but only for CPUs that are online.
Waiting for all CPUs to come online is a problem for platforms where CPUs are brought online late by user-space.
Add a helper that walks every possible cache, until it finds the one identified by cache-id, then return the level.
Add a cleanup based free-ing mechanism for acpi_get_table().
---
Changes since RFC:
* acpi_count_levels() now returns a value.
* Converted the table-get stuff to use Jonathan's cleanup helper.
* Dropped Sudeep's Review tag due to the cleanup change.
--- drivers/acpi/pptt.c | 64 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 17 ++++++++++++ 2 files changed, 81 insertions(+)
drivers/acpi/pptt.c +64 -0
@@ -907,3 +907,67 @@ int find_acpi_cpu_topology_hetero_id(unsigned int cpu)return find_acpi_cpu_topology_tag (cpu,PPTT_ABORT_PACKAGE ,ACPI_PPTT_ACPI_IDENTICAL ); } + +/** +* find_acpi_cache_level_from_id() - Get the level of the specified cache +* @cache_id: The id field of the unified cache +* +* Determine the level relative to any CPU for the unified cache identified by +* cache_id. This allows the property to be found even if the CPUs are offline. +* +* The returned level can be used to group unified caches that are peers. +* +* The PPTT table must be rev 3 or later, +* +* If one CPUs L2 is shared with another as L3, this function will return +* an unpredictable value. +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. +* Otherwise returns a value which represents the level of the specified cache. +*/ +int find_acpi_cache_level_from_id (u32 cache_id) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0); + +if (IS_ERR (table)) +return PTR_ERR (table); + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each CPU +* to find the level... +*/ +for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return -ENOENT ; + num_levels =acpi_count_levels (table, cpu_node,NULL ); + +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node); +if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +return level; + } + } + +return -ENOENT ; +}
include/linux/acpi.h +17 -0
@@ -8,6 +8,7 @@#ifndef _LINUX_ACPI_H#define _LINUX_ACPI_H +#include <linux/cleanup.h> #include <linux/errno.h> #include <linux/ioport.h> /* for struct resource */ #include <linux/resource_ext.h>
@@ -221,6 +222,17 @@ void acpi_reserve_initial_tables (void);void acpi_table_init_complete (void );int acpi_table_init (void ); +static inline struct acpi_table_header *acpi_get_table_ret (char *signature,u32 instance) +{ +struct acpi_table_header *table; +int status =acpi_get_table (signature, instance, &table); + +if (ACPI_FAILURE (status)) +return ERR_PTR (-ENOENT ); +return table; +} +DEFINE_FREE(acpi_table, struct acpi_table_header *, if (!IS_ERR(_T)) acpi_put_table(_T)) +int acpi_table_parse (char *id,acpi_tbl_table_handler handler); int __init_or_acpilib acpi_table_parse_entries(char *id, unsigned long table_size, int entry_id,
@@ -1542,6 +1554,7 @@ int find_acpi_cpu_topology_cluster(unsigned int cpu);int find_acpi_cpu_topology_package (unsigned int cpu);int find_acpi_cpu_topology_hetero_id (unsigned int cpu);void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus); +int find_acpi_cache_level_from_id (u32 cache_id);#else static inline int acpi_pptt_cpu_is_thread (unsigned int cpu) {
@@ -1565,6 +1578,10 @@ static inline int find_acpi_cpu_topology_hetero_id(unsigned int cpu) }static inline void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) { } +static inline int find_acpi_cache_level_from_id (u32 cache_id) +{ +return -EINVAL ; +}#endif void acpi_arch_init (void );
Signature · 1 line
2.20.1
Re: [PATCH 05/33] ACPI / PPTT: Find cache level by cache-id
…
Quoted 1 line
+++
b/include/linux/acpi.h
…
Quoted 3 lines
include/linux/acpi.h +0
-0@@ -221,6 +222,17 @@ void acpi_reserve_initial_tables (void);void acpi_table_init_complete (void );int acpi_table_init (void );
…
include/linux/acpi.h void acpi_reserve_initial_tables (void); · line 236
Quoted 3 lines
include/linux/acpi.h +2
-0+DEFINE_FREE(acpi_table, struct acpi_table_header *, if (!IS_ERR(_T)) acpi_put_table(_T)) +int acpi_table_parse (char *id,acpi_tbl_table_handler handler);
…
How do you think about to offer the addition of such a special macro call by another separate update step?
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?h=v6.17-rc2#n81
Regards,
Markus
James Morse
maintainerRe: [PATCH 05/33] ACPI / PPTT: Find cache level by cache-id
Hi Markus,
Quoted 7 lines · Markus Elfring
…
Quoted 1 line
+++
b/include/linux/acpi.h…
Quoted 3 lines
include/linux/acpi.h +0
-0@@ -221,6 +222,17 @@ void acpi_reserve_initial_tables (void);void acpi_table_init_complete (void );int acpi_table_init (void );…
Quoted 8 lines
include/linux/acpi.h void acpi_reserve_initial_tables (void); · line 236
Quoted 3 lines
include/linux/acpi.h +2
-0+DEFINE_FREE(acpi_table, struct acpi_table_header *, if (!IS_ERR(_T)) acpi_put_table(_T)) +int acpi_table_parse (char *id,acpi_tbl_table_handler handler);…
How do you think about to offer the addition of such a special macro call by another separate update step?
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?h=v6.17-rc2#n81
As it goes via the same tree I don't think there is a strong reason either way.
Dave points out on an earlier patch that the PPTT code doesn't care about the reference counting anyway, so this stuff can go.
Thanks,
James
Ben Horgan
maintainerRe: [PATCH 05/33] ACPI / PPTT: Find cache level by cache-id
In reply to James MorseHi James,
Quoted 112 lines · James Morse
The MPAM table identifies caches by id. The MPAM driver also wants to know the cache level to determine if the platform is of the shape that can be managed via resctrl. Cacheinfo has this information, but only for CPUs that are online.
Waiting for all CPUs to come online is a problem for platforms where CPUs are brought online late by user-space.
Add a helper that walks every possible cache, until it finds the one identified by cache-id, then return the level.
Add a cleanup based free-ing mechanism foracpi_get_table().CC Jonathan Cameron <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
*acpi_count_levels()now returns a value.
* Converted the table-get stuff to use Jonathan's cleanup helper.
* Dropped Sudeep's Review tag due to the cleanup change.--- drivers/acpi/pptt.c | 64 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 17 ++++++++++++ 2 files changed, 81 insertions(+)drivers/acpi/pptt.c +64
-0@@ -907,3 +907,67 @@ int find_acpi_cpu_topology_hetero_id(unsigned int cpu)return find_acpi_cpu_topology_tag (cpu,PPTT_ABORT_PACKAGE ,ACPI_PPTT_ACPI_IDENTICAL ); } + +/** +* find_acpi_cache_level_from_id() - Get the level of the specified cache +* @cache_id: The id field of the unified cache +* +* Determine the level relative to any CPU for the unified cache identified by +* cache_id. This allows the property to be found even if the CPUs are offline. +* +* The returned level can be used to group unified caches that are peers. +* +* The PPTT table must be rev 3 or later, +* +* If one CPUs L2 is shared with another as L3, this function will return +* an unpredictable value. +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. +* Otherwise returns a value which represents the level of the specified cache. +*/ +int find_acpi_cache_level_from_id (u32 cache_id) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0); + +if (IS_ERR (table)) +return PTR_ERR (table); + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each CPU +* to find the level... +*/ +for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return -ENOENT ; + num_levels =acpi_count_levels (table, cpu_node,NULL ); + +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node); +if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +return level; + } + } + +return -ENOENT ; +}include/linux/acpi.h +1
-0@@ -8,6 +8,7 @@#ifndef _LINUX_ACPI_H#define _LINUX_ACPI_H +#include <linux/cleanup.h> #include <linux/errno.h> #include <linux/ioport.h> /* for struct resource */ #include <linux/resource_ext.h> @@ -221,6 +222,17 @@ void acpi_reserve_initial_tables (void);void acpi_table_init_complete (void );int acpi_table_init (void );
include/linux/acpi.h void acpi_reserve_initial_tables (void); · line 234
Quoted 10 lines
include/linux/acpi.h +10
-0+static inline struct acpi_table_header *acpi_get_table_ret (char *signature,u32 instance) +{ +struct acpi_table_header *table; +int status =acpi_get_table (signature, instance, &table); + +if (ACPI_FAILURE (status)) +return ERR_PTR (-ENOENT ); +return table; +} +DEFINE_FREE(acpi_table, struct acpi_table_header *, if (!IS_ERR(_T)) acpi_put_table(_T))
nit: Is it useful to change the condition from !IS_ERR(_T) to !IS_ERR_OR_NULL(_T)? This seems to be the common pattern. I do note that acpi_put_table() can take NULL, so there is no real danger.
Quoted 13 lines
include/linux/acpi.h +2
-0+int acpi_table_parse (char *id,acpi_tbl_table_handler handler); int __init_or_acpilib acpi_table_parse_entries(char *id, unsigned long table_size, int entry_id,@@ -1542,6 +1554,7 @@ int find_acpi_cpu_topology_cluster(unsigned int cpu);int find_acpi_cpu_topology_package (unsigned int cpu);int find_acpi_cpu_topology_hetero_id (unsigned int cpu);void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus); +int find_acpi_cache_level_from_id (u32 cache_id);#else static inline int acpi_pptt_cpu_is_thread (unsigned int cpu) {
include/linux/acpi.h static inline int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 1587
Quoted 10 lines
include/linux/acpi.h +4
-0@@ -1565,6 +1578,10 @@ static inline int find_acpi_cpu_topology_hetero_id(unsigned int cpu) }static inline void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) { } +static inline int find_acpi_cache_level_from_id (u32 cache_id) +{ +return -EINVAL ; +}#endif void acpi_arch_init (void );
Thanks,
Ben
James Morse
maintainerRe: [PATCH 05/33] ACPI / PPTT: Find cache level by cache-id
Hi Ben,
Quoted 21 lines · Ben Horgan
Quoted 20 lines · James Morse
The MPAM table identifies caches by id. The MPAM driver also wants to know the cache level to determine if the platform is of the shape that can be managed via resctrl. Cacheinfo has this information, but only for CPUs that are online.
Waiting for all CPUs to come online is a problem for platforms where CPUs are brought online late by user-space.
Add a helper that walks every possible cache, until it finds the one identified by cache-id, then return the level.
Add a cleanup based free-ing mechanism foracpi_get_table().include/linux/acpi.h +0
-0@@ -221,6 +222,17 @@ void acpi_reserve_initial_tables (void);void acpi_table_init_complete (void );int acpi_table_init (void );
Quoted 14 lines
include/linux/acpi.h void acpi_reserve_initial_tables (void); · line 234
Quoted 10 lines
include/linux/acpi.h +10
-0+static inline struct acpi_table_header *acpi_get_table_ret (char *signature,u32 instance) +{ +struct acpi_table_header *table; +int status =acpi_get_table (signature, instance, &table); + +if (ACPI_FAILURE (status)) +return ERR_PTR (-ENOENT ); +return table; +} +DEFINE_FREE(acpi_table, struct acpi_table_header *, if (!IS_ERR(_T)) acpi_put_table(_T))nit: Is it useful to change the condition from !IS_ERR(_T) to !IS_ERR_OR_NULL(_T)? This seems to be the common pattern. I do note that
acpi_put_table()can take NULL, so there is no real danger.
If it's the common pattern, sure.
But this code got dropped as Dave pointed out the PPTT doesn't care about the reference counting anyway, its acpi_get_pptt() helper just uses the same reference for everything.
This might come back for the MPAM driver..
Quoted 4 lines
include/linux/acpi.h void acpi_reserve_initial_tables (void); · line 238
Quoted 4 lines
include/linux/acpi.h +1
-0+int acpi_table_parse (char *id,acpi_tbl_table_handler handler); int __init_or_acpilib acpi_table_parse_entries(char *id, unsigned long table_size, int entry_id,
Thanks,
James
Re: [PATCH 05/33] ACPI / PPTT: Find cache level by cache-id
In reply to James MorseHi,
Quoted 11 lines · James Morse
The MPAM table identifies caches by id. The MPAM driver also wants to know the cache level to determine if the platform is of the shape that can be managed via resctrl. Cacheinfo has this information, but only for CPUs that are online.
Waiting for all CPUs to come online is a problem for platforms where CPUs are brought online late by user-space.
Add a helper that walks every possible cache, until it finds the one identified by cache-id, then return the level.
Add a cleanup based free-ing mechanism foracpi_get_table().
Does this mean that the early secondaries must be spread out across the whole topology so that everything can be probed?
(i.e., a random subset is no good?)
If so, is this documented somewhere, such as in booting.rst?
Maybe this is not a new requirement -- it's not an area that I'm very familiar with.
Quoted 27 lines
CC Jonathan Cameron <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
*acpi_count_levels()now returns a value.
* Converted the table-get stuff to use Jonathan's cleanup helper.
* Dropped Sudeep's Review tag due to the cleanup change.--- drivers/acpi/pptt.c | 64 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 17 ++++++++++++ 2 files changed, 81 insertions(+)drivers/acpi/pptt.c +6
-0@@ -907,3 +907,67 @@ int find_acpi_cpu_topology_hetero_id(unsigned int cpu)return find_acpi_cpu_topology_tag (cpu,PPTT_ABORT_PACKAGE ,ACPI_PPTT_ACPI_IDENTICAL ); } + +/** +* find_acpi_cache_level_from_id() - Get the level of the specified cache +* @cache_id: The id field of the unified cache +* +* Determine the level relative to any CPU for the unified cache identified by
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 925
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+* cache_id. This allows the property to be found even if the CPUs are offline. +* +* The returned level can be used to group unified caches that are peers. +* +* The PPTT table must be rev 3 or later, +* +* If one CPUs L2 is shared with another as L3, this function will return +* an unpredictable value. +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found.
Nit: doesn't exist or its revision is too old.
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 935
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+* Otherwise returns a value which represents the level of the specified cache. +*/ +int find_acpi_cache_level_from_id (u32 cache_id) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0);
acpi_get_pptt() ? (See comment on patch 3.)
Comments there also suggest that the acpi_put_table() may be unnecessary, at least on some paths.
I haven't tried to understand the ins and outs of this.
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 946
Quoted 11 lines
drivers/acpi/pptt.c +11
-0+ +if (IS_ERR (table)) +return PTR_ERR (table); + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each CPU +* to find the level... +*/
^ Possibly confusing comment? The cache id is the starting point for calling this function. Is there a world in which we are at this point without first having found the cache node?
(If the comment is just a restatement of part of the kerneldoc description, maybe just drop it.)
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 952
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return -ENOENT ; + num_levels =acpi_count_levels (table, cpu_node,NULL );
Is the initial call to acpi_count_levels() really needed here?
It feels a bit like we end up enumerating the whole topology two or three times here; once to count how many levels there are, and then again to examine the nodes, and once more inside acpi_find_cache_node().
Why can't we just walk until we run out of levels?
I may be missing some details of how these functions interact -- if this is only run at probe time, compact, well-factored code is more important than making things as fast as possible.
Quoted 25 lines
drivers/acpi/pptt.c +21
-0+ +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node); +if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +return level; + } + } + +return -ENOENT ; +}include/linux/acpi.h +0
-0
[...]
include/linux/acpi.h void acpi_reserve_initial_tables (void); · line 233
Quoted 13 lines
include/linux/acpi.h +9
-0@@ -221,6 +222,17 @@ void acpi_reserve_initial_tables (void);void acpi_table_init_complete (void );int acpi_table_init (void ); +static inline struct acpi_table_header *acpi_get_table_ret (char *signature,u32 instance) +{ +struct acpi_table_header *table; +int status =acpi_get_table (signature, instance, &table); + +if (ACPI_FAILURE (status)) +return ERR_PTR (-ENOENT ); +return table; +}
This feels like something that ought to exist already. If not, why not? If so, are there open-coded versions of this spread around the ACPI tree that should be ported to use it?
[...]
Cheers
---Dave
James Morse
maintainerRe: [PATCH 05/33] ACPI / PPTT: Find cache level by cache-id
Hi Dave,
Quoted 11 lines · Dave Martin
Hi,
Quoted 8 lines · James Morse
The MPAM table identifies caches by id. The MPAM driver also wants to know the cache level to determine if the platform is of the shape that can be managed via resctrl. Cacheinfo has this information, but only for CPUs that are online.
Waiting for all CPUs to come online is a problem for platforms where CPUs are brought online late by user-space.
Quoted 8 lines
Quoted 3 lines
Add a helper that walks every possible cache, until it finds the one identified by cache-id, then return the level.
Add a cleanup based free-ing mechanism foracpi_get_table().Does this mean that the early secondaries must be spread out across the whole topology so that everything can be probed?
(i.e., a random subset is no good?)
For the mpam driver - it needs to see each cache with mpam hardware, which means a CPU associated with each cache needs to be online. Random is fine - provided you get lucky.
Quoted 1 line
If so, is this documented somewhere, such as in
booting.rst?
booting.rst is for the bootloader.
Late secondaries is a bit of a niche sport, I've only seen it commonly done in VMs. Most platforms so far have their MPAM controls on a global L3, so this requirement doesn't make much of a difference.
The concern is that if resctrl gets probed after user-space has started, whatever user-space service is supposed to set it up will have concluded its not supported. Working with cache-ids for offline CPUs means you don't have to bring all the CPUs online - only enough so that every piece of hardware is reachable.
Quoted 2 lines
Maybe this is not a new requirement -- it's not an area that I'm very familiar with.
Hard to say - its a potentially surprising side effect of glomming OS accessible registers onto the side of hardware that can be automatically powered off. (PSCI CPU_SUSPEND).
I did try getting cacheinfo to populate all the CPUs at boot, regardless of whether they were online. Apparently that doesn't work for PowerPC where the properties of CPUs can change while they are offline. (presumably due to RAS or a firmware update)
Quoted 18 lines
Quoted 18 lines
drivers/acpi/pptt.c +10
-0@@ -907,3 +907,67 @@ int find_acpi_cpu_topology_hetero_id(unsigned int cpu)return find_acpi_cpu_topology_tag (cpu,PPTT_ABORT_PACKAGE ,ACPI_PPTT_ACPI_IDENTICAL ); } + +/** +* find_acpi_cache_level_from_id() - Get the level of the specified cache +* @cache_id: The id field of the unified cache +* +* Determine the level relative to any CPU for the unified cache identified by +* cache_id. This allows the property to be found even if the CPUs are offline. +* +* The returned level can be used to group unified caches that are peers. +*
Quoted 8 lines
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 925
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+* The PPTT table must be rev 3 or later, +* +* If one CPUs L2 is shared with another as L3, this function will return +* an unpredictable value. +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. Nit: doesn't exist or its revision is too old.
... its not old, but there is no published spec for that revision... unsupported?
Quoted 4 lines
Quoted 4 lines
drivers/acpi/pptt.c +4
-0+* Otherwise returns a value which represents the level of the specified cache. +*/ +int find_acpi_cache_level_from_id (u32 cache_id) +{
Quoted 8 lines
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 935
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0);
acpi_get_pptt()? (See comment on patch 3.)
Yup,
Quoted 4 lines
Comments there also suggest that the
acpi_put_table()may be unnecessary, at least on some paths.I haven't tried to understand the ins and outs of this.
It's grabbing one reference and using it for everything, because it needs to 'map' the table in atomic context due to cpuhp, but can't.
Given how frequently its used, there is no problem just leaving it mapped.
Quoted 16 lines
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 946
Quoted 11 lines
drivers/acpi/pptt.c +11
-0+ +if (IS_ERR (table)) +return PTR_ERR (table); + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each CPU +* to find the level... +*/ ^ Possibly confusing comment? The cache id is the starting point for calling this function. Is there a world in which we are at this point without first having found the cache node?
Quoted 2 lines
(If the comment is just a restatement of part of the kerneldoc description, maybe just drop it.)
It's describing the alternate world where the table is searched to find the cache first, but then we'd still need to walk the table another NR_CPUs times, which can't be avoided. I'll drop it - it was justifying why its done this way round...
Quoted 9 lines
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 952
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return -ENOENT ; + num_levels =acpi_count_levels (table, cpu_node,NULL );Is the initial call to
acpi_count_levels()really needed here?
Quoted 5 lines
It feels a bit like we end up enumerating the whole topology two or three times here; once to count how many levels there are, and then again to examine the nodes, and once more inside
acpi_find_cache_node().Why can't we just walk until we run out of levels?
This is looking for a unified cache - and we don't know where those start. We could walk the first 100 caches, and stop once we start getting unified caches, then they stop again ... but this seemed simpler.
Quoted 16 lines
I may be missing some details of how these functions interact -- if this is only run at probe time, compact, well-factored code is more important than making things as fast as possible.
Quoted 4 lines
include/linux/acpi.h +0
-0[...]
Quoted 4 lines
include/linux/acpi.h +0
-0@@ -221,6 +222,17 @@ void acpi_reserve_initial_tables (void);void acpi_table_init_complete (void );int acpi_table_init (void );
Quoted 13 lines
include/linux/acpi.h void acpi_reserve_initial_tables (void); · line 233
Quoted 9 lines
include/linux/acpi.h +9
-0+static inline struct acpi_table_header *acpi_get_table_ret (char *signature,u32 instance) +{ +struct acpi_table_header *table; +int status =acpi_get_table (signature, instance, &table); + +if (ACPI_FAILURE (status)) +return ERR_PTR (-ENOENT ); +return table; +}This feels like something that ought to exist already. If not, why not? If so, are there open-coded versions of this spread around the ACPI tree that should be ported to use it?
It's a cleanup idiom helper that lets the compiler do this automagically - but its moot as its not going to be needed in the pptt because of the acpi_get_pptt() thing.
Thanks,
James
Re: [PATCH 05/33] ACPI / PPTT: Find cache level by cache-id
Hi James,
Quoted 18 lines · James Morse
Hi Dave,
Quoted 14 lines · Dave Martin
Hi,
Quoted 11 lines · James Morse
The MPAM table identifies caches by id. The MPAM driver also wants to know the cache level to determine if the platform is of the shape that can be managed via resctrl. Cacheinfo has this information, but only for CPUs that are online.
Waiting for all CPUs to come online is a problem for platforms where CPUs are brought online late by user-space.
Add a helper that walks every possible cache, until it finds the one identified by cache-id, then return the level.
Add a cleanup based free-ing mechanism foracpi_get_table().
Quoted 7 lines
Quoted 4 lines
Does this mean that the early secondaries must be spread out across the whole topology so that everything can be probed?
(i.e., a random subset is no good?)
For the mpam driver - it needs to see each cache with mpam hardware, which means a CPU associated with each cache needs to be online. Random is fine - provided you get lucky.
"Fine" = "not dependent on luck". So, random is not fine.
Quoted 16 lines
Quoted 1 line
If so, is this documented somewhere, such as in
booting.rst?
booting.rstis for the bootloader.
Late secondaries is a bit of a niche sport, I've only seen it commonly done in VMs. Most platforms so far have their MPAM controls on a global L3, so this requirement doesn't make much of a difference.The concern is that if resctrl gets probed after user-space has started, whatever user-space service is supposed to set it up will have concluded its not supported. Working with cache-ids for offline CPUs means you don't have to bring all the CPUs online - only enough so that every piece of hardware is reachable.
Quoted 2 lines
Maybe this is not a new requirement -- it's not an area that I'm very familiar with.
Quoted 6 lines
Hard to say - its a potentially surprising side effect of glomming OS accessible registers onto the side of hardware that can be automatically powered off. (PSCI
CPU_SUSPEND).I did try getting cacheinfo to populate all the CPUs at boot, regardless of whether they were online. Apparently that doesn't work for PowerPC where the properties of CPUs can change while they are offline. (presumably due to RAS or a firmware update)
So, it sounds like there is a requirement, but we don't document it, and if the requirement is not met then the user is presented with an obscure failure in the MPAM driver. This seems a bit unhelpful?
I'm not saying booting.rst is the right place for this -- maybe the appropriate document doesn't exist yet.
I wonder whether the required property is reasonable and general enough that it should be treated as a kernel boot requirement.
Or, we require caches to be symmetric for non-early CPUs and reject those that don't match when they try to come online (similarly to the way cpufeatures deals with mismatches).
Quoted 27 lines
Quoted 18 lines
Quoted 18 lines
drivers/acpi/pptt.c +10
-0@@ -907,3 +907,67 @@ int find_acpi_cpu_topology_hetero_id(unsigned int cpu)return find_acpi_cpu_topology_tag (cpu,PPTT_ABORT_PACKAGE ,ACPI_PPTT_ACPI_IDENTICAL ); } + +/** +* find_acpi_cache_level_from_id() - Get the level of the specified cache +* @cache_id: The id field of the unified cache +* +* Determine the level relative to any CPU for the unified cache identified by +* cache_id. This allows the property to be found even if the CPUs are offline. +* +* The returned level can be used to group unified caches that are peers. +* Quoted 8 lines
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 925
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+* The PPTT table must be rev 3 or later, +* +* If one CPUs L2 is shared with another as L3, this function will return +* an unpredictable value. +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. Nit: doesn't exist or its revision is too old.
Quoted 1 line
... its not old, but there is no published spec for that revision... unsupported?
That seems OK, say,
"... if the PPTT doesn't exist or has an unsupported format, or ..."
Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
drivers/acpi/pptt.c +4
-0+* Otherwise returns a value which represents the level of the specified cache. +*/ +int find_acpi_cache_level_from_id (u32 cache_id) +{Quoted 8 lines
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 935
Quoted 6 lines
drivers/acpi/pptt.c +6
-0+u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0);
acpi_get_pptt()? (See comment on patch 3.)Yup,
Quoted 8 lines
Quoted 4 lines
Comments there also suggest that the
acpi_put_table()may be unnecessary, at least on some paths.I haven't tried to understand the ins and outs of this.
It's grabbing one reference and using it for everything, because it needs to 'map' the table in atomic context due to cpuhp, but can't.
Given how frequently its used, there is no problem just leaving it mapped.
That's rather what I thought -- in which case we can use it (as you already concluded, by the looks of it).
Quoted 16 lines
Quoted 16 lines
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 946
Quoted 11 lines
drivers/acpi/pptt.c +11
-0+ +if (IS_ERR (table)) +return PTR_ERR (table); + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each CPU +* to find the level... +*/ ^ Possibly confusing comment? The cache id is the starting point for calling this function. Is there a world in which we are at this point without first having found the cache node?
Quoted 6 lines
Quoted 2 lines
(If the comment is just a restatement of part of the kerneldoc description, maybe just drop it.)
It's describing the alternate world where the table is searched to find the cache first, but then we'd still need to walk the table another
NR_CPUstimes, which can't be avoided. I'll drop it - it was justifying why its done this way round...
Oh, I see, this is "if the code had been written in such-and-such a way", not "if such-and-such a runtime precondition is met" ?
The comment can be read both ways, as it stands.
Quoted 14 lines
Quoted 14 lines
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 952
Quoted 7 lines
drivers/acpi/pptt.c +6
-0+for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return -ENOENT ; + num_levels =acpi_count_levels (table, cpu_node,NULL );Is the initial call to
acpi_count_levels()really needed here?It feels a bit like we end up enumerating the whole topology two or three times here; once to count how many levels there are, and then again to examine the nodes, and once more inside
acpi_find_cache_node().
Quoted 5 lines
Quoted 1 line
Why can't we just walk until we run out of levels?
This is looking for a unified cache - and we don't know where those start. We could walk the first 100 caches, and stop once we start getting unified caches, then they stop again ... but this seemed simpler.
I'm still a bit confused.
We start at level one, and then trace parents until we hit a unified cache or run out of levels.
Why do we need to know a priori how many levels there are, when the way to determine that is part of the same procedure we're already doing (i.e., start at level one and trace parents until we run out of levels)?
Quoted 3 lines
Quoted 3 lines
I may be missing some details of how these functions interact -- if this is only run at probe time, compact, well-factored code is more important than making things as fast as possible.
(This still stands.)
Quoted 1 line
Quoted 1 line
Quoted 1 line
include/linux/acpi.h +0
-0
[...]
Quoted 19 lines
Quoted 4 lines
Quoted 4 lines
include/linux/acpi.h +0
-0@@ -221,6 +222,17 @@ void acpi_reserve_initial_tables (void);void acpi_table_init_complete (void );int acpi_table_init (void );Quoted 13 lines
include/linux/acpi.h void acpi_reserve_initial_tables (void); · line 233
Quoted 9 lines
include/linux/acpi.h +9
-0+static inline struct acpi_table_header *acpi_get_table_ret (char *signature,u32 instance) +{ +struct acpi_table_header *table; +int status =acpi_get_table (signature, instance, &table); + +if (ACPI_FAILURE (status)) +return ERR_PTR (-ENOENT ); +return table; +}This feels like something that ought to exist already. If not, why not? If so, are there open-coded versions of this spread around the ACPI tree that should be ported to use it?
Quoted 2 lines
It's a cleanup idiom helper that lets the compiler do this automagically - but its moot as its not going to be needed in the pptt because of the
acpi_get_pptt()thing.
Ah, OK.
Cheers
---Dave
James Morse
maintainerRe: [PATCH 05/33] ACPI / PPTT: Find cache level by cache-id
Hi Dave,
Quoted 18 lines · Dave Martin
Quoted 17 lines · James Morse
Quoted 16 lines · Dave Martin
Quoted 11 lines · James Morse
The MPAM table identifies caches by id. The MPAM driver also wants to know the cache level to determine if the platform is of the shape that can be managed via resctrl. Cacheinfo has this information, but only for CPUs that are online.
Waiting for all CPUs to come online is a problem for platforms where CPUs are brought online late by user-space.
Add a helper that walks every possible cache, until it finds the one identified by cache-id, then return the level.
Add a cleanup based free-ing mechanism foracpi_get_table().Does this mean that the early secondaries must be spread out across the whole topology so that everything can be probed?
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
(i.e., a random subset is no good?)
For the mpam driver - it needs to see each cache with mpam hardware, which means a CPU associated with each cache needs to be online. Random is fine - provided you get lucky.
"Fine" = "not dependent on luck". So, random is not fine.
As in it doesn't matter which CPUs - as long as you manage to represent each cache.
I really don't care if people configure their platform such that the mpam driver can't complete its probing. Everything continues to work, you just don't get to use the unusable features.
I've only really seen it done in VMs, which are most likely to have a single global MSC because the thing has to be emulated.
Quoted 19 lines
Quoted 19 lines
Quoted 1 line
If so, is this documented somewhere, such as in
booting.rst?
booting.rstis for the bootloader.
Late secondaries is a bit of a niche sport, I've only seen it commonly done in VMs. Most platforms so far have their MPAM controls on a global L3, so this requirement doesn't make much of a difference.The concern is that if resctrl gets probed after user-space has started, whatever user-space service is supposed to set it up will have concluded its not supported. Working with cache-ids for offline CPUs means you don't have to bring all the CPUs online - only enough so that every piece of hardware is reachable.
Quoted 2 lines
Maybe this is not a new requirement -- it's not an area that I'm very familiar with.
Hard to say - its a potentially surprising side effect of glomming OS accessible registers onto the side of hardware that can be automatically powered off. (PSCI
CPU_SUSPEND).
Quoted 7 lines
Quoted 3 lines
I did try getting cacheinfo to populate all the CPUs at boot, regardless of whether they were online. Apparently that doesn't work for PowerPC where the properties of CPUs can change while they are offline. (presumably due to RAS or a firmware update)
So, it sounds like there is a requirement, but we don't document it, and if the requirement is not met then the user is presented with an obscure failure in the MPAM driver. This seems a bit unhelpful?
Not a failure. MPAM isn't yet available. Bring the rest of the system up and it will spring into life. The same goes for any device you keep turned off.
Quoted 5 lines
I'm not saying
booting.rstis the right place for this -- maybe the appropriate document doesn't exist yet.I wonder whether the required property is reasonable and general enough that it should be treated as a kernel boot requirement.
It's not a boot requirement. Linux boots just fine.
Quoted 3 lines
Or, we require caches to be symmetric for non-early CPUs and reject those that don't match when they try to come online (similarly to the way cpufeatures deals with mismatches).
MPAM isn't an interesting enough feature to reject CPUs!
We can't check the the MSC properties until we can schedule, (thank the 'hide it in firmware' folk for that one), meaning the CPU has to be online before we realise the MPAM properties are mismatched.
The best approach here is to wait until everything has been seen before declaring that we know what is going on. The architecture even calls this out as something that needs doing, meaning the firmware tables have to describe all possible MSC.
I agree its not nice - but it is what MPAM is.
I think the people clever enough to manage late online-ing secondaries will work this out.
Quoted 26 lines
Quoted 20 lines
Quoted 20 lines
drivers/acpi/pptt.c int find_acpi_cpu_topology_hetero_id(unsigned int cpu) · line 914
Quoted 13 lines
drivers/acpi/pptt.c +6
-0@@ -907,3 +907,67 @@ int find_acpi_cpu_topology_hetero_id(unsigned int cpu) +for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return -ENOENT ; + num_levels =acpi_count_levels (table, cpu_node,NULL );Is the initial call to
acpi_count_levels()really needed here?It feels a bit like we end up enumerating the whole topology two or three times here; once to count how many levels there are, and then again to examine the nodes, and once more inside
acpi_find_cache_node().Quoted 5 lines
Quoted 1 line
Why can't we just walk until we run out of levels?
This is looking for a unified cache - and we don't know where those start. We could walk the first 100 caches, and stop once we start getting unified caches, then they stop again ... but this seemed simpler.
Quoted 8 lines
I'm still a bit confused.
We start at level one, and then trace parents until we hit a unified cache or run out of levels.
Why do we need to know a priori how many levels there are, when the way to determine that is part of the same procedure we're already doing (i.e., start at level one and trace parents until we run out of levels)?
| level = 1; | acpi_find_cache_node(table, acpi_cpu_id, ACPI_PPTT_CACHE_TYPE_UNIFIED, | level, &cpu_node);
Fails. Do we stop the loop, or try level=2?
level = 1 fails because the L1 (probably) isn't a unified cache. Is L2? We don't know.
It's simpler to know how many levels there are, and walk the lot, than it is to try and work out where the unified bit of the hierarchy starts - and start walking from there.
Yes the PPTT parsing could be different, but this is the kind of shape it has. Doing it like this is in-keeping with the rest of it.
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
I may be missing some details of how these functions interact -- if this is only run at probe time, compact, well-factored code is more important than making things as fast as possible.
(This still stands.)
This is all done at probe time, and never called again.
Nothing else in here caches values - it derives it all from the table every time so that its stateless.
James
James Morse
maintainerPATCH 5/33 ACPI / PPTT: Find cache level by cache-id
In reply to James MorseThe MPAM table identifies caches by id. The MPAM driver also wants to know the cache level to determine if the platform is of the shape that can be managed via resctrl. Cacheinfo has this information, but only for CPUs that are online.
Waiting for all CPUs to come online is a problem for platforms where CPUs are brought online late by user-space.
Add a helper that walks every possible cache, until it finds the one identified by cache-id, then return the level.
Add a cleanup based free-ing mechanism for acpi_get_table().
---
Changes since RFC:
* acpi_count_levels() now returns a value.
* Converted the table-get stuff to use Jonathan's cleanup helper.
* Dropped Sudeep's Review tag due to the cleanup change.
--- drivers/acpi/pptt.c | 64 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 17 ++++++++++++ 2 files changed, 81 insertions(+)
drivers/acpi/pptt.c +64 -0
@@ -907,3 +907,67 @@ int find_acpi_cpu_topology_hetero_id(unsigned int cpu)return find_acpi_cpu_topology_tag (cpu,PPTT_ABORT_PACKAGE ,ACPI_PPTT_ACPI_IDENTICAL ); } + +/** +* find_acpi_cache_level_from_id() - Get the level of the specified cache +* @cache_id: The id field of the unified cache +* +* Determine the level relative to any CPU for the unified cache identified by +* cache_id. This allows the property to be found even if the CPUs are offline. +* +* The returned level can be used to group unified caches that are peers. +* +* The PPTT table must be rev 3 or later, +* +* If one CPUs L2 is shared with another as L3, this function will return +* an unpredictable value. +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. +* Otherwise returns a value which represents the level of the specified cache. +*/ +int find_acpi_cache_level_from_id (u32 cache_id) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0); + +if (IS_ERR (table)) +return PTR_ERR (table); + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each CPU +* to find the level... +*/ +for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return -ENOENT ; + num_levels =acpi_count_levels (table, cpu_node,NULL ); + +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node); +if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +return level; + } + } + +return -ENOENT ; +}
include/linux/acpi.h +17 -0
@@ -8,6 +8,7 @@#ifndef _LINUX_ACPI_H#define _LINUX_ACPI_H +#include <linux/cleanup.h> #include <linux/errno.h> #include <linux/ioport.h> /* for struct resource */ #include <linux/resource_ext.h>
@@ -221,6 +222,17 @@ void acpi_reserve_initial_tables (void);void acpi_table_init_complete (void );int acpi_table_init (void ); +static inline struct acpi_table_header *acpi_get_table_ret (char *signature,u32 instance) +{ +struct acpi_table_header *table; +int status =acpi_get_table (signature, instance, &table); + +if (ACPI_FAILURE (status)) +return ERR_PTR (-ENOENT ); +return table; +} +DEFINE_FREE(acpi_table, struct acpi_table_header *, if (!IS_ERR(_T)) acpi_put_table(_T)) +int acpi_table_parse (char *id,acpi_tbl_table_handler handler); int __init_or_acpilib acpi_table_parse_entries(char *id, unsigned long table_size, int entry_id,
@@ -1542,6 +1554,7 @@ int find_acpi_cpu_topology_cluster(unsigned int cpu);int find_acpi_cpu_topology_package (unsigned int cpu);int find_acpi_cpu_topology_hetero_id (unsigned int cpu);void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus); +int find_acpi_cache_level_from_id (u32 cache_id);#else static inline int acpi_pptt_cpu_is_thread (unsigned int cpu) {
@@ -1565,6 +1578,10 @@ static inline int find_acpi_cpu_topology_hetero_id(unsigned int cpu) }static inline void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus) { } +static inline int find_acpi_cache_level_from_id (u32 cache_id) +{ +return -EINVAL ; +}#endif void acpi_arch_init (void );
Signature · 1 line
2.20.1
James Morse
maintainerPATCH 6/33 ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
In reply to James MorseMPAM identifies CPUs by the cache_id in the PPTT cache structure.
The driver needs to know which CPUs are associated with the cache, the CPUs may not all be online, so cacheinfo does not have the information.
Add a helper to pull this information out of the PPTT.
---
Changes since RFC:
* acpi_count_levels() now returns a value.
* Converted the table-get stuff to use Jonathan's cleanup helper.
* Dropped Sudeep's Review tag due to the cleanup change.
--- drivers/acpi/pptt.c | 62 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 6 +++++ 2 files changed, 68 insertions(+)
drivers/acpi/pptt.c +62 -0
@@ -971,3 +971,65 @@ int find_acpi_cache_level_from_id(u32 cache_id)return -ENOENT ; } + +/** +* acpi_pptt_get_cpumask_from_cache_id() - Get the cpus associated with the +* specified cache +* @cache_id: The id field of the unified cache +* @cpus: Where to build the cpumask +* +* Determine which CPUs are below this cache in the PPTT. This allows the property +* to be found even if the CPUs are offline. +* +* The PPTT table must be rev 3 or later, +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. +* Otherwise returns 0 and sets the cpus in the provided cpumask. +*/ +int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0); + +cpumask_clear (cpus); + +if (IS_ERR (table)) +return -ENOENT ; + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each cpu. +*/ +for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return 0 ; + num_levels =acpi_count_levels (table, cpu_node,NULL ); + +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node); +if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +cpumask_set_cpu (cpu, cpus); + } + } + +return 0 ; +}
include/linux/acpi.h +6 -0
@@ -1555,6 +1555,7 @@ int find_acpi_cpu_topology_package(unsigned int cpu);int find_acpi_cpu_topology_hetero_id (unsigned int cpu);void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus);int find_acpi_cache_level_from_id (u32 cache_id); +int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus);#else static inline int acpi_pptt_cpu_is_thread (unsigned int cpu) {
@@ -1582,6 +1583,11 @@ static inline int find_acpi_cache_level_from_id(u32 cache_id) {return -EINVAL ; } +static inline int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id, +cpumask_t *cpus) +{ +return -EINVAL ; +}#endif void acpi_arch_init (void );
Signature · 1 line
2.20.1
Re: [PATCH 06/33] ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
Hi James,
Quoted 5 lines · James Morse
MPAM identifies CPUs by the
cache_idin the PPTT cache structure.The driver needs to know which CPUs are associated with the cache, the CPUs may not all be online, so cacheinfo does not have the information.
Nit: cacheinfo lacking the information is not a consequence of the driver needing it.
Maybe split the sentence:
-> "[...] associated with the cache. The CPUs may not [...]"
Quoted 71 lines
Add a helper to pull this information out of the PPTT.
CC Rohit Mathew <[email protected]>Signed-off-by James Morse <[email protected]>Reviewed-by Sudeep Holla <[email protected]>---
Changes since RFC:
*acpi_count_levels()now returns a value.
* Converted the table-get stuff to use Jonathan's cleanup helper.
* Dropped Sudeep's Review tag due to the cleanup change.--- drivers/acpi/pptt.c | 62 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 6 +++++ 2 files changed, 68 insertions(+)drivers/acpi/pptt.c +47
-0@@ -971,3 +971,65 @@ int find_acpi_cache_level_from_id(u32 cache_id)return -ENOENT ; } + +/** +* acpi_pptt_get_cpumask_from_cache_id() - Get the cpus associated with the +* specified cache +* @cache_id: The id field of the unified cache +* @cpus: Where to build the cpumask +* +* Determine which CPUs are below this cache in the PPTT. This allows the property +* to be found even if the CPUs are offline. +* +* The PPTT table must be rev 3 or later, +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. +* Otherwise returns 0 and sets the cpus in the provided cpumask. +*/ +int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0); + +cpumask_clear (cpus); + +if (IS_ERR (table)) +return -ENOENT ; + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each cpu. +*/ +for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return 0 ; + num_levels =acpi_count_levels (table, cpu_node,NULL ); + +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node);
drivers/acpi/pptt.c int find_acpi_cache_level_from_id(u32 cache_id) · line 1030
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +cpumask_set_cpu (cpu, cpus);
Again, it feels like we are repeating the same walk multiple times to determine how deep the table is (on which point the table is self- describing anyway), and then again to derive some static property, and then we are then doing all of that work multiple times to derive different static properties, etc.
Can we not just walk over the tables once and stash the derived properties somewhere?
I'm still getting my head around this parsing code, so I'm not saying that the approach is incorrect here -- just wondering whether there is a way to make it simpler.
[...]
Cheers
---Dave
James Morse
maintainerRe: [PATCH 06/33] ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
Hi Dave,
Quoted 7 lines · Dave Martin
Quoted 5 lines · James Morse
MPAM identifies CPUs by the
cache_idin the PPTT cache structure.The driver needs to know which CPUs are associated with the cache, the CPUs may not all be online, so cacheinfo does not have the information.
Quoted 6 lines
Nit: cacheinfo lacking the information is not a consequence of the driver needing it.
Maybe split the sentence:
-> "[...] associated with the cache. The CPUs may not [...]"
Sure,
Quoted 74 lines
Quoted 57 lines
Add a helper to pull this information out of the PPTT.
drivers/acpi/pptt.c +47
-0@@ -971,3 +971,65 @@ int find_acpi_cache_level_from_id(u32 cache_id)return -ENOENT ; } + +/** +* acpi_pptt_get_cpumask_from_cache_id() - Get the cpus associated with the +* specified cache +* @cache_id: The id field of the unified cache +* @cpus: Where to build the cpumask +* +* Determine which CPUs are below this cache in the PPTT. This allows the property +* to be found even if the CPUs are offline. +* +* The PPTT table must be rev 3 or later, +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. +* Otherwise returns 0 and sets the cpus in the provided cpumask. +*/ +int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0); + +cpumask_clear (cpus); + +if (IS_ERR (table)) +return -ENOENT ; + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each cpu. +*/ +for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return 0 ; + num_levels =acpi_count_levels (table, cpu_node,NULL ); + +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node);drivers/acpi/pptt.c int find_acpi_cache_level_from_id(u32 cache_id) · line 1030
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +cpumask_set_cpu (cpu, cpus);Again, it feels like we are repeating the same walk multiple times to determine how deep the table is (on which point the table is self- describing anyway), and then again to derive some static property, and then we are then doing all of that work multiple times to derive different static properties, etc.
Quoted 2 lines
Can we not just walk over the tables once and stash the derived properties somewhere?
That is possible - but its a more invasive change to the PPTT parsing code. Before the introduction of the leaf flag, the search for a processor also included a search to check if the discovered node was a leaf.
I think this is trading time - walking over the table multiple times, against the memory you'd need to de-serialise the tree to find the necessary properties quickly. I think the reason Jeremy L went this way was because there may never be another request into this code, so being ready with a quick answer was a waste of memory.
MPAM doesn't change this - all these things are done up front during driver probing, and the values are cached by the driver.
Quoted 3 lines
I'm still getting my head around this parsing code, so I'm not saying that the approach is incorrect here -- just wondering whether there is a way to make it simpler.
It's walked at boot, and on cpu-hotplug. Neither are particularly performance critical.
I agree that as platforms get bigger, there will be a tipping point ... I don't think anyone has complained yet!
Thanks,
James
Re: [PATCH 06/33] ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
Hi,
Quoted 10 lines · James Morse
Hi Dave,
Quoted 7 lines · Dave Martin
Quoted 5 lines · James Morse
MPAM identifies CPUs by the
cache_idin the PPTT cache structure.The driver needs to know which CPUs are associated with the cache, the CPUs may not all be online, so cacheinfo does not have the information.
Quoted 8 lines
Quoted 6 lines
Nit: cacheinfo lacking the information is not a consequence of the driver needing it.
Maybe split the sentence:
-> "[...] associated with the cache. The CPUs may not [...]"
Sure,
OK
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
drivers/acpi/pptt.c +0
-0@@ -971,3 +971,65 @@ int find_acpi_cache_level_from_id(u32 cache_id)
[...]
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
+ *
acpi_pptt_get_cpumask_from_cache_id()- Get the cpus associated with the
+ * specified cache
+ * @cache_id: The id field of the unified cache
+ * @cpus: Where to build the cpumask
+ *
+ * Determine which CPUs are below this cache in the PPTT. This allows the property
+ * to be found even if the CPUs are offline.
+ *
+ * The PPTT table must be rev 3 or later,
+ *
+ * Return:-ENOENTif the PPTT doesn't exist, or the cache cannot be found.
+ * Otherwise returns 0 and sets the cpus in the provided cpumask.
+ */drivers/acpi/pptt.c +2
-0+int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus) +{
[...]
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
drivers/acpi/pptt.c +4
-0+/* +* If we found the cache first, we'd still need to walk from each cpu. +*/ +for_each_possible_cpu (cpu) {
[...]
Quoted 13 lines
Quoted 8 lines
Again, it feels like we are repeating the same walk multiple times to determine how deep the table is (on which point the table is self- describing anyway), and then again to derive some static property, and then we are then doing all of that work multiple times to derive different static properties, etc.
Can we not just walk over the tables once and stash the derived properties somewhere?
That is possible - but its a more invasive change to the PPTT parsing code. Before the introduction of the leaf flag, the search for a processor also included a search to check if the discovered node was a leaf.
Quoted 7 lines
I think this is trading time - walking over the table multiple times, against the memory you'd need to de-serialise the tree to find the necessary properties quickly. I think the reason Jeremy L went this way was because there may never be another request into this code, so being ready with a quick answer was a waste of memory.
MPAM doesn't change this - all these things are done up front during driver probing, and the values are cached by the driver.
I guess that's true.
Quoted 5 lines
Quoted 3 lines
I'm still getting my head around this parsing code, so I'm not saying that the approach is incorrect here -- just wondering whether there is a way to make it simpler.
It's walked at boot, and on cpu-hotplug. Neither are particularly performance critical.
Do we do this only for unknown late secondaries (e.g., that haven't previously come online?) I haven't gone to track this down but, if not, this cuts across the assertion that "there may never be another request into this code".
cpu hotlug is slow in practice, but gratuitous cost on this path should still be avoided where feasible.
Quoted 2 lines
I agree that as platforms get bigger, there will be a tipping point ... I don't think anyone has complained yet!
Ack -- when in ACPI, do as the ACPI folks do, I guess.
Cheers
---Dave
James Morse
maintainerRe: [PATCH 06/33] ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
Hi Dave,
Quoted 65 lines · Dave Martin
Quoted 13 lines · James Morse
Quoted 12 lines · Dave Martin
Quoted 11 lines · James Morse
MPAM identifies CPUs by the
cache_idin the PPTT cache structure.The driver needs to know which CPUs are associated with the cache, the CPUs may not all be online, so cacheinfo does not have the information.
drivers/acpi/pptt.c +0
-0@@ -971,3 +971,65 @@ int find_acpi_cache_level_from_id(u32 cache_id)[...]
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
+ *
acpi_pptt_get_cpumask_from_cache_id()- Get the cpus associated with the
+ * specified cache
+ * @cache_id: The id field of the unified cache
+ * @cpus: Where to build the cpumask
+ *
+ * Determine which CPUs are below this cache in the PPTT. This allows the property
+ * to be found even if the CPUs are offline.
+ *
+ * The PPTT table must be rev 3 or later,
+ *
+ * Return:-ENOENTif the PPTT doesn't exist, or the cache cannot be found.
+ * Otherwise returns 0 and sets the cpus in the provided cpumask.
+ */drivers/acpi/pptt.c +2
-0+int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus) +{[...]
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
drivers/acpi/pptt.c +4
-0+/* +* If we found the cache first, we'd still need to walk from each cpu. +*/ +for_each_possible_cpu (cpu) {[...]
Quoted 13 lines
Quoted 8 lines
Again, it feels like we are repeating the same walk multiple times to determine how deep the table is (on which point the table is self- describing anyway), and then again to derive some static property, and then we are then doing all of that work multiple times to derive different static properties, etc.
Can we not just walk over the tables once and stash the derived properties somewhere?
That is possible - but its a more invasive change to the PPTT parsing code. Before the introduction of the leaf flag, the search for a processor also included a search to check if the discovered node was a leaf.
Quoted 7 lines
I think this is trading time - walking over the table multiple times, against the memory you'd need to de-serialise the tree to find the necessary properties quickly. I think the reason Jeremy L went this way was because there may never be another request into this code, so being ready with a quick answer was a waste of memory.
MPAM doesn't change this - all these things are done up front during driver probing, and the values are cached by the driver.
I guess that's true.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
I'm still getting my head around this parsing code, so I'm not saying that the approach is incorrect here -- just wondering whether there is a way to make it simpler.
It's walked at boot, and on cpu-hotplug. Neither are particularly performance critical.
Do we do this only for unknown late secondaries (e.g., that haven't previously come online?)
No, each time a CPU comes online.
Quoted 3 lines
I haven't gone to track this down but, if not,
this cuts across the assertion that "there may never be another request into this code".
CPU hotplug is optional - you don't have to bounce CPUs. It's very common on mobile parts for power saving. I think its fairly unusual on server parts, once CPUs are online they stay online.
The cacheinfo code doesn't cache this, it re-reads it every time. That turns out to be because of PowerPC where some of these properties can be changed while a CPU is offline. Sure, we could have a Kconfig thing to say ARCH_STATIC_TABLES_ARE_STATIC, but that would be a different piece of work.
(I've had a couple of stabs at this, but cacheinfo is the shape it needs to be)
Quoted 7 lines
cpu hotlug is slow in practice, but gratuitous cost on this path should still be avoided where feasible.
Quoted 2 lines
I agree that as platforms get bigger, there will be a tipping point ... I don't think anyone has complained yet!
Ack -- when in ACPI, do as the ACPI folks do, I guess.
Thanks,
James
James Morse
maintainerPATCH 6/33 ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
In reply to James MorseMPAM identifies CPUs by the cache_id in the PPTT cache structure.
The driver needs to know which CPUs are associated with the cache, the CPUs may not all be online, so cacheinfo does not have the information.
Add a helper to pull this information out of the PPTT.
---
Changes since RFC:
* acpi_count_levels() now returns a value.
* Converted the table-get stuff to use Jonathan's cleanup helper.
* Dropped Sudeep's Review tag due to the cleanup change.
--- drivers/acpi/pptt.c | 62 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 6 +++++ 2 files changed, 68 insertions(+)
drivers/acpi/pptt.c +62 -0
@@ -971,3 +971,65 @@ int find_acpi_cache_level_from_id(u32 cache_id)return -ENOENT ; } + +/** +* acpi_pptt_get_cpumask_from_cache_id() - Get the cpus associated with the +* specified cache +* @cache_id: The id field of the unified cache +* @cpus: Where to build the cpumask +* +* Determine which CPUs are below this cache in the PPTT. This allows the property +* to be found even if the CPUs are offline. +* +* The PPTT table must be rev 3 or later, +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. +* Otherwise returns 0 and sets the cpus in the provided cpumask. +*/ +int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0); + +cpumask_clear (cpus); + +if (IS_ERR (table)) +return -ENOENT ; + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each cpu. +*/ +for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return 0 ; + num_levels =acpi_count_levels (table, cpu_node,NULL ); + +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node); +if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +cpumask_set_cpu (cpu, cpus); + } + } + +return 0 ; +}
include/linux/acpi.h +6 -0
@@ -1555,6 +1555,7 @@ int find_acpi_cpu_topology_package(unsigned int cpu);int find_acpi_cpu_topology_hetero_id (unsigned int cpu);void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus);int find_acpi_cache_level_from_id (u32 cache_id); +int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus);#else static inline int acpi_pptt_cpu_is_thread (unsigned int cpu) {
@@ -1582,6 +1583,11 @@ static inline int find_acpi_cache_level_from_id(u32 cache_id) {return -EINVAL ; } +static inline int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id, +cpumask_t *cpus) +{ +return -EINVAL ; +}#endif void acpi_arch_init (void );
Signature · 1 line
2.20.1
Re: [PATCH 06/33] ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
Quoted 59 lines · James Morse
MPAM identifies CPUs by the
cache_idin the PPTT cache structure.The driver needs to know which CPUs are associated with the cache, the CPUs may not all be online, so cacheinfo does not have the information.
Add a helper to pull this information out of the PPTT.
CC Rohit Mathew <[email protected]>Signed-off-by James Morse <[email protected]>Reviewed-by Sudeep Holla <[email protected]>---
Changes since RFC:
*acpi_count_levels()now returns a value.
* Converted the table-get stuff to use Jonathan's cleanup helper.
* Dropped Sudeep's Review tag due to the cleanup change.--- drivers/acpi/pptt.c | 62 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/acpi.h | 6 +++++ 2 files changed, 68 insertions(+)drivers/acpi/pptt.c +30
-0@@ -971,3 +971,65 @@ int find_acpi_cache_level_from_id(u32 cache_id)return -ENOENT ; } + +/** +* acpi_pptt_get_cpumask_from_cache_id() - Get the cpus associated with the +* specified cache +* @cache_id: The id field of the unified cache +* @cpus: Where to build the cpumask +* +* Determine which CPUs are below this cache in the PPTT. This allows the property +* to be found even if the CPUs are offline. +* +* The PPTT table must be rev 3 or later, +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. +* Otherwise returns 0 and sets the cpus in the provided cpumask. +*/ +int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0); + +cpumask_clear (cpus); + +if (IS_ERR (table)) +return -ENOENT ; + +if (table->revision <3 )
drivers/acpi/pptt.c int find_acpi_cache_level_from_id(u32 cache_id) · line 1013
Quoted 10 lines
drivers/acpi/pptt.c +10
-0+return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each cpu. +*/ +for_each_possible_cpu (cpu) { + acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return 0 ;
If for a possible cpu you don't get an acpi_pptt_processor node we return 0, is that correct ? Should not the loop continue ? Forgive me if that's a dumb question.
Quoted 35 lines
drivers/acpi/pptt.c +22
-0+ num_levels =acpi_count_levels (table, cpu_node,NULL ); + +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node); +if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +cpumask_set_cpu (cpu, cpus); + } + } + +return 0 ; +}include/linux/acpi.h +1
-0@@ -1555,6 +1555,7 @@ int find_acpi_cpu_topology_package(unsigned int cpu);int find_acpi_cpu_topology_hetero_id (unsigned int cpu);void acpi_pptt_get_cpus_from_container (u32 acpi_cpu_id,cpumask_t *cpus);int find_acpi_cache_level_from_id (u32 cache_id); +int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus);#else static inline int acpi_pptt_cpu_is_thread (unsigned int cpu) {
include/linux/acpi.h static inline int find_acpi_cache_level_from_id(u32 cache_id) · line 1589
Quoted 7 lines
include/linux/acpi.h +4
-0@@ -1582,6 +1583,11 @@ static inline int find_acpi_cache_level_from_id(u32 cache_id) {return -EINVAL ; } +static inline int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id, +cpumask_t *cpus) +{ +return -EINVAL ;
Nit: You might want the return value here to be coherent with what the function documentation states (ie return -ENOENT;)
Other than that:
Quoted 7 lines
include/linux/acpi.h +1
-0+}#endif void acpi_arch_init (void );Signature · 1 line
2.20.1
James Morse
maintainerRe: [PATCH 06/33] ACPI / PPTT: Add a helper to fill a cpumask from a cache_id
Hi Lorenzo,
Quoted 53 lines · Lorenzo Pieralisi
Quoted 52 lines · James Morse
MPAM identifies CPUs by the
cache_idin the PPTT cache structure.The driver needs to know which CPUs are associated with the cache, the CPUs may not all be online, so cacheinfo does not have the information.
Add a helper to pull this information out of the PPTT.
drivers/acpi/pptt.c +36
-0@@ -971,3 +971,65 @@ int find_acpi_cache_level_from_id(u32 cache_id)return -ENOENT ; } + +/** +* acpi_pptt_get_cpumask_from_cache_id() - Get the cpus associated with the +* specified cache +* @cache_id: The id field of the unified cache +* @cpus: Where to build the cpumask +* +* Determine which CPUs are below this cache in the PPTT. This allows the property +* to be found even if the CPUs are offline. +* +* The PPTT table must be rev 3 or later, +* +* Return: -ENOENT if the PPTT doesn't exist, or the cache cannot be found. +* Otherwise returns 0 and sets the cpus in the provided cpumask. +*/ +int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id,cpumask_t *cpus) +{ +u32 acpi_cpu_id; +int level, cpu, num_levels; +struct acpi_pptt_cache *cache; +struct acpi_pptt_cache_v1 *cache_v1; +struct acpi_pptt_processor *cpu_node; + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_PPTT, 0); + +cpumask_clear (cpus); + +if (IS_ERR (table)) +return -ENOENT ; + +if (table->revision <3 ) +return -ENOENT ; + +/* +* If we found the cache first, we'd still need to walk from each cpu. +*/ +for_each_possible_cpu (cpu) {
Quoted 8 lines
drivers/acpi/pptt.c int find_acpi_cache_level_from_id(u32 cache_id) · line 1013
Quoted 4 lines
drivers/acpi/pptt.c +4
-0+ acpi_cpu_id =get_acpi_id_for_cpu (cpu); + cpu_node =acpi_find_processor_node (table, acpi_cpu_id); +if (!cpu_node) +return 0 ;If for a possible cpu you don't get an
acpi_pptt_processornode we return 0, is that correct ? Should not the loop continue ? Forgive me if that's a dumb question.
That looks like me throwing my hands up in the air and bailing out! Yes, the loop continue-ing would be better as only possible CPUs that are missing a PPTT description (...and cache hierarchy...) would be missing form the bitmap.
It's probably worth a WARN_ON_ONCE() too.
Thanks for spotting that!
Quoted 28 lines
Quoted 27 lines
drivers/acpi/pptt.c +22
-0+ num_levels =acpi_count_levels (table, cpu_node,NULL ); + +/* Start at 1 for L1 */ +for (level =1 ; level <= num_levels; level++) { + cache =acpi_find_cache_node (table, acpi_cpu_id, +ACPI_PPTT_CACHE_TYPE_UNIFIED , + level, &cpu_node); +if (!cache) +continue ; + + cache_v1 = ACPI_ADD_PTR(struct acpi_pptt_cache_v1, + cache, +sizeof (struct acpi_pptt_cache )); + +if (cache->flags &ACPI_PPTT_CACHE_ID_VALID && + cache_v1->cache_id == cache_id) +cpumask_set_cpu (cpu, cpus); + } + } + +return 0 ; +}include/linux/acpi.h +0
-0@@ -1555,6 +1555,7 @@ int find_acpi_cpu_topology_package(unsigned int cpu);
Quoted 11 lines
include/linux/acpi.h static inline int find_acpi_cache_level_from_id(u32 cache_id) · line 1589
Quoted 8 lines
include/linux/acpi.h +4
-0@@ -1582,6 +1583,11 @@ static inline int find_acpi_cache_level_from_id(u32 cache_id) {return -EINVAL ; } +static inline int acpi_pptt_get_cpumask_from_cache_id (u32 cache_id, +cpumask_t *cpus) +{ +return -EINVAL ;Nit: You might want the return value here to be coherent with what the function documentation states (ie return
-ENOENT;)
Makes sense,
Quoted 3 lines
Other than that:
Reviewed-by Lorenzo Pieralisi <[email protected]>
Thanks!
James
James Morse
maintainerPATCH 7/33 arm64: kconfig: Add Kconfig entry for MPAM
In reply to James MorseThe bulk of the MPAM driver lives outside the arch code because it largely manages MMIO devices that generate interrupts. The driver needs a Kconfig symbol to enable it, as MPAM is only found on arm64 platforms, that is where the Kconfig option makes the most sense.
This Kconfig option will later be used by the arch code to enable or disable the MPAM context-switch code, and registering the CPUs properties with the MPAM driver.
--- arch/arm64/Kconfig | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
arch/arm64/Kconfig +17 -0
@@ -2060,6 +2060,23 @@ config ARM64_TLB_RANGE ARMv8.4-TLBI provides TLBI invalidation instruction that apply to a range of input addresses. +config ARM64_MPAM + bool "Enable support for MPAM" + help + Memory Partitioning and Monitoring is an optional extension + that allows the CPUs to mark load and store transactions with + labels for partition-id and performance-monitoring-group. + System components, such as the caches, can use the partition-id + to apply a performance policy. MPAM monitors can use the + partition-id and performance-monitoring-group to measure the + cache occupancy or data throughput. + + Use of this extension requires CPU support, support in the + memory system components (MSC), and a description from firmware + of where the MSC are in the address space. + + MPAM is exposed to user-space via the resctrl pseudo filesystem. + endmenu # "ARMv8.4 architectural features" menu "ARMv8.5 architectural features"
Signature · 1 line
2.20.1
Ben Horgan
maintainerRe: [PATCH 07/33] arm64: kconfig: Add Kconfig entry for MPAM
Hi James,
Quoted 31 lines · James Morse
The bulk of the MPAM driver lives outside the arch code because it largely manages MMIO devices that generate interrupts. The driver needs a
Kconfigsymbol to enable it, as MPAM is only found on arm64 platforms, that is where theKconfigoption makes the most sense.This
Kconfigoption will later be used by the arch code to enable or disable the MPAM context-switch code, and registering the CPUs properties with the MPAM driver.Signed-off-by James Morse <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>--- arch/arm64/Kconfig | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)arch/arm64/Kconfig +8
-0@@ -2060,6 +2060,23 @@ config ARM64_TLB_RANGE ARMv8.4-TLBI provides TLBI invalidation instruction that apply to a range of input addresses. +config ARM64_MPAM + bool "Enable support for MPAM" + help + Memory Partitioning and Monitoring is an optional extension + that allows the CPUs to mark load and store transactions with + labels for partition-id and performance-monitoring-group. + System components, such as the caches, can use the partition-id + to apply a performance policy. MPAM monitors can use the
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2080
Quoted 10 lines
arch/arm64/Kconfig +9
-0+ partition-id and performance-monitoring-group to measure the + cache occupancy or data throughput. + + Use of this extension requires CPU support, support in the + memory system components (MSC), and a description from firmware + of where the MSC are in the address space. + + MPAM is exposed to user-space via the resctrl pseudo filesystem. + endmenu # "ARMv8.4 architectural features"
Should this be moved to "ARMv8.2 architectural features" rather than the 8.4 menu? In the arm reference manual, version L.b, I see FEAT_MPAM listed in the section A2.2.3.1 Features added to the Armv8.2 extension in later releases.
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2082
Quoted 2 lines
arch/arm64/Kconfig +0
-0menu "ARMv8.5 architectural features"
Thanks,
Ben
James Morse
maintainerRe: [PATCH 07/33] arm64: kconfig: Add Kconfig entry for MPAM
Hi Ben,
Quoted 33 lines · Ben Horgan
Quoted 32 lines · James Morse
The bulk of the MPAM driver lives outside the arch code because it largely manages MMIO devices that generate interrupts. The driver needs a
Kconfigsymbol to enable it, as MPAM is only found on arm64 platforms, that is where theKconfigoption makes the most sense.This
Kconfigoption will later be used by the arch code to enable or disable the MPAM context-switch code, and registering the CPUs properties with the MPAM driver.arch/arm64/Kconfig +15
-0@@ -2060,6 +2060,23 @@ config ARM64_TLB_RANGE ARMv8.4-TLBI provides TLBI invalidation instruction that apply to a range of input addresses. +config ARM64_MPAM + bool "Enable support for MPAM" + help + Memory Partitioning and Monitoring is an optional extension + that allows the CPUs to mark load and store transactions with + labels for partition-id and performance-monitoring-group. + System components, such as the caches, can use the partition-id + to apply a performance policy. MPAM monitors can use the + partition-id and performance-monitoring-group to measure the + cache occupancy or data throughput. + + Use of this extension requires CPU support, support in the + memory system components (MSC), and a description from firmware + of where the MSC are in the address space. +
Quoted 8 lines
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2080
Quoted 3 lines
arch/arm64/Kconfig +2
-0+ MPAM is exposed to user-space via the resctrl pseudo filesystem. + endmenu # "ARMv8.4 architectural features"Should this be moved to "ARMv8.2 architectural features" rather than the 8.4 menu? In the arm reference manual, version L.b, I see
FEAT_MPAMlisted in the section A2.2.3.1 Features added to the Armv8.2 extension in later releases.
Hmmm, I don't think we've done that anywhere else. I'm only aware of one v8.2 platform that had it, and those are not widely available. As it was a headline v8.4 feature I'd prefer to keep it there.
I think its more confusing to put it under v8.2!
Thanks,
James
Ben Horgan
maintainerRe: [PATCH 07/33] arm64: kconfig: Add Kconfig entry for MPAM
Hi James,
Quoted 45 lines · James Morse
Hi Ben,
Quoted 33 lines · Ben Horgan
Quoted 32 lines · James Morse
The bulk of the MPAM driver lives outside the arch code because it largely manages MMIO devices that generate interrupts. The driver needs a
Kconfigsymbol to enable it, as MPAM is only found on arm64 platforms, that is where theKconfigoption makes the most sense.This
Kconfigoption will later be used by the arch code to enable or disable the MPAM context-switch code, and registering the CPUs properties with the MPAM driver.arch/arm64/Kconfig +15
-0@@ -2060,6 +2060,23 @@ config ARM64_TLB_RANGE ARMv8.4-TLBI provides TLBI invalidation instruction that apply to a range of input addresses. +config ARM64_MPAM + bool "Enable support for MPAM" + help + Memory Partitioning and Monitoring is an optional extension + that allows the CPUs to mark load and store transactions with + labels for partition-id and performance-monitoring-group. + System components, such as the caches, can use the partition-id + to apply a performance policy. MPAM monitors can use the + partition-id and performance-monitoring-group to measure the + cache occupancy or data throughput. + + Use of this extension requires CPU support, support in the + memory system components (MSC), and a description from firmware + of where the MSC are in the address space. +Quoted 8 lines
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2080
Quoted 3 lines
arch/arm64/Kconfig +2
-0+ MPAM is exposed to user-space via the resctrl pseudo filesystem. + endmenu # "ARMv8.4 architectural features"Should this be moved to "ARMv8.2 architectural features" rather than the 8.4 menu? In the arm reference manual, version L.b, I see
FEAT_MPAMlisted in the section A2.2.3.1 Features added to the Armv8.2 extension in later releases.
Quoted 5 lines
Hmmm, I don't think we've done that anywhere else. I'm only aware of one v8.2 platform that had it, and those are not widely available. As it was a headline v8.4 feature I'd prefer to keep it there.
I think its more confusing to put it under v8.2!
Ok, always best to minimise confusion. Keep it in v8.4.
Quoted 4 lines
Thanks,
James
Signature · 3 lines
Thanks, Ben
Re: [PATCH 07/33] arm64: kconfig: Add Kconfig entry for MPAM
In reply to James MorseHi,
<super-pedantic mode enabled>
(Since this likely be people's go-to patch for understanding what MPAM is, it is probably worth going the extra mile.)
Quoted 3 lines · James Morse
The bulk of the MPAM driver lives outside the arch code because it largely manages MMIO devices that generate interrupts. The driver needs a
Kconfigsymbol to enable it, as MPAM is only found on arm64
Prefer -> "[...] to enable it. As MPAM is only [...]"
Quoted 1 line
platforms, that is where the
Kconfigoption makes the most sense.
It could be clearer what "where" refers to, here.
Maybe reword from ", that is [...]" -> ", the arm64 tree is the most natural home for the Kconfig option."
(Or something like that.)
Quoted 2 lines
This
Kconfigoption will later be used by the arch code to enable or disable the MPAM context-switch code, and registering the CPUs
Nit: "registering" -> "to register"
Quoted 1 line
properties with the MPAM driver.
Nit: "CPUs properties" -> "properties of CPUs" ?
(Maybe there was just a missed apostrophe, but it may be more readable here if written out longhand.)
Quoted 11 lines
Signed-off-by James Morse <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>--- arch/arm64/Kconfig | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)arch/arm64/Kconfig +0
-0
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2065
Quoted 6 lines
arch/arm64/Kconfig +3
-0@@ -2060,6 +2060,23 @@ config ARM64_TLB_RANGE ARMv8.4-TLBI provides TLBI invalidation instruction that apply to a range of input addresses. +config ARM64_MPAM + bool "Enable support for MPAM" + help
<pedantic mode on>
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2067
Quoted 2 lines
arch/arm64/Kconfig +2
-0+ Memory Partitioning and Monitoring is an optional extension + that allows the CPUs to mark load and store transactions with
Nit: "memory transactions" ?
(I'm wondering whether there are some transactions such as atomic exchanges that are not neatly characterised as "load" or "store". Possibly MPAM labels some transactions that really are neither.)
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2068
Quoted 1 line
arch/arm64/Kconfig +1
-0+ labels for partition-id and performance-monitoring-group.
Nit: the hyphenation suggests that these are known terms (in this specific, hyphenated, form) with specific definitions somewhere. I don't think that this is the case? At least, I have not seen the terms presented in this way anywhere else.
Also, the partition ID is itself a label, so "label for partition-id" is a tautology.
How about:
--8<--
Memory System Resource Partitioning and Monitoring (MPAM) is an optional extension to the Arm architecture that allows each transaction issued to the memory system to be labelled with a Partition identifier (PARTID) and Performance Monitoring Group identifier (PMG).
-->8--
(Yes, that really seems to be what MPAM stands for in the published specs. That's quite a mounthful, and news to me... I can't say I paid much attention to the document titles beyond "MPAM"!)
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2070
Quoted 2 lines
arch/arm64/Kconfig +2
-0+ System components, such as the caches, can use the partition-id + to apply a performance policy. MPAM monitors can use the
What is a "performance policy"?
The MPAM specs talk about resource controls; it's probably best to stick to the same terminology.
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2072
Quoted 2 lines
arch/arm64/Kconfig +2
-0+ partition-id and performance-monitoring-group to measure the + cache occupancy or data throughput.
So, how about something like:
--8<--
Memory system components, such as the caches, can be configured with policies to control how much of various physical resources (such as memory bandwidth or cache memory) the transactions labelled with each PARTID can consume. Depending on the capabilities of the hardware, the PARTID and PMG can also be used as filtering criteria to measure the memory system resource consumption of different parts of a workload.
-->8--
(Where "Memory system components" is used in a generic sense and so not capitalised.)
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2075
Quoted 3 lines
arch/arm64/Kconfig +3
-0+ + Use of this extension requires CPU support, support in the + memory system components (MSC), and a description from firmware
But here, we are explicitly using an architectural term now, so
"Memory System Components" (MSC)
makes sense.
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2076
Quoted 1 line
arch/arm64/Kconfig +1
-0+ of where the MSC are in the address space.
Prefer "MSCs" ? (Not everyone agrees about whether TLAs are pluralisable but it is easier on the reader if "are" has an obviously plural noun to bind to.)
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2082
Quoted 6 lines
arch/arm64/Kconfig +3
-0+ + MPAM is exposed to user-space via the resctrl pseudo filesystem. + endmenu # "ARMv8.4 architectural features" menu "ARMv8.5 architectural features"
Cheers
---Dave
James Morse
maintainerRe: [PATCH 07/33] arm64: kconfig: Add Kconfig entry for MPAM
Hi Dave,
Quoted 1 line · Dave Martin
<super-pedantic mode enabled>
Uh oh!
Quoted 8 lines
(Since this likely be people's go-to patch for understanding what MPAM is, it is probably worth going the extra mile.)
Quoted 3 lines · James Morse
The bulk of the MPAM driver lives outside the arch code because it largely manages MMIO devices that generate interrupts. The driver needs a
Kconfigsymbol to enable it, as MPAM is only found on arm64
Quoted 5 lines
Prefer -> "[...] to enable it. As MPAM is only [...]"
Quoted 1 line
platforms, that is where the
Kconfigoption makes the most sense.It could be clearer what "where" refers to, here.
Sure,
Quoted 4 lines
Maybe reword from ", that is [...]" -> ", the arm64 tree is the most natural home for the
Kconfigoption."(Or something like that.)
Sure,
Quoted 11 lines
Quoted 2 lines
This
Kconfigoption will later be used by the arch code to enable or disable the MPAM context-switch code, and registering the CPUsNit: "registering" -> "to register"
Quoted 1 line
properties with the MPAM driver.
Nit: "CPUs properties" -> "properties of CPUs" ?
(Maybe there was just a missed apostrophe, but it may be more readable here if written out longhand.)
Done, it just takes one person to think its clearer!
Quoted 14 lines
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2065
Quoted 11 lines
arch/arm64/Kconfig +3
-0@@ -2060,6 +2060,23 @@ config ARM64_TLB_RANGE ARMv8.4-TLBI provides TLBI invalidation instruction that apply to a range of input addresses. +config ARM64_MPAM + bool "Enable support for MPAM" + help<pedantic mode on>
Quoted 4 lines
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2067
Quoted 2 lines
arch/arm64/Kconfig +2
-0+ Memory Partitioning and Monitoring is an optional extension + that allows the CPUs to mark load and store transactions withNit: "memory transactions" ?
Sure,
Quoted 3 lines
(I'm wondering whether there are some transactions such as atomic exchanges that are not neatly characterised as "load" or "store". Possibly MPAM labels some transactions that really are neither.)
Equally instruction fetch and possibly even CMOs get these labels. I wanted something other than 'transactions' so it wasn't confused with transactional memory - and traffic seemed to vauge.
I don't think anyone expects a formal definition in the Kconfig text...
Quoted 14 lines
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2068
Quoted 1 line
arch/arm64/Kconfig +1
-0+ labels for partition-id and performance-monitoring-group.Nit: the hyphenation suggests that these are known terms (in this specific, hyphenated, form) with specific definitions somewhere. I don't think that this is the case? At least, I have not seen the terms presented in this way anywhere else.
Also, the partition ID is itself a label, so "label for partition-id" is a tautology.
How about:
--8<--
Quoted 7 lines
Memory System Resource Partitioning and Monitoring (MPAM) is an optional extension to the Arm architecture that allows each transaction issued to the memory system to be labelled with a Partition identifier (PARTID) and Performance Monitoring Group identifier (PMG).
-->8--
Done,
Quoted 8 lines
(Yes, that really seems to be what MPAM stands for in the published specs. That's quite a mounthful, and news to me... I can't say I paid much attention to the document titles beyond "MPAM"!)
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2070
Quoted 2 lines
arch/arm64/Kconfig +2
-0+ System components, such as the caches, can use the partition-id + to apply a performance policy. MPAM monitors can use theWhat is a "performance policy"?
A bunch of controls, the value of which reflect some kind of policy.
Quoted 18 lines
The MPAM specs talk about resource controls; it's probably best to stick to the same terminology.
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2072
Quoted 2 lines
arch/arm64/Kconfig +2
-0+ partition-id and performance-monitoring-group to measure the + cache occupancy or data throughput.So, how about something like:
--8<--
Memory system components, such as the caches, can be configured with policies to control how much of various physical resources (such as memory bandwidth or cache memory) the transactions labelled with each PARTID can consume. Depending on the capabilities of the hardware, the PARTID and PMG can also be used as filtering criteria to measure the memory system resource consumption of different parts of a workload.
Quoted 1 line
-->8--
Done,
Quoted 2 lines
(Where "Memory system components" is used in a generic sense and so not capitalised.)
(I can't wait for the Memory System Component on the Memory Side Cache!)
Quoted 8 lines
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2075
Quoted 3 lines
arch/arm64/Kconfig +3
-0+ + Use of this extension requires CPU support, support in the + memory system components (MSC), and a description from firmwareBut here, we are explicitly using an architectural term now, so
"Memory System Components" (MSC)
Quoted 7 lines
makes sense.
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2076
Quoted 1 line
arch/arm64/Kconfig +1
-0+ of where the MSC are in the address space.Prefer "MSCs" ? (Not everyone agrees about whether TLAs are pluralisable but it is easier on the reader if "are" has an obviously plural noun to bind to.)
Sure,
Thanks,
James
Re: [PATCH 07/33] arm64: kconfig: Add Kconfig entry for MPAM
Hi,
Quoted 24 lines · James Morse
Hi Dave,
Quoted 1 line · Dave Martin
<super-pedantic mode enabled>
Uh oh!
Quoted 8 lines
(Since this likely be people's go-to patch for understanding what MPAM is, it is probably worth going the extra mile.)
Quoted 3 lines · James Morse
The bulk of the MPAM driver lives outside the arch code because it largely manages MMIO devices that generate interrupts. The driver needs a
Kconfigsymbol to enable it, as MPAM is only found on arm64Quoted 5 lines
Prefer -> "[...] to enable it. As MPAM is only [...]"
Quoted 1 line
platforms, that is where the
Kconfigoption makes the most sense.It could be clearer what "where" refers to, here.
Sure,
Quoted 6 lines
Quoted 4 lines
Maybe reword from ", that is [...]" -> ", the arm64 tree is the most natural home for the
Kconfigoption."(Or something like that.)
Sure,
[... etc., etc. ...]
Quoted 17 lines
Quoted 15 lines
arch/arm64/Kconfig config ARM64_TLB_RANGE · line 2072
Quoted 2 lines
arch/arm64/Kconfig +2
-0+ partition-id and performance-monitoring-group to measure the + cache occupancy or data throughput.So, how about something like:
--8<--
Memory system components, such as the caches, can be configured with policies to control how much of various physical resources (such as memory bandwidth or cache memory) the transactions labelled with each PARTID can consume. Depending on the capabilities of the hardware, the PARTID and PMG can also be used as filtering criteria to measure the memory system resource consumption of different parts of a workload.
Quoted 1 line
-->8--
Quoted 7 lines
Done,
Quoted 2 lines
(Where "Memory system components" is used in a generic sense and so not capitalised.)
(I can't wait for the Memory System Component on the Memory Side Cache!)
Urk.
MSC² ?
[...]
Cheers
---Dave
James Morse
maintainerPATCH 7/33 arm64: kconfig: Add Kconfig entry for MPAM
In reply to James MorseThe bulk of the MPAM driver lives outside the arch code because it largely manages MMIO devices that generate interrupts. The driver needs a Kconfig symbol to enable it, as MPAM is only found on arm64 platforms, that is where the Kconfig option makes the most sense.
This Kconfig option will later be used by the arch code to enable or disable the MPAM context-switch code, and registering the CPUs properties with the MPAM driver.
--- arch/arm64/Kconfig | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
arch/arm64/Kconfig +17 -0
@@ -2060,6 +2060,23 @@ config ARM64_TLB_RANGE ARMv8.4-TLBI provides TLBI invalidation instruction that apply to a range of input addresses. +config ARM64_MPAM + bool "Enable support for MPAM" + help + Memory Partitioning and Monitoring is an optional extension + that allows the CPUs to mark load and store transactions with + labels for partition-id and performance-monitoring-group. + System components, such as the caches, can use the partition-id + to apply a performance policy. MPAM monitors can use the + partition-id and performance-monitoring-group to measure the + cache occupancy or data throughput. + + Use of this extension requires CPU support, support in the + memory system components (MSC), and a description from firmware + of where the MSC are in the address space. + + MPAM is exposed to user-space via the resctrl pseudo filesystem. + endmenu # "ARMv8.4 architectural features" menu "ARMv8.5 architectural features"
Signature · 1 line
2.20.1
James Morse
maintainerPATCH 8/33 ACPI / MPAM: Parse the MPAM table
In reply to James MorseAdd code to parse the arm64 specific MPAM table, looking up the cache level from the PPTT and feeding the end result into the MPAM driver.
---
Changes since RFC:
* Used DEFINE_RES_IRQ_NAMED() and friends macros.
* Additional error handling.
* Check for zero sized MSC.
* Allow table revisions greater than 1. (no spec for revision 0!)
* Use cleanup helpers to retrive ACPI tables, which allows some functions to be folded together.
--- arch/arm64/Kconfig | 1 + drivers/acpi/arm64/Kconfig | 3 + drivers/acpi/arm64/Makefile | 1 + drivers/acpi/arm64/mpam.c | 331 ++++++++++++++++++++++++++++++++++++ drivers/acpi/tables.c | 2 +- include/linux/arm_mpam.h | 46 +++++ 6 files changed, 383 insertions(+), 1 deletion(-) create mode 100644 drivers/acpi/arm64/mpam.c create mode 100644 include/linux/arm_mpam.h
arch/arm64/Kconfig +1 -0
@@ -2062,6 +2062,7 @@ config ARM64_TLB_RANGE config ARM64_MPAM bool "Enable support for MPAM" + select ACPI_MPAM if ACPI help Memory Partitioning and Monitoring is an optional extension that allows the CPUs to mark load and store transactions with
drivers/acpi/arm64/Kconfig +3 -0
@@ -21,3 +21,6 @@ config ACPI_AGDI config ACPI_APMT bool + +config ACPI_MPAM + bool
drivers/acpi/arm64/Makefile +1 -0
@@ -4,6 +4,7 @@ obj-$(CONFIG_ACPI_APMT) += apmt.o obj-$(CONFIG_ACPI_FFH) += ffh.o obj-$(CONFIG_ACPI_GTDT) += gtdt.o obj-$(CONFIG_ACPI_IORT) += iort.o +obj-$(CONFIG_ACPI_MPAM) += mpam.o obj-$(CONFIG_ACPI_PROCESSOR_IDLE) += cpuidle.o obj-$(CONFIG_ARM_AMBA) += amba.o obj-y += dma.o init.o
drivers/acpi/arm64/mpam.c new file +331 -0
331 lines not on this page · raw message
drivers/acpi/tables.c +1 -1
@@ -408,7 +408,7 @@ static const char table_sigs[][ACPI_NAMESEG_SIZE] __nonstring_array __initconstACPI_SIG_PSDT ,ACPI_SIG_RSDT ,ACPI_SIG_XSDT ,ACPI_SIG_SSDT ,ACPI_SIG_IORT ,ACPI_SIG_NFIT ,ACPI_SIG_HMAT ,ACPI_SIG_PPTT ,ACPI_SIG_NHLT ,ACPI_SIG_AEST ,ACPI_SIG_CEDT ,ACPI_SIG_AGDI , - ACPI_SIG_NBFT }; + ACPI_SIG_NBFT, ACPI_SIG_MPAM };#define ACPI_HEADER_SIZE sizeof(struct acpi_table_header)
include/linux/arm_mpam.h new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH 08/33] ACPI / MPAM: Parse the MPAM table
…
Quoted 2 lines
+++
b/drivers/acpi/arm64/mpam.c(quoted diff) +0
-0@@ -0,0 +1,331 @@
…
drivers/acpi/arm64/mpam.c line 178
Quoted 4 lines
drivers/acpi/arm64/mpam.c new file +4
-0+static int __init acpi_mpam_parse (void ) +{ + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0); +char *table_end, *table_offset = (char *)(table +1 );
…
Please replace eight space characters by a tab character here.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/coding-style.rst?h=v6.17-rc2#n18
Are further source code places similarly improvable?
Regards,
Markus
James Morse
maintainerRe: [PATCH 08/33] ACPI / MPAM: Parse the MPAM table
Hi Markus,
Quoted 10 lines · Markus Elfring
…
Quoted 2 lines
+++
b/drivers/acpi/arm64/mpam.c(quoted diff) +0
-0@@ -0,0 +1,331 @@…
Quoted 4 lines
drivers/acpi/arm64/mpam.c new file +4
-0+static int __init acpi_mpam_parse (void ) +{ + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0); +char *table_end, *table_offset = (char *)(table +1 );…
Quoted 4 lines
Please replace eight space characters by a tab character here.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/coding-style.rst?h=v6.17-rc2#n18Are further source code places similarly improvable?
Yes - I just copied this rune from the email from Jonathan that suggested it. 'Fixed',
Thanks,
James
Re: [PATCH 08/33] ACPI / MPAM: Parse the MPAM table
In reply to James MorseHi,
[Note, looks like I crossed over with Rob here -- apologies for any duplicate or conflicting comments.]
Quoted 2 lines · James Morse
Add code to parse the arm64 specific MPAM table, looking up the cache level from the PPTT and feeding the end result into the MPAM driver.
Might be worth mentioning that the hook for feeding the parsed factoids into the driver (mpam_ris_create()) is not implemented for now.
Quoted 82 lines
CC Carl Worth <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* UsedDEFINE_RES_IRQ_NAMED()and friends macros.
* Additional error handling.
* Check for zero sized MSC.
* Allow table revisions greater than 1. (no spec for revision 0!)
* Use cleanup helpers to retrive ACPI tables, which allows some functions to be folded together.--- arch/arm64/Kconfig | 1 + drivers/acpi/arm64/Kconfig | 3 + drivers/acpi/arm64/Makefile | 1 + drivers/acpi/arm64/mpam.c | 331 ++++++++++++++++++++++++++++++++++++ drivers/acpi/tables.c | 2 +- include/linux/arm_mpam.h | 46 +++++ 6 files changed, 383 insertions(+), 1 deletion(-) create mode 100644 drivers/acpi/arm64/mpam.c create mode 100644 include/linux/arm_mpam.harch/arm64/Kconfig +1
-0@@ -2062,6 +2062,7 @@ config ARM64_TLB_RANGE config ARM64_MPAM bool "Enable support for MPAM" + select ACPI_MPAM if ACPI help Memory Partitioning and Monitoring is an optional extension that allows the CPUs to mark load and store transactions withdrivers/acpi/arm64/Kconfig +3
-0@@ -21,3 +21,6 @@ config ACPI_AGDI config ACPI_APMT bool + +config ACPI_MPAM + booldrivers/acpi/arm64/Makefile +1
-0@@ -4,6 +4,7 @@ obj-$(CONFIG_ACPI_APMT) += apmt.o obj-$(CONFIG_ACPI_FFH) += ffh.o obj-$(CONFIG_ACPI_GTDT) += gtdt.o obj-$(CONFIG_ACPI_IORT) += iort.o +obj-$(CONFIG_ACPI_MPAM) += mpam.o obj-$(CONFIG_ACPI_PROCESSOR_IDLE) += cpuidle.o obj-$(CONFIG_ARM_AMBA) += amba.o obj-y += dma.o init.odrivers/acpi/arm64/mpam.c new file +18
-0@@ -0,0 +1,331 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2025 Arm Ltd. + +/* Parse the MPAM ACPI table feeding the discovered nodes into the driver */ + +#define pr_fmt (fmt) "ACPI MPAM: " fmt + +#include <linux/acpi.h> +#include <linux/arm_mpam.h> +#include <linux/bits.h> +#include <linux/cpu.h> +#include <linux/cpumask.h> +#include <linux/platform_device.h> + +#include <acpi/processor.h> + +/* +* Flags for acpi_table_mpam_msc.*_interrupt_flags.
drivers/acpi/arm64/mpam.c line 28
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+* See 2.1.1 Interrupt Flags, Table 5, of DEN0065B_MPAM_ACPI_3.0-bet. +*/ +#define ACPI_MPAM_MSC_IRQ_MODE_MASK BIT(0) +#define ACPI_MPAM_MSC_IRQ_TYPE_MASK GENMASK(2, 1) +#define ACPI_MPAM_MSC_IRQ_TYPE_WIRED 0 +#define ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER BIT(3) +#define ACPI_MPAM_MSC_IRQ_AFFINITY_VALID BIT(4) + +static bool frob_irq (struct platform_device *pdev,int intid,u32 flags, +int *irq,u32 processor_container_uid)
Can this have a name, please?
drivers/acpi/arm64/mpam.c line 39
Quoted 11 lines
drivers/acpi/arm64/mpam.c new file +11
-0+{ +int sense; + +if (!intid) +return false; + +if (FIELD_GET (ACPI_MPAM_MSC_IRQ_TYPE_MASK , flags) != +ACPI_MPAM_MSC_IRQ_TYPE_WIRED ) +return false; + + sense =FIELD_GET (ACPI_MPAM_MSC_IRQ_MODE_MASK , flags);
Do we handle cross-endian ACPI tables?
ACPI defers to the relevant specification regarding the endianness of externally defined tables, but as of v3.0 (beta) of for MPAM ACPI spec [1], no statement is made about this.
Following the spirit of the ACPI core specs, I suspect that the "correct" answer is that MPAM tables are always little-endian, even if it not written down anywhere.
If the kernel is big-endian, we lose.
Maybe it is sufficient to make CONFIG_ACPI_MPAM depend on !CONFIG_CPU_BIG_ENDIAN for now.
I haven't tried to understand how this is handled for other tables.
drivers/acpi/arm64/mpam.c line 43
Quoted 4 lines
drivers/acpi/arm64/mpam.c new file +4
-0+ +/* +* If the GSI is in the GIC's PPI range, try and create a partitioned +* percpu interrupt.
But actually we don't even try? Or did I miss something?
drivers/acpi/arm64/mpam.c line 45
Quoted 2 lines
drivers/acpi/arm64/mpam.c new file +2
-0+*/ +if (16 <= intid && intid <32 && processor_container_uid != ~0 ) {
checkpatch.pl says:
| WARNING: Comparisons should place the constant on the right side of the test
| #108: FILE: drivers/acpi/arm64/mpam.c:45:
| + if (16 <= intid && intid < 32 && processor_container_uid != ~0) {(Dubious whether this is "wrong" IMHO, but still probably best avoided since it is not what people are used to seeing.)
drivers/acpi/arm64/mpam.c line 53
Quoted 8 lines
drivers/acpi/arm64/mpam.c new file +8
-0+pr_err_once ("Partitioned interrupts not supported\n" ); +return false; + } + + *irq =acpi_register_gsi (&pdev->dev, intid, sense,ACPI_ACTIVE_HIGH ); +if (*irq <=0 ) { +pr_err_once ("Failed to register interrupt 0x%x with ACPI\n" , + intid);
Are we going to get a lot of duplicate error messages with the same interrupt? If not, perhaps make this a pr_err() so that all the affected interrupts are notified?
(Either way, hopefully the user will take the hint that they messed something up though.)
drivers/acpi/arm64/mpam.c line 65
Quoted 12 lines
drivers/acpi/arm64/mpam.c new file +12
-0+return false; + } + +return true; +} + +static void acpi_mpam_parse_irqs (struct platform_device *pdev, +struct acpi_mpam_msc_node *tbl_msc, +struct resource *res,int *res_idx) +{ +u32 flags, aff; +int irq;
We may still get in here if MPAMF_IDR.HAS_ERR_MSI and/or MPAMF_MSMON_IDR.HAS_OFLW_MSI is set. If so, there is no wired interrupt. Does it matter if we still parse and allocate the wired interrupts here?
drivers/acpi/arm64/mpam.c line 72
Quoted 7 lines
drivers/acpi/arm64/mpam.c new file +7
-0+ + flags = tbl_msc->overflow_interrupt_flags; +if (flags &ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER ) + aff = tbl_msc->overflow_interrupt_affinity; +else + aff = ~0 ;
(u32)~0 is used as an exceptional UID all over the place. If this is not a pre-existing convention, it could be worth having a #define for this. (grep '~0' drivers/acpi/ suggests that this is new.)
drivers/acpi/arm64/mpam.c line 74
Quoted 2 lines
drivers/acpi/arm64/mpam.c new file +2
-0+if (frob_irq (pdev, tbl_msc->overflow_interrupt, flags, &irq, aff)) + res[(*res_idx)++] =DEFINE_RES_IRQ_NAMED (irq,"overflow" );
I couldn't find a statement in the spec of how the table can specify that there is no interrupt.
Are the interrupts always required for ACPI-based MPAM systems?
overflow_interrupt and error_interrupt are GSIVs, which seems to be an ACPI thing. The examples in the ACPI spec suggest that 0 can be a valid value. No exceptional value seems to be defined.
The flags fields have some invalid encodings, but no explicit "no interrupt" encoding that I can see.
Quoted 36 lines
drivers/acpi/arm64/mpam.c new file +36
-0+ + flags = tbl_msc->error_interrupt_flags; +if (flags &ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER ) + aff = tbl_msc->error_interrupt_affinity; +else + aff = ~0 ; +if (frob_irq (pdev, tbl_msc->error_interrupt, flags, &irq, aff)) + res[(*res_idx)++] =DEFINE_RES_IRQ_NAMED (irq,"error" ); +} + +static int acpi_mpam_parse_resource (struct mpam_msc *msc, +struct acpi_mpam_resource_node *res) +{ +int level, nid; +u32 cache_id; + +switch (res->locator_type) { +case ACPI_MPAM_LOCATION_TYPE_PROCESSOR_CACHE : + cache_id = res->locator.cache_locator.cache_reference; + level =find_acpi_cache_level_from_id (cache_id); +if (level <=0 ) { +pr_err_once ("Bad level (%u) for cache with id %u\n" , level, cache_id); +return -EINVAL ; + } +return mpam_ris_create (msc, res->ris_index,MPAM_CLASS_CACHE , + level, cache_id); +case ACPI_MPAM_LOCATION_TYPE_MEMORY : + nid =pxm_to_node (res->locator.memory_locator.proximity_domain); +if (nid ==NUMA_NO_NODE ) + nid =0 ; +return mpam_ris_create (msc, res->ris_index,MPAM_CLASS_MEMORY , +255 , nid); +default : +/* These get discovered later and treated as unknown */ +return 0 ;
drivers/acpi/arm64/mpam.c line 120
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+ } +} + +int acpi_mpam_parse_resources (struct mpam_msc *msc, +struct acpi_mpam_msc_node *tbl_msc) +{ +int i, err; +struct acpi_mpam_resource_node *resources; + + resources = (struct acpi_mpam_resource_node *)(tbl_msc +1 );
Should we check that we don't go out of the bounds of the MSC node (or, at the very least, of the MPAM table)?
If tbl_msc->length was already validated, that can be used for the bounds check.
drivers/acpi/arm64/mpam.c line 122
Quoted 2 lines
drivers/acpi/arm64/mpam.c new file +2
-0+for (i =0 ; i < tbl_msc->num_resource_nodes; i++) { + err =acpi_mpam_parse_resource (msc, &resources[i]);
Isn't the length of each resource node variable? According to [2], the length depends on the num_functional_deps field. It looks like the functional dependency descriptors (if any) are appended contiguously to the resource node, unless I've misunderstood something.
Quoted 9 lines
drivers/acpi/arm64/mpam.c new file +9
-0+if (err) +return err; + } + +return 0 ; +} + +static bool __init parse_msc_pm_link(struct acpi_mpam_msc_node *tbl_msc, + struct platform_device *pdev,
drivers/acpi/arm64/mpam.c line 141
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+ u32 *acpi_id) +{ +bool acpi_id_valid = false; +struct acpi_device *buddy; +char hid[16 ], uid[16 ]; +int err; + +memset (&hid,0 ,sizeof (hid)); +memcpy (hid, &tbl_msc->hardware_id_linked_device, +sizeof (tbl_msc->hardware_id_linked_device));
This is safe by semi-accident, since 16 > 8.
It might be cleaner to declare
char hid[sizeof(tbl_msc->hardware_id_linked_device)];
which can never be wrong.
memset()+memcpy() might be better replaced with strscpy() (or just use snprintf again, since this avoids having to think about multiple different ways of avoiding buffer overflows at the same time. This is not a fast path.)
drivers/acpi/arm64/mpam.c line 148
Quoted 7 lines
drivers/acpi/arm64/mpam.c new file +7
-0+ +if (!strcmp (hid,ACPI_PROCESSOR_CONTAINER_HID )) { + *acpi_id = tbl_msc->instance_id_linked_device; + acpi_id_valid = true; + } + + err = snprintf(uid, sizeof(uid), "%u",
char uid[11]; would be sufficient, here. The instance ID is strictly 32-bit. Adding a safety margin is worthless here, since snprintf() checks the bounds -- either the size is sufficient for all possible u32 values, or it isn't.
drivers/acpi/arm64/mpam.c line 149
Quoted 1 line
drivers/acpi/arm64/mpam.c new file +1
-0+ tbl_msc->instance_id_linked_device);
Can snprintf() return < 0 on error?
I don't know, but elsewhere you do check for this. I tend to the view that is cleaner to assume that the kernel's snprintf() is just as hostile as C's version (if not more so).
(-1 >= sizeof(foo) is always true thanks to the C arithmetic conversion rules, but it's probably best not to rely on it.)
drivers/acpi/arm64/mpam.c line 151
Quoted 2 lines
drivers/acpi/arm64/mpam.c new file +2
-0+if (err >=sizeof (uid)) +return acpi_id_valid;
Possibly return true on error? Why?
Quoted 16 lines
drivers/acpi/arm64/mpam.c new file +16
-0+ + buddy =acpi_dev_get_first_match_dev (hid, uid,-1 ); +if (buddy) +device_link_add (&pdev->dev, &buddy->dev,DL_FLAG_STATELESS ); + +return acpi_id_valid; +} + +static int decode_interface_type (struct acpi_mpam_msc_node *tbl_msc, +enum mpam_msc_iface *iface) +{ +switch (tbl_msc->interface_type) { +case 0 : + *iface =MPAM_IFACE_MMIO ; +return 0 ; +case 0xa :
drivers/acpi/arm64/mpam.c line 177
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+ *iface =MPAM_IFACE_PCC ; +return 0 ; +default : +return -EINVAL ; + } +} + +static int __init acpi_mpam_parse (void ) +{ + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);
checkpatch.pl says:
| ERROR: code indent should use tabs where possible | #240: FILE: drivers/acpi/arm64/mpam.c:177: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$ | | WARNING: please, no spaces at the start of a line | #240: FILE: drivers/acpi/arm64/mpam.c:177: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$
Quoted 22 lines
drivers/acpi/arm64/mpam.c new file +22
-0+char *table_end, *table_offset = (char *)(table +1 ); +struct property_entry props[4 ];/* needs a sentinel */ +struct acpi_mpam_msc_node *tbl_msc; +int next_res, next_prop, err =0 ; +struct acpi_device *companion; +struct platform_device *pdev; +enum mpam_msc_iface iface; +struct resource res[3 ]; +char uid[16 ]; +u32 acpi_id; + +if (acpi_disabled || !system_supports_mpam () ||IS_ERR (table)) +return 0 ; + +if (IS_ERR (table)) +return 0 ; + +if (table->revision <1 ) +return 0 ; + + table_end = (char *)table + table->length; +
drivers/acpi/arm64/mpam.c line 209
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+while (table_offset < table_end) { + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + table_offset += tbl_msc->length; + +/* +* If any of the reserved fields are set, make no attempt to +* parse the msc structure. This will prevent the driver from +* probing all the MSC, meaning it can't discover the system +* wide supported partid and pmg ranges. This avoids whatever +* this MSC is truncating the partids and creating a screaming
Mangled sentence?
I have not so far found any reference in [2] to the reset value of the MPAMF_ECR.INTEN bit. Do we rely on the error interrupt(s) for all MSCs to be disabled at the interrupt controller? If the same interrupt may be shared by multiple MSCs, that's bad.
drivers/acpi/arm64/mpam.c line 213
Quoted 4 lines
drivers/acpi/arm64/mpam.c new file +4
-0+* error interrupt. +*/ +if (tbl_msc->reserved || tbl_msc->reserved1 || tbl_msc->reserved2) +continue ;
The specs are not clear about how backwards compatibility is supposed to work.
I would feel a bit uneasy about silently throwing away MSCs based on critera that may not indicate incompatibility, and without even a diagnostic.
drivers/acpi/arm64/mpam.c line 219
Quoted 6 lines
drivers/acpi/arm64/mpam.c new file +6
-0+ +if (!tbl_msc->mmio_size) +continue ; + +if (decode_interface_type (tbl_msc, &iface)) +continue ;
Ditto regarding diagnostics.
drivers/acpi/arm64/mpam.c line 226
Quoted 7 lines
drivers/acpi/arm64/mpam.c new file +7
-0+ + next_res =0 ; + next_prop =0 ; +memset (res,0 ,sizeof (res)); +memset (props,0 ,sizeof (props)); + + pdev =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);
If the tbl_msc->identifier values contain duplicates, we will get a platform device with a duplicate name here. I don't know whether it matters.
drivers/acpi/arm64/mpam.c line 235
Quoted 9 lines
drivers/acpi/arm64/mpam.c new file +9
-0+if (!pdev) { + err = -ENOMEM ; +break ; + } + +if (tbl_msc->length <sizeof (*tbl_msc)) { + err = -EINVAL ; +break ; + }
No check for oversized tbl_msc->length? (See also acpi_mpam_count_msc().)
drivers/acpi/arm64/mpam.c line 240
Quoted 5 lines
drivers/acpi/arm64/mpam.c new file +5
-0+ +/* Some power management is described in the namespace: */ + err =snprintf (uid,sizeof (uid),"%u" , tbl_msc->identifier); +if (err >0 && err <sizeof (uid)) { + companion =acpi_dev_get_first_match_dev ("ARMHAA5C" , uid,-1 );
Diagnostic?
Quoted 46 lines
drivers/acpi/arm64/mpam.c new file +46
-0+if (companion) +ACPI_COMPANION_SET (&pdev->dev, companion); + } + +if (iface ==MPAM_IFACE_MMIO ) { + res[next_res++] =DEFINE_RES_MEM_NAMED (tbl_msc->base_address, + tbl_msc->mmio_size, +"MPAM:MSC" ); + }else if (iface ==MPAM_IFACE_PCC ) { + props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , + tbl_msc->base_address); + next_prop++; + } + +acpi_mpam_parse_irqs (pdev, tbl_msc, res, &next_res); + err =platform_device_add_resources (pdev, res, next_res); +if (err) +break ; + + props[next_prop++] =PROPERTY_ENTRY_U32 ("arm,not-ready-us" , + tbl_msc->max_nrdy_usec); + +/* +* The MSC's CPU affinity is described via its linked power +* management device, but only if it points at a Processor or +* Processor Container. +*/ +if (parse_msc_pm_link (tbl_msc, pdev, &acpi_id)) { + props[next_prop++] =PROPERTY_ENTRY_U32 ("cpu_affinity" , + acpi_id); + } + + err =device_create_managed_software_node (&pdev->dev, props, +NULL ); +if (err) +break ; + +/* Come back later if you want the RIS too */ + err =platform_device_add_data (pdev, tbl_msc, tbl_msc->length); +if (err) +break ; + + err =platform_device_add (pdev); +if (err) +break ; + }
drivers/acpi/arm64/mpam.c line 296
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+ +if (err) +platform_device_put (pdev); + +return err; +} + +int acpi_mpam_count_msc (void ) +{ + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);
checkpatch.pl says:
| ERROR: code indent should use tabs where possible | #359: FILE: drivers/acpi/arm64/mpam.c:296: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$ | | WARNING: please, no spaces at the start of a line | #359: FILE: drivers/acpi/arm64/mpam.c:296: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$
drivers/acpi/arm64/mpam.c line 307
Quoted 11 lines
drivers/acpi/arm64/mpam.c new file +11
-0+char *table_end, *table_offset = (char *)(table +1 ); +struct acpi_mpam_msc_node *tbl_msc; +int count =0 ; + +if (IS_ERR (table)) +return 0 ; + +if (table->revision <1 ) +return 0 ; + + tbl_msc = (struct acpi_mpam_msc_node *)table_offset;
Can this be moved into the loop? It looks like it just duplicates the update to tbl_msc at the end of the loop; the loop termination condition does not depend on this variable.
drivers/acpi/arm64/mpam.c line 216
Quoted 5 lines
drivers/acpi/arm64/mpam.c new file +5
-0+ table_end = (char *)table + table->length; + +while (table_offset < table_end) { +if (!tbl_msc->mmio_size) +continue ;
This is 0 for non-usable PCC-based MSCs, right?
(Why explicitly unusable MSCs are listed in the table at all is a mystery to me, but that's what the spec says. I guess there must be a reason.)
drivers/acpi/arm64/mpam.c line 315
Quoted 3 lines
drivers/acpi/arm64/mpam.c new file +3
-0+ +if (tbl_msc->length <sizeof (*tbl_msc)) +return -EINVAL ;
Should we also have something like (not tested):
if (tbl_msc->length > table_end - table_offset)
return -EINVAL;
Also, is it an error if a length is not a multiple of four bytes?
(I'm guessing that the core ACPI code doesn't try to understand the contents of the MPAM table and so doesn't check this.)
Quoted 35 lines
drivers/acpi/arm64/mpam.c new file +16
-0+ + count++; + + table_offset += tbl_msc->length; + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + } + +return count; +} + +/* +* Call after ACPI devices have been created, which happens behind acpi_scan_init() +* called from subsys_initcall(). PCC requires the mailbox driver, which is +* initialised from postcore_initcall(). +*/ +subsys_initcall_sync (acpi_mpam_parse);drivers/acpi/tables.c +1
-1@@ -408,7 +408,7 @@ static const char table_sigs[][ACPI_NAMESEG_SIZE] __nonstring_array __initconstACPI_SIG_PSDT ,ACPI_SIG_RSDT ,ACPI_SIG_XSDT ,ACPI_SIG_SSDT ,ACPI_SIG_IORT ,ACPI_SIG_NFIT ,ACPI_SIG_HMAT ,ACPI_SIG_PPTT ,ACPI_SIG_NHLT ,ACPI_SIG_AEST ,ACPI_SIG_CEDT ,ACPI_SIG_AGDI , - ACPI_SIG_NBFT }; + ACPI_SIG_NBFT, ACPI_SIG_MPAM };#define ACPI_HEADER_SIZE sizeof(struct acpi_table_header)include/linux/arm_mpam.h new file +0
-0
include/linux/arm_mpam.h line 2
Quoted 2 lines
include/linux/arm_mpam.h new file +2
-0@@ -0,0 +1,46 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2025 Arm Ltd. */
checkpatch.pl says:
| WARNING: Improper SPDX comment style for 'include/linux/arm_mpam.h', please use '/*' instead | #414: FILE: include/linux/arm_mpam.h:1: | +// SPDX-License-Identifier: GPL-2.0 | | WARNING: Missing or malformed SPDX-License-Identifier tag in line 1 | #414: FILE: include/linux/arm_mpam.h:1: | +// SPDX-License-Identifier: GPL-2.0
(That's probably the same error twice.)
(I never understood why the SPDX folks couldn't have allowed either type of comment -- or at least, the same style in .c and .h files. But I'm sure they had a reason that they believed was good.)
[...]
Cheers
---Dave
[1] ACPI for Memory System Resource Partitioning and Monitoring, 3.0 beta
https://developer.arm.com/documentation/den0065/3-0bet/?lang=en
[2] Arm Memory System Resource Partitioning and Monitoring (MPAM) System Component Specification, ARM IHI 0099A.a
James Morse
maintainerRe: [PATCH 08/33] ACPI / MPAM: Parse the MPAM table
Hi Dave,
Quoted 6 lines · Dave Martin
Quoted 2 lines · James Morse
Add code to parse the arm64 specific MPAM table, looking up the cache level from the PPTT and feeding the end result into the MPAM driver.
Might be worth mentioning that the hook for feeding the parsed factoids into the driver (
mpam_ris_create()) is not implemented for now.
Sure,
Quoted 7 lines
Quoted 6 lines
drivers/acpi/arm64/mpam.c new file +0
-0@@ -0,0 +1,331 @@
Quoted 10 lines
drivers/acpi/arm64/mpam.c line 28
Quoted 8 lines
drivers/acpi/arm64/mpam.c new file +8
-0+#define ACPI_MPAM_MSC_IRQ_MODE_MASK BIT(0) +#define ACPI_MPAM_MSC_IRQ_TYPE_MASK GENMASK(2, 1) +#define ACPI_MPAM_MSC_IRQ_TYPE_WIRED 0 +#define ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER BIT(3) +#define ACPI_MPAM_MSC_IRQ_AFFINITY_VALID BIT(4) + +static bool frob_irq (struct platform_device *pdev,int intid,u32 flags, +int *irq,u32 processor_container_uid)Can this have a name, please?
parse_irq()? (but not all of it - only the bits that are duplicated)
How about parse_irq_common().
Quoted 5 lines
Quoted 5 lines
drivers/acpi/arm64/mpam.c new file +5
-0+{ +int sense; + +if (!intid) +return false;
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 39
Quoted 6 lines
drivers/acpi/arm64/mpam.c new file +6
-0+ +if (FIELD_GET (ACPI_MPAM_MSC_IRQ_TYPE_MASK , flags) != +ACPI_MPAM_MSC_IRQ_TYPE_WIRED ) +return false; + + sense =FIELD_GET (ACPI_MPAM_MSC_IRQ_MODE_MASK , flags);Do we handle cross-endian ACPI tables?
ACPI depends on UEFI, which is little endian only.
Quoted 8 lines
ACPI defers to the relevant specification regarding the endianness of externally defined tables, but as of v3.0 (beta) of for MPAM ACPI spec [1], no statement is made about this.
Following the spirit of the ACPI core specs, I suspect that the "correct" answer is that MPAM tables are always little-endian, even if it not written down anywhere.
Quoted 7 lines
If the kernel is big-endian, we lose.
Maybe it is sufficient to make
CONFIG_ACPI_MPAMdepend on !CONFIG_CPU_BIG_ENDIAN for now.I haven't tried to understand how this is handled for other tables.
There is no way to hand the tables to the OS. I don't think this needs explict handling, it all just falls out in the wash.
Quoted 6 lines
drivers/acpi/arm64/mpam.c line 43
Quoted 4 lines
drivers/acpi/arm64/mpam.c new file +4
-0+ +/* +* If the GSI is in the GIC's PPI range, try and create a partitioned +* percpu interrupt. But actually we don't even try? Or did I miss something?
Ah - I ripped that out on the assumption no-one was using it. I'll drop the comment. No-one has squealed yet!
It's works out the box on DT systems, and can be described in the MPAM ACPI table - but without a platform, there is no need for the irqchip juggling to make it work. Hence the pr_err_once().
Quoted 11 lines
drivers/acpi/arm64/mpam.c line 25
Quoted 2 lines
drivers/acpi/arm64/mpam.c new file +2
-0+*/ +if (16 <= intid && intid <32 && processor_container_uid != ~0 ) {checkpatch.pl says:
| WARNING: Comparisons should place the constant on the right side of the test | #108: FILE: drivers/acpi/arm64/mpam.c:45: | + if (16 <= intid && intid < 32 && processor_container_uid != ~0) {(Dubious whether this is "wrong" IMHO, but still probably best avoided since it is not what people are used to seeing.)
If it were a single comparison I'd agree, but as its two, I find this a lot easier to read as 'intid is between 16 and 32' than the alternative.
Quoted 13 lines
drivers/acpi/arm64/mpam.c line 53
Quoted 8 lines
drivers/acpi/arm64/mpam.c new file +8
-0+pr_err_once ("Partitioned interrupts not supported\n" ); +return false; + } + + *irq =acpi_register_gsi (&pdev->dev, intid, sense,ACPI_ACTIVE_HIGH ); +if (*irq <=0 ) { +pr_err_once ("Failed to register interrupt 0x%x with ACPI\n" , + intid);Are we going to get a lot of duplicate error messages with the same interrupt? If not, perhaps make this a
pr_err()so that all the affected interrupts are notified?
Quoted 2 lines
(Either way, hopefully the user will take the hint that they messed something up though.)
The hardware people figured they could get the OS to distribute the configuration to all the nodes in the mesh, so there are platforms with hundreds of MSC. They also like to share the interrupts, so if you mess the interrupt description up - you get hundreds of messages.
To avoid spamming the log for what is likely to be a frequently repeated mistake, these are all 'once'.
Quoted 7 lines
Quoted 7 lines
drivers/acpi/arm64/mpam.c new file +7
-0+return false; + } + +return true; +} + +static void acpi_mpam_parse_irqs(struct platform_device *pdev,
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 65
Quoted 5 lines
drivers/acpi/arm64/mpam.c new file +5
-0+struct acpi_mpam_msc_node *tbl_msc, + struct resource *res, int *res_idx) +{ + u32 flags, aff; +int irq;We may still get in here if MPAMF_IDR.HAS_ERR_MSI and/or MPAMF_MSMON_IDR.HAS_OFLW_MSI is set.
(this code can't know that)
Quoted 3 lines
If so, there is no wired
interrupt. Does it matter if we still parse and allocate the wired interrupts here?
I don't think we do - frob_irq() will return false if the 'wired' flag is not set in the table, so the 'res[(*res_idx)++]' step will be skipped, and the callers array is left unmodified. The struct property_entry array is null-terminated, and gets copied anyway.
Quoted 10 lines
drivers/acpi/arm64/mpam.c line 72
Quoted 6 lines
drivers/acpi/arm64/mpam.c new file +6
-0+ flags = tbl_msc->overflow_interrupt_flags; +if (flags &ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER ) + aff = tbl_msc->overflow_interrupt_affinity; +else + aff = ~0 ;(u32)~0 is used as an exceptional UID all over the place. If this is not a pre-existing convention, it could be worth having a #define for this. (grep '~0'
drivers/acpi/suggests that this is new.)
It's not normal to describe the affinity of an interrupt in tables like this... The MPAM ACPI spec allows you to describe the PPI affinity because the MSC can be local to a CPU, (e.g. the L2 cache), and the MPAM architecture spec says it can be a PPI. We have support for this on DT systems, so it wasn't possible to argue "no one would ever do that"!
Quoted 5 lines
drivers/acpi/arm64/mpam.c line 74
Quoted 2 lines
drivers/acpi/arm64/mpam.c new file +2
-0+if (frob_irq (pdev, tbl_msc->overflow_interrupt, flags, &irq, aff)) + res[(*res_idx)++] =DEFINE_RES_IRQ_NAMED (irq,"overflow" );I couldn't find a statement in the spec of how the table can specify that there is no interrupt.
With zero in the GSI field, from the spec:
| If this MSC does not support overflow interrupts or monitors, this field must be set | to 0.
Which is detected in frob_irq() as:
| if (!intid)
| return false;
Quoted 1 line
Are the interrupts always required for ACPI-based MPAM systems?
In practice they're optional.
Quoted 2 lines
overflow_interruptanderror_interruptare GSIVs, which seems to be an ACPI thing.
It's a flat space of interrupt IDs - no need to shift the values around for the SGI/PPI/SPI range. (aka IPI, percpu interrupt or plain old wired interrupt)
Quoted 1 line
The examples in the ACPI spec suggest that 0 can be a valid value.
Pretty sure its not - it would be a secure SGI. The MPAM spec uses it as 'invalid'.
Quoted 4 lines
No exceptional value seems to be defined.
The flags fields have some invalid encodings, but no explicit "no interrupt" encoding that I can see.
I think you missed it hidden as 'and another thing' in the description of the field.
Quoted 50 lines
Quoted 36 lines
drivers/acpi/arm64/mpam.c new file +36
-0+ + flags = tbl_msc->error_interrupt_flags; +if (flags &ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER ) + aff = tbl_msc->error_interrupt_affinity; +else + aff = ~0 ; +if (frob_irq (pdev, tbl_msc->error_interrupt, flags, &irq, aff)) + res[(*res_idx)++] =DEFINE_RES_IRQ_NAMED (irq,"error" ); +} + +static int acpi_mpam_parse_resource (struct mpam_msc *msc, +struct acpi_mpam_resource_node *res) +{ +int level, nid; +u32 cache_id; + +switch (res->locator_type) { +case ACPI_MPAM_LOCATION_TYPE_PROCESSOR_CACHE : + cache_id = res->locator.cache_locator.cache_reference; + level =find_acpi_cache_level_from_id (cache_id); +if (level <=0 ) { +pr_err_once ("Bad level (%u) for cache with id %u\n" , level, cache_id); +return -EINVAL ; + } +return mpam_ris_create (msc, res->ris_index,MPAM_CLASS_CACHE , + level, cache_id); +case ACPI_MPAM_LOCATION_TYPE_MEMORY : + nid =pxm_to_node (res->locator.memory_locator.proximity_domain); +if (nid ==NUMA_NO_NODE ) + nid =0 ; +return mpam_ris_create (msc, res->ris_index,MPAM_CLASS_MEMORY , +255 , nid); +default : +/* These get discovered later and treated as unknown */ +return 0 ;drivers/acpi/arm64/mpam.c line 120
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+ } +} + +int acpi_mpam_parse_resources (struct mpam_msc *msc, +struct acpi_mpam_msc_node *tbl_msc) +{ +int i, err; +struct acpi_mpam_resource_node *resources; + + resources = (struct acpi_mpam_resource_node *)(tbl_msc +1 );Should we check that we don't go out of the bounds of the MSC node (or, at the very least, of the MPAM table)?
Quoted 2 lines
If
tbl_msc->lengthwas already validated, that can be used for the bounds check.
I'm not a fan of trying to validate the APCI tables - its extra work for something that just has to be right. e.g. we can't check the base-address, we just have to trust its correct.
I didn't want to pass the table in here, and grabbing another reference means I'd have to work out if its the same mapping...
But as tbl_msc->length is to hand...
Quoted 7 lines
drivers/acpi/arm64/mpam.c line 122
Quoted 2 lines
drivers/acpi/arm64/mpam.c new file +2
-0+for (i =0 ; i < tbl_msc->num_resource_nodes; i++) { + err =acpi_mpam_parse_resource (msc, &resources[i]);Isn't the length of each resource node variable? According to [2], the length depends on the
num_functional_depsfield. It looks like the functional dependency descriptors (if any) are appended contiguously to the resource node, unless I've misunderstood something.
No - it proably is that broken. They forgot to include a length field, painting themselves into a corner if they ever want to modify this!
I was ignoring the 'functional dependencies' until someone comes up with a platform that needs it. But yes, this assumes that field is zero.
I'm not entirely clear what the MPAM driver is supposed to do with functional
dependencies: 'make up a configuration' ... but when the controls formats are different, its going to get exciting.
I'ved fixed that up:
| int acpi_mpam_parse_resources(struct mpam_msc *msc,
| struct acpi_mpam_msc_node *tbl_msc)
| {
| int i, err;
| char *ptr, *last_byte;
| struct acpi_mpam_resource_node *resource;
|
| ptr = (char *)(tbl_msc + 1);
| last_byte = ptr + tbl_msc->length;
| for (i = 0; i < tbl_msc->num_resource_nodes; i++) {
| if (ptr + sizeof(*resource) > last_byte)
| return -EINVAL;
|
| resource = (struct acpi_mpam_resource_node *)ptr;
| err = acpi_mpam_parse_resource(msc, resource);
| if (err)
| return err;
|
| ptr += sizeof(*resource);
| ptr += resource->num_functional_deps * sizeof(struct acpi_mpam_func_deps);
| }
|
| return 0;
| }Quoted 22 lines
Quoted 9 lines
drivers/acpi/arm64/mpam.c new file +9
-0+if (err) +return err; + } + +return 0 ; +} + +static bool __init parse_msc_pm_link(struct acpi_mpam_msc_node *tbl_msc, + struct platform_device *pdev,drivers/acpi/arm64/mpam.c line 141
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+ u32 *acpi_id) +{ +bool acpi_id_valid = false; +struct acpi_device *buddy; +char hid[16 ], uid[16 ]; +int err; + +memset (&hid,0 ,sizeof (hid)); +memcpy (hid, &tbl_msc->hardware_id_linked_device, +sizeof (tbl_msc->hardware_id_linked_device));This is safe by semi-accident, since 16 > 8.
Quoted 5 lines
It might be cleaner to declare
char hid[sizeof(tbl_msc->hardware_id_linked_device)];which can never be wrong.
But can be too precise!
Quoted 4 lines
memset()+memcpy() might be better replaced with
strscpy()(or just use snprintf again, since this avoids having to think about multiple different ways of avoiding buffer overflows at the same time. This is not a fast path.)
snprintf() demands space for a NULL byte - but these APCI tables fields are fixed width, so don't have them.
The array sizes picked are the next larger power of 2, and the memset is ensure there is a NULL byte after the fixed size region is copied in.
Where snprintf() is being used - its to convert from integer to string. For the hid, what we really want is a fixed size memcpy() ... its possible to cook up a format string that will do that, but its going to be an eye sore.
I can change this to;
| char hid[sizeof(tbl_msc->hardware_id_linked_device) + 1];
But I don't think this is better... How about a comment?!
| /* | * tbl_msc->hardware_id_linked_device is an 8 byte fixed width string. | * hid[] is the next larger power of 2, and is zero'd to give us a | * null terminated string for acpi_dev_get_first_match_dev(). | */
This is all because the MPAM stuff was forced into a static table, even though it needs to refer to stuff in the namespace.
Quoted 4 lines
Quoted 4 lines
drivers/acpi/arm64/mpam.c new file +4
-0+ +if (!strcmp (hid,ACPI_PROCESSOR_CONTAINER_HID )) { + *acpi_id = tbl_msc->instance_id_linked_device; + acpi_id_valid = true;
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 148
Quoted 3 lines
drivers/acpi/arm64/mpam.c new file +3
-0+ } + + err = snprintf(uid, sizeof(uid), "%u",char uid[11]; would be sufficient, here. The instance ID is strictly 32-bit. Adding a safety margin is worthless here, since
snprintf()checks the bounds -- either the size is sufficient for all possible u32 values, or it isn't.
I just plumped for the next largest power of 2.
Quoted 3 lines
drivers/acpi/arm64/mpam.c line 149
Quoted 1 line
drivers/acpi/arm64/mpam.c new file +1
-0+ tbl_msc->instance_id_linked_device);Can
snprintf()return < 0 on error?
Documented as returning a positive number.
Quoted 11 lines
I don't know, but elsewhere you do check for this. I tend to the view that is cleaner to assume that the kernel's
snprintf()is just as hostile as C's version (if not more so).(-1 >= sizeof(foo) is always true thanks to the C arithmetic conversion rules, but it's probably best not to rely on it.)
drivers/acpi/arm64/mpam.c line 130
Quoted 2 lines
drivers/acpi/arm64/mpam.c new file +2
-0+if (err >=sizeof (uid)) +return acpi_id_valid;Possibly return true on error? Why?
(this is an error case that will never happen - but error handling is important!)
The acpi_id was parsed out of the table and returned to the caller. Hence true is returned. This gets used to find the CPU-affinity, which is how the driver avoids accessing caches that might not be on. There are sanity checks around this value. It can't be present for some caches and not others.
Due to this error, the link to the buddy device was not added - which is silently ignored. If it matters, it will show up as an access to a device that didn't get turned on by the power-management code, because of that missing link.
Because I don't think this can happpen, I didn't try to handle it explicitly - just carry on like nothing is wrong.
Quoted 28 lines
Quoted 15 lines
drivers/acpi/arm64/mpam.c new file +15
-0+ buddy =acpi_dev_get_first_match_dev (hid, uid,-1 ); +if (buddy) +device_link_add (&pdev->dev, &buddy->dev,DL_FLAG_STATELESS ); + +return acpi_id_valid; +} + +static int decode_interface_type (struct acpi_mpam_msc_node *tbl_msc, +enum mpam_msc_iface *iface) +{ +switch (tbl_msc->interface_type) { +case 0 : + *iface =MPAM_IFACE_MMIO ; +return 0 ; +case 0xa :drivers/acpi/arm64/mpam.c line 177
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+ *iface =MPAM_IFACE_PCC ; +return 0 ; +default : +return -EINVAL ; + } +} + +static int __init acpi_mpam_parse (void ) +{ + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);checkpatch.pl says:
Quoted 7 lines
| ERROR: code indent should use tabs where possible | #240: FILE: drivers/acpi/arm64/mpam.c:177: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$ | | WARNING: please, no spaces at the start of a line | #240: FILE: drivers/acpi/arm64/mpam.c:177: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$
Pff, I copied that from Jonathan's email because of its rune like nature.
Quoted 26 lines
Quoted 26 lines
drivers/acpi/arm64/mpam.c new file +26
-0+char *table_end, *table_offset = (char *)(table +1 ); +struct property_entry props[4 ];/* needs a sentinel */ +struct acpi_mpam_msc_node *tbl_msc; +int next_res, next_prop, err =0 ; +struct acpi_device *companion; +struct platform_device *pdev; +enum mpam_msc_iface iface; +struct resource res[3 ]; +char uid[16 ]; +u32 acpi_id; + +if (acpi_disabled || !system_supports_mpam () ||IS_ERR (table)) +return 0 ; + +if (IS_ERR (table)) +return 0 ; + +if (table->revision <1 ) +return 0 ; + + table_end = (char *)table + table->length; + +while (table_offset < table_end) { + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + table_offset += tbl_msc->length; +
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 209
Quoted 6 lines
drivers/acpi/arm64/mpam.c new file +6
-0+/* +* If any of the reserved fields are set, make no attempt to +* parse the msc structure. This will prevent the driver from +* probing all the MSC, meaning it can't discover the system +* wide supported partid and pmg ranges. This avoids whatever +* this MSC is truncating the partids and creating a screaming Mangled sentence?
just hard to parse:
"whatever this MSC is" "truncating the parids..."
Fixed as
| This avoids this MSC truncating the partids and creating a screaming error interrupt.
(although its not strictly the MSC that does that)
Quoted 4 lines
I have not so far found any reference in [2] to the reset value of the MPAMF_ECR.INTEN bit. Do we rely on the error interrupt(s) for all MSCs to be disabled at the interrupt controller? If the same interrupt may be shared by multiple MSCs, that's bad.
They are anticipated as shared because the ESR register lets us know if this MSC triggered the error.
The interrupt won't be delivered until the driver requests it - it'll just stay pending.
Yes - if some firwmare component used an out-of-range PARTID then linux will believe that this was caused by linux and disable MPAM. Not much we can do about that.
(I've even seen a platform where this happens!)
Quoted 7 lines
drivers/acpi/arm64/mpam.c line 213
Quoted 4 lines
drivers/acpi/arm64/mpam.c new file +4
-0+* error interrupt. +*/ +if (tbl_msc->reserved || tbl_msc->reserved1 || tbl_msc->reserved2) +continue ;The specs are not clear about how backwards compatibility is supposed to work.
How about that!
The spec people thought this would allow them to add things that aren't MSC in, re-using one of these reserved fields as a type.
But - the architecture spec mandates the OS run around the MSC and collect the minimum PARTID/PMG values (so people don't have to build an SoC where everything fits together nicely). Because of that, the OS needs to know how many MSC there are, so that it knows it has probed them all.
Quoted 3 lines
I would feel a bit uneasy about silently throwing away MSCs based on critera that may not indicate incompatibility, and without even a diagnostic.
Yes the MSC gets thrown away - but it was still counted by acpi_mpam_count_msc(). The MPAM driver will never probe all the MSC, so it will never call mpam_enable().
(you used to be able to see which cpuhp callback was registeered, to know it was stuck in discovery, but now a helper does that which always uses the same name. I may fix that)
I think its fine for MPAM to silently fail to probe like this. The system still works, all the known hardware gets probed and none of the unknown hardware is touched. (so it can't blow us up). You don't get resctrl - but we can't know if the unknown hardware was going to be important to supporting MPAM. e.g. it lowers the minimum usable PARTID/PMG.
It's between a rock and a hard place - I think this behaviour is the best we can do.
Regarding it being silent - whatever happens it needs a kernel upgrade to learn about the newly described hardware. I don't think its worth printing anything out in this case.
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 219
Quoted 6 lines
drivers/acpi/arm64/mpam.c new file +6
-0+ +if (!tbl_msc->mmio_size) +continue ; + +if (decode_interface_type (tbl_msc, &iface)) +continue ;Ditto regarding diagnostics.
A third interface type will result in those MSC being untouched, and MPAM unavailable. I don't think printing "maybe upgrade your kernel?" is going to help anyone.
I'll chuck some pr_debug() in here so someone could find out which of these it is.
Quoted 11 lines
drivers/acpi/arm64/mpam.c line 226
Quoted 7 lines
drivers/acpi/arm64/mpam.c new file +7
-0+ + next_res =0 ; + next_prop =0 ; +memset (res,0 ,sizeof (res)); +memset (props,0 ,sizeof (props)); + + pdev =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);If the
tbl_msc->identifiervalues contain duplicates, we will get a platform device with a duplicate name here. I don't know whether it matters.
The identifiers must be unique. I don't think this is the sort of table validation we need to do for the firmware vendor.
Quoted 11 lines
drivers/acpi/arm64/mpam.c line 235
Quoted 9 lines
drivers/acpi/arm64/mpam.c new file +9
-0+if (!pdev) { + err = -ENOMEM ; +break ; + } + +if (tbl_msc->length <sizeof (*tbl_msc)) { + err = -EINVAL ; +break ; + }No check for oversized
tbl_msc->length?
You want it to parse the whole thing to check if firmware got the length field right? I don't think we can know that. If its wrong, firmware may just have easily got the number of RIS, or some other field wrong.
Historically x86 using to rummage around in memory to see if it could find something that looked like ACPI tables. I'd imagine it was important to parse those carefully, as they may never have been APCI table. We don't have this problem on arm64 - the ACPI tables were handed over from UEFI. If the value is wrong - the firmware vendor meant it to be wrong.
Quoted 69 lines
(See also
acpi_mpam_count_msc().)drivers/acpi/arm64/mpam.c line 240
Quoted 5 lines
drivers/acpi/arm64/mpam.c new file +5
-0+ +/* Some power management is described in the namespace: */ + err =snprintf (uid,sizeof (uid),"%u" , tbl_msc->identifier); +if (err >0 && err <sizeof (uid)) { + companion =acpi_dev_get_first_match_dev ("ARMHAA5C" , uid,-1 );Diagnostic?
Quoted 46 lines
drivers/acpi/arm64/mpam.c new file +46
-0+if (companion) +ACPI_COMPANION_SET (&pdev->dev, companion); + } + +if (iface ==MPAM_IFACE_MMIO ) { + res[next_res++] =DEFINE_RES_MEM_NAMED (tbl_msc->base_address, + tbl_msc->mmio_size, +"MPAM:MSC" ); + }else if (iface ==MPAM_IFACE_PCC ) { + props[next_prop++] =PROPERTY_ENTRY_U32 ("pcc-channel" , + tbl_msc->base_address); + next_prop++; + } + +acpi_mpam_parse_irqs (pdev, tbl_msc, res, &next_res); + err =platform_device_add_resources (pdev, res, next_res); +if (err) +break ; + + props[next_prop++] =PROPERTY_ENTRY_U32 ("arm,not-ready-us" , + tbl_msc->max_nrdy_usec); + +/* +* The MSC's CPU affinity is described via its linked power +* management device, but only if it points at a Processor or +* Processor Container. +*/ +if (parse_msc_pm_link (tbl_msc, pdev, &acpi_id)) { + props[next_prop++] =PROPERTY_ENTRY_U32 ("cpu_affinity" , + acpi_id); + } + + err =device_create_managed_software_node (&pdev->dev, props, +NULL ); +if (err) +break ; + +/* Come back later if you want the RIS too */ + err =platform_device_add_data (pdev, tbl_msc, tbl_msc->length); +if (err) +break ; + + err =platform_device_add (pdev); +if (err) +break ; + }drivers/acpi/arm64/mpam.c line 296
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+ +if (err) +platform_device_put (pdev); + +return err; +} + +int acpi_mpam_count_msc (void ) +{ + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);checkpatch.pl says:
Quoted 8 lines
| ERROR: code indent should use tabs where possible | #359: FILE: drivers/acpi/arm64/mpam.c:296: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$ | | WARNING: please, no spaces at the start of a line | #359: FILE: drivers/acpi/arm64/mpam.c:296: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$
Yup, runes copied from Jonathan's email.
Quoted 7 lines
Quoted 7 lines
drivers/acpi/arm64/mpam.c new file +7
-0+char *table_end, *table_offset = (char *)(table +1 ); +struct acpi_mpam_msc_node *tbl_msc; +int count =0 ; + +if (IS_ERR (table)) +return 0 ; +
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 201
Quoted 4 lines
drivers/acpi/arm64/mpam.c new file +4
-0+if (table->revision <1 ) +return 0 ; + + tbl_msc = (struct acpi_mpam_msc_node *)table_offset;Can this be moved into the loop? It looks like it just duplicates the update to
tbl_mscat the end of the loop; the loop termination condition does not depend on this variable.
Yup - looks like this used to be a do {} while () ; loop.
Quoted 11 lines
drivers/acpi/arm64/mpam.c line 216
Quoted 5 lines
drivers/acpi/arm64/mpam.c new file +5
-0+ table_end = (char *)table + table->length; + +while (table_offset < table_end) { +if (!tbl_msc->mmio_size) +continue ;This is 0 for non-usable PCC-based MSCs, right?
(Why explicitly unusable MSCs are listed in the table at all is a mystery to me, but that's what the spec says. I guess there must be a reason.)
Some firmwware stacks have pre-built ACPI tables, then go knock out the enabled bits based on what the platform actually has. This is easier than rebuilding the table...
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 315
Quoted 3 lines
drivers/acpi/arm64/mpam.c new file +3
-0+ +if (tbl_msc->length <sizeof (*tbl_msc)) +return -EINVAL ;Should we also have something like (not tested):
if (
tbl_msc->length>table_end-table_offset)return -EINVAL;
Sure,
Quoted 1 line
Also, is it an error if a length is not a multiple of four bytes?
Hmmm, because all the subtables in the sepc are multiples of four bytes? I think this falls in the bucket of 'table validation the OS shouldn't have to do'. If the ACPI tables are that wrong, we're going to have bigger problems.
Quoted 2 lines
(I'm guessing that the core ACPI code doesn't try to understand the contents of the MPAM table and so doesn't check this.)
It doesn't.
Quoted 29 lines
Quoted 16 lines
drivers/acpi/arm64/mpam.c new file +9
-0+ + count++; + + table_offset += tbl_msc->length; + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + } + +return count; +}include/linux/arm_mpam.h new file +0
-0include/linux/arm_mpam.h line 2
Quoted 2 lines
include/linux/arm_mpam.h new file +2
-0@@ -0,0 +1,46 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2025 Arm Ltd. */ checkpatch.pl says:
| WARNING: Improper SPDX comment style for 'include/linux/arm_mpam.h', please use '/*' instead | #414: FILE: include/linux/arm_mpam.h:1: | +// SPDX-License-Identifier: GPL-2.0 | | WARNING: Missing or malformed SPDX-License-Identifier tag in line 1 | #414: FILE: include/linux/arm_mpam.h:1: | +// SPDX-License-Identifier: GPL-2.0
Quoted 5 lines
(That's probably the same error twice.)
(I never understood why the SPDX folks couldn't have allowed either type of comment -- or at least, the same style in .c and .h files. But I'm sure they had a reason that they believed was good.)
Fixed,
Thanks,
James
Re: [PATCH 08/33] ACPI / MPAM: Parse the MPAM table
Hi James,
Quoted 13 lines · James Morse
Hi Dave,
Quoted 6 lines · Dave Martin
Quoted 2 lines · James Morse
Add code to parse the arm64 specific MPAM table, looking up the cache level from the PPTT and feeding the end result into the MPAM driver.
Might be worth mentioning that the hook for feeding the parsed factoids into the driver (
mpam_ris_create()) is not implemented for now.Sure,
Quoted 1 line
Quoted 1 line
drivers/acpi/arm64/mpam.c +0
-0
[...]
Quoted 7 lines
Quoted 4 lines
drivers/acpi/arm64/mpam.c line 28
Quoted 2 lines
drivers/acpi/arm64/mpam.c +2
-0+static bool frob_irq (struct platform_device *pdev,int intid,u32 flags, +int *irq,u32 processor_container_uid)Can this have a name, please?
parse_irq()? (but not all of it - only the bits that are duplicated) How aboutparse_irq_common().
That is inoffensive, but since this is where the interrupt is validated as usable and an attempt is made to reqister it -- and this is basically all the function seems to do right now, would
acpi_mpam_register_irq()
or similar make sense, here?
Quoted 27 lines
Quoted 5 lines
Quoted 5 lines
drivers/acpi/arm64/mpam.c +5
-0+{ +int sense; + +if (!intid) +return false;Quoted 8 lines
drivers/acpi/arm64/mpam.c line 39
Quoted 6 lines
drivers/acpi/arm64/mpam.c +6
-0+ +if (FIELD_GET (ACPI_MPAM_MSC_IRQ_TYPE_MASK , flags) != +ACPI_MPAM_MSC_IRQ_TYPE_WIRED ) +return false; + + sense =FIELD_GET (ACPI_MPAM_MSC_IRQ_MODE_MASK , flags);Do we handle cross-endian ACPI tables?
ACPI depends on UEFI, which is little endian only.
Quoted 10 lines
ACPI defers to the relevant specification regarding the endianness of externally defined tables, but as of v3.0 (beta) of for MPAM ACPI spec [1], no statement is made about this.
Following the spirit of the ACPI core specs, I suspect that the "correct" answer is that MPAM tables are always little-endian, even if it not written down anywhere.
If the kernel is big-endian, we lose.
Quoted 8 lines
Quoted 5 lines
Maybe it is sufficient to make
CONFIG_ACPI_MPAMdepend on !CONFIG_CPU_BIG_ENDIAN for now.I haven't tried to understand how this is handled for other tables.
There is no way to hand the tables to the OS. I don't think this needs explict handling, it all just falls out in the wash.
(I'm not sure what you mean by not being able to hand tables to the OS. How did Linux get them?)
But in any case, if the BE case is not handled by the Linux ACPI code in general then it almost certainly doesn't make sense to handle it here.
(There are a lot of BE-handling macros in the imported ACPICA headers in the kernel tree, which made me wonder whether this was a thing.)
Quoted 7 lines
Quoted 6 lines
drivers/acpi/arm64/mpam.c line 43
Quoted 4 lines
drivers/acpi/arm64/mpam.c +4
-0+ +/* +* If the GSI is in the GIC's PPI range, try and create a partitioned +* percpu interrupt. But actually we don't even try? Or did I miss something?
Quoted 6 lines
Ah - I ripped that out on the assumption no-one was using it. I'll drop the comment. No-one has squealed yet!
It's works out the box on DT systems, and can be described in the MPAM ACPI table - but without a platform, there is no need for the irqchip juggling to make it work. Hence the
pr_err_once().
OK. I've no strong feeling about that.
Quoted 9 lines
Quoted 9 lines
drivers/acpi/arm64/mpam.c
Quoted 2 lines
drivers/acpi/arm64/mpam.c +2
-0+*/ +if (16 <= intid && intid <32 && processor_container_uid != ~0 ) {checkpatch.pl says:
| WARNING: Comparisons should place the constant on the right side of the test | #108: FILE: drivers/acpi/arm64/mpam.c:45: | + if (16 <= intid && intid < 32 && processor_container_uid != ~0) {
Quoted 5 lines
Quoted 2 lines
(Dubious whether this is "wrong" IMHO, but still probably best avoided since it is not what people are used to seeing.)
If it were a single comparison I'd agree, but as its two, I find this a lot easier to read as 'intid is between 16 and 32' than the alternative.
Sure, just flagging it.
(There is a slight temptation with this idiom to accidentally write e.g., 16 <= intid < 32, which means something very different. But although the (correct) idiom appears uncommon in the kernel, there is some precedent. Either way, it works.)
Quoted 16 lines
Quoted 13 lines
drivers/acpi/arm64/mpam.c line 53
Quoted 8 lines
drivers/acpi/arm64/mpam.c +8
-0+pr_err_once ("Partitioned interrupts not supported\n" ); +return false; + } + + *irq =acpi_register_gsi (&pdev->dev, intid, sense,ACPI_ACTIVE_HIGH ); +if (*irq <=0 ) { +pr_err_once ("Failed to register interrupt 0x%x with ACPI\n" , + intid);Are we going to get a lot of duplicate error messages with the same interrupt? If not, perhaps make this a
pr_err()so that all the affected interrupts are notified?Quoted 2 lines
(Either way, hopefully the user will take the hint that they messed something up though.)
Quoted 6 lines
The hardware people figured they could get the OS to distribute the configuration to all the nodes in the mesh, so there are platforms with hundreds of MSC. They also like to share the interrupts, so if you mess the interrupt description up - you get hundreds of messages.
To avoid spamming the log for what is likely to be a frequently repeated mistake, these are all 'once'.
OK, fair enough.
1 error message is better than none, either way.
Quoted 40 lines
Quoted 16 lines
drivers/acpi/arm64/mpam.c line 65
Quoted 13 lines
drivers/acpi/arm64/mpam.c +12
-0+return false; + } + +return true; +} + +static void acpi_mpam_parse_irqs (struct platform_device *pdev, +struct acpi_mpam_msc_node *tbl_msc, +struct resource *res,int *res_idx) +{ +u32 flags, aff; +int irq;We may still get in here if MPAMF_IDR.HAS_ERR_MSI and/or MPAMF_MSMON_IDR.HAS_OFLW_MSI is set.
(this code can't know that)
Quoted 3 lines
If so, there is no wired
interrupt. Does it matter if we still parse and allocate the wired interrupts here?I don't think we do -
frob_irq()will return false if the 'wired' flag is not set in the table, so the 'res[(*res_idx)++]' step will be skipped, and the callers array is left unmodified. The structproperty_entryarray is null-terminated, and gets copied anyway.Quoted 10 lines
drivers/acpi/arm64/mpam.c line 72
Quoted 6 lines
drivers/acpi/arm64/mpam.c +6
-0+ flags = tbl_msc->overflow_interrupt_flags; +if (flags &ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER ) + aff = tbl_msc->overflow_interrupt_affinity; +else + aff = ~0 ;(u32)~0 is used as an exceptional UID all over the place. If this is not a pre-existing convention, it could be worth having a #define for this. (grep '~0'
drivers/acpi/suggests that this is new.)
Quoted 5 lines
It's not normal to describe the affinity of an interrupt in tables like this... The MPAM ACPI spec allows you to describe the PPI affinity because the MSC can be local to a CPU, (e.g. the L2 cache), and the MPAM architecture spec says it can be a PPI. We have support for this on DT systems, so it wasn't possible to argue "no one would ever do that"!
I guess this is OK.
(My misgivings about ~0 are partly due to the way C evaluates expressions. The conversion to the destination type occus only after the ~ is evaulated, so if the affected type is changed to a 64-bit type during maintenance (or by copy-pasting into another context), then it's easy to end up with with wrong values in the high bits. The definition
u64 x = ~0;
does indeed set x to all ones, but I find the reason _why_ this works counterintuitive, and superficially similar expressions can go wrong. Having a #define only requires this to be got right in one place.)
Quoted 27 lines
Quoted 5 lines
drivers/acpi/arm64/mpam.c line 74
Quoted 2 lines
drivers/acpi/arm64/mpam.c +2
-0+if (frob_irq (pdev, tbl_msc->overflow_interrupt, flags, &irq, aff)) + res[(*res_idx)++] =DEFINE_RES_IRQ_NAMED (irq,"overflow" );I couldn't find a statement in the spec of how the table can specify that there is no interrupt.
With zero in the GSI field, from the spec:
| If this MSC does not support overflow interrupts or monitors, this field must be set | to 0.Which is detected in
frob_irq()as:
| if (!intid)
| return false;Quoted 1 line
Are the interrupts always required for ACPI-based MPAM systems?
In practice they're optional.
Quoted 2 lines
overflow_interruptanderror_interruptare GSIVs, which seems to be an ACPI thing.It's a flat space of interrupt IDs - no need to shift the values around for the SGI/PPI/SPI range. (aka IPI, percpu interrupt or plain old wired interrupt)
Quoted 7 lines
Quoted 5 lines
The examples in the ACPI spec suggest that 0 can be a valid value. No exceptional value seems to be defined.
The flags fields have some invalid encodings, but no explicit "no interrupt" encoding that I can see.
I think you missed it hidden as 'and another thing' in the description of the field.
I was being diplomatic. I meant: "explicitly uses this as an example of a valid value."
Quoted 1 line
Pretty sure its not - it would be a secure SGI. The MPAM spec uses it as 'invalid'.
I guess that's is a Arm-ism, then. So long as this is standard for ACPI on Arm systems, then I guess there is no problem -- and anyway, the ACPI MPAM spec would take precedence for interpreting this field.
Now I look more closely, you're right: the ACPI MPAM spec says, e.g.:
"If the MSC supports MSI, as indicated by the MPAMF_MSMON_IDR.HAS_OFLW_MSI bit, then this field must be set to 0 and ignored by the OS. If this MSC does not support overflow interrupts or monitors, this field must be set to 0".
This does not say that the value 0 means there is no wired interrupt, but that seems to be the intention.
It could be better worded, but the intention does seem to be that 0 means "no (wired) interrupt", here...
Quoted 9 lines
Quoted 9 lines
drivers/acpi/arm64/mpam.c line 83
Quoted 9 lines
drivers/acpi/arm64/mpam.c +9
-0+ + flags = tbl_msc->error_interrupt_flags; +if (flags &ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER ) + aff = tbl_msc->error_interrupt_affinity; +else + aff = ~0 ; +if (frob_irq (pdev, tbl_msc->error_interrupt, flags, &irq, aff)) + res[(*res_idx)++] =DEFINE_RES_IRQ_NAMED (irq,"error" );
intid 0 gets ignored by frob_irq(), so I guess we are OK in the MSI case (the spec says we must ignore it, but also says that it must be zero).
It would be onerous to have to map the MSC and examine its ID regs here assuming that the intid really is 0 in the MSI case seems reasonable.
Quoted 44 lines
Quoted 41 lines
Quoted 27 lines
drivers/acpi/arm64/mpam.c +27
-0+} + +static int acpi_mpam_parse_resource (struct mpam_msc *msc, +struct acpi_mpam_resource_node *res) +{ +int level, nid; +u32 cache_id; + +switch (res->locator_type) { +case ACPI_MPAM_LOCATION_TYPE_PROCESSOR_CACHE : + cache_id = res->locator.cache_locator.cache_reference; + level =find_acpi_cache_level_from_id (cache_id); +if (level <=0 ) { +pr_err_once ("Bad level (%u) for cache with id %u\n" , level, cache_id); +return -EINVAL ; + } +return mpam_ris_create (msc, res->ris_index,MPAM_CLASS_CACHE , + level, cache_id); +case ACPI_MPAM_LOCATION_TYPE_MEMORY : + nid =pxm_to_node (res->locator.memory_locator.proximity_domain); +if (nid ==NUMA_NO_NODE ) + nid =0 ; +return mpam_ris_create (msc, res->ris_index,MPAM_CLASS_MEMORY , +255 , nid); +default : +/* These get discovered later and treated as unknown */ +return 0 ;drivers/acpi/arm64/mpam.c line 120
Quoted 10 lines
drivers/acpi/arm64/mpam.c +10
-0+ } +} + +int acpi_mpam_parse_resources (struct mpam_msc *msc, +struct acpi_mpam_msc_node *tbl_msc) +{ +int i, err; +struct acpi_mpam_resource_node *resources; + + resources = (struct acpi_mpam_resource_node *)(tbl_msc +1 );Should we check that we don't go out of the bounds of the MSC node (or, at the very least, of the MPAM table)?
Quoted 2 lines
If
tbl_msc->lengthwas already validated, that can be used for the bounds check.
Quoted 7 lines
I'm not a fan of trying to validate the APCI tables - its extra work for something that just has to be right. e.g. we can't check the base-address, we just have to trust its correct.
I didn't want to pass the table in here, and grabbing another reference means I'd have to work out if its the same mapping...
But as
tbl_msc->lengthis to hand...
That gets my vote, except that it would be trivial to validate length in length in acpi_mpam_parse(). Checking against length without first validating length woule be completely pointless.
I agree that there are things that we can't validate and things that it is redundant or pointless to check. But it feels a bit lazy not to care whether the parser wanders out of the table while trying to parse it.
If the rest of the ACPI code takes a similarly relaxed attitude (I've not checked) then I guess it is pointless to be more careful here.
But otherwise, I would prefer to see all the bounds checks done. This is not hard, and there are not many things to check. (In my experience, it requires far fewer brain cycles to just make these things correct by construction than to try to figure out the situations in which the corner-cutting might or might not matter.)
Quoted 10 lines
Quoted 7 lines
drivers/acpi/arm64/mpam.c line 122
Quoted 2 lines
drivers/acpi/arm64/mpam.c +2
-0+for (i =0 ; i < tbl_msc->num_resource_nodes; i++) { + err =acpi_mpam_parse_resource (msc, &resources[i]);Isn't the length of each resource node variable? According to [2], the length depends on the
num_functional_depsfield. It looks like the functional dependency descriptors (if any) are appended contiguously to the resource node, unless I've misunderstood something.No - it proably is that broken. They forgot to include a length field, painting themselves into a corner if they ever want to modify this!
The spec doesn't look particularly broken -- i.e., it can be compatibly extended, albeit in a more awkward way that would be ideal. The main impact is that the list of resources has to be parsed sequentially, but that's all code is ever likely to do with the table anyway.
Quoted 8 lines
I was ignoring the 'functional dependencies' until someone comes up with a platform that needs it. But yes, this assumes that field is zero.
I'm not entirely clear what the MPAM driver is supposed to do with functional
dependencies: 'make up a configuration' ... but when the controls formats are different, its going to get exciting.I'ved fixed that up:
Quoted 9 lines
| int acpi_mpam_parse_resources(struct mpam_msc *msc,
| struct acpi_mpam_msc_node *tbl_msc)
| {
| int i, err;
| char *ptr, *last_byte;
| struct acpi_mpam_resource_node *resource;
|
| ptr = (char *)(tbl_msc + 1);
| last_byte = ptr + tbl_msc->length;Nit: misnomer. This is not the address of the last byte.
(I tend to call similar pointers something like "limit".)
Quoted 11 lines
| for (i = 0; i < tbl_msc->num_resource_nodes; i++) {
| if (ptr + sizeof(*resource) > last_byte)
| return -EINVAL;
|
| resource = (struct acpi_mpam_resource_node *)ptr;
| err = acpi_mpam_parse_resource(msc, resource);
| if (err)
| return err;
|
| ptr += sizeof(*resource);
| ptr += resource->num_functional_deps * sizeof(struct acpi_mpam_func_deps);This mostly looks sane.
But unless ptr is guaranteed to be less than 0xffffffff00000002 (?), then the bounds check at the start of the loop may bogusly pass.
Would a check like this before advancing ptr across the functional deps be reasonable? (untested)
if (resource->num_functional_deps > (last_byte - ptr) / sizeof(struct acpi_mpam_func_deps) return -EINVAL;
(Why the table allows more functional dependencies to be declared than could possibly fit in an MSC node is a mystery.)
[...]
Quoted 22 lines
Quoted 22 lines
Quoted 9 lines
drivers/acpi/arm64/mpam.c +9
-0+if (err) +return err; + } + +return 0 ; +} + +static bool __init parse_msc_pm_link(struct acpi_mpam_msc_node *tbl_msc, + struct platform_device *pdev,drivers/acpi/arm64/mpam.c line 141
Quoted 10 lines
drivers/acpi/arm64/mpam.c +10
-0+ u32 *acpi_id) +{ +bool acpi_id_valid = false; +struct acpi_device *buddy; +char hid[16 ], uid[16 ]; +int err; + +memset (&hid,0 ,sizeof (hid)); +memcpy (hid, &tbl_msc->hardware_id_linked_device, +sizeof (tbl_msc->hardware_id_linked_device));This is safe by semi-accident, since 16 > 8.
Quoted 7 lines
Quoted 5 lines
It might be cleaner to declare
char hid[sizeof(tbl_msc->hardware_id_linked_device)];which can never be wrong.
But can be too precise!
But only when the correct length was unknown. So you are saying that this can only be correct when it is not known to be correct? ;)
Sometimes an oversized array has useful defensive value against future mis-maintenance, but since the ACPI fields are all fixed-size, I don't see any benefit here.
Quoted 8 lines
Quoted 4 lines
memset()+memcpy() might be better replaced with
strscpy()(or just use snprintf again, since this avoids having to think about multiple different ways of avoiding buffer overflows at the same time. This is not a fast path.)
snprintf()demands space for a NULL byte - but these APCI tables fields are fixed width, so don't have them.
Quoted 6 lines
The array sizes picked are the next larger power of 2, and the memset is ensure there is a NULL byte after the fixed size region is copied in.
Where
snprintf()is being used - its to convert from integer to string. For the hid, what we really want is a fixed sizememcpy()... its possible to cook up a format string that will do that, but its going to be an eye sore.
Well, it's a matter of style.
Quoted 3 lines
I can change this to;
| char hid[sizeof(tbl_msc->hardware_id_linked_device) + 1];
Can do (and this would probably have shut me up), but:
Quoted 7 lines
But I don't think this is better... How about a comment?!
| /* | * tbl_msc->hardware_id_linked_device is an 8 byte fixed width string. | * hid[] is the next larger power of 2, and is zero'd to give us a | * null terminated string for acpi_dev_get_first_match_dev(). | */
this doesn't explain why the size needs to be a power of two, or what the wasted bytes are for.
(Either way, this is obviously no big deal...)
Quoted 19 lines
This is all because the MPAM stuff was forced into a static table, even though it needs to refer to stuff in the namespace.
Quoted 4 lines
Quoted 4 lines
drivers/acpi/arm64/mpam.c +4
-0+ +if (!strcmp (hid,ACPI_PROCESSOR_CONTAINER_HID )) { + *acpi_id = tbl_msc->instance_id_linked_device; + acpi_id_valid = true;Quoted 8 lines
drivers/acpi/arm64/mpam.c line 148
Quoted 3 lines
drivers/acpi/arm64/mpam.c +3
-0+ } + + err = snprintf(uid, sizeof(uid), "%u",char uid[11]; would be sufficient, here. The instance ID is strictly 32-bit. Adding a safety margin is worthless here, since
snprintf()checks the bounds -- either the size is sufficient for all possible u32 values, or it isn't.
Quoted 1 line
I just plumped for the next largest power of 2.
This is harmless, but since the only way to know that this is sufficient is to know what the correct value would have been, it seems pointless (as above) to lie to the compiler about how much space can be used.
Quoted 7 lines
Quoted 5 lines
drivers/acpi/arm64/mpam.c line 149
Quoted 3 lines
drivers/acpi/arm64/mpam.c +1
-0+ tbl_msc->instance_id_linked_device);Can
snprintf()return < 0 on error?Documented as returning a positive number.
Not in C23, and never standard for the printf() family in general. But it does appear that this is true for the kernel's snprintf(), and this assmuption is widely relied upon in the kernel. So, a check here would be pointless compilexity after all.
(The only documented error condition in C23 seems the case of a multibyte encoding error, which is not relevant here although it does not rule out other error conditions -- the wording is characteristically ambiguous.)
[...]
Quoted 12 lines
Quoted 4 lines
drivers/acpi/arm64/mpam.c
Quoted 2 lines
drivers/acpi/arm64/mpam.c +2
-0+if (err >=sizeof (uid)) +return acpi_id_valid;Possibly return true on error? Why?
(this is an error case that will never happen - but error handling is important!)
The
acpi_idwas parsed out of the table and returned to the caller. Hence true is returned. This gets used to find the CPU-affinity, which is how the driver avoids accessing caches that might not be on. There are sanity checks around this value. It can't be present for some caches and not others.
Quoted 6 lines
Due to this error, the link to the buddy device was not added - which is silently ignored. If it matters, it will show up as an access to a device that didn't get turned on by the power-management code, because of that missing link.
Because I don't think this can happpen, I didn't try to handle it explicitly - just carry on like nothing is wrong.
What I meant was, if the MPAM table contains garbage, why not just flag this and give up? That feels semantically simpler than trying to limp on, with possibly undiagnosed invalid results.
[...]
Quoted 15 lines
Quoted 6 lines
drivers/acpi/arm64/mpam.c line 177
Quoted 3 lines
drivers/acpi/arm64/mpam.c +3
-0+static int __init acpi_mpam_parse (void ) +{ + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);checkpatch.pl says:
Quoted 7 lines
| ERROR: code indent should use tabs where possible | #240: FILE: drivers/acpi/arm64/mpam.c:177: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$ | | WARNING: please, no spaces at the start of a line | #240: FILE: drivers/acpi/arm64/mpam.c:177: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$Pff, I copied that from Jonathan's email because of its rune like nature.
[...]
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
drivers/acpi/arm64/mpam.c +4
-0+while (table_offset < table_end) { + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + table_offset += tbl_msc->length; +Quoted 8 lines
drivers/acpi/arm64/mpam.c line 209
Quoted 6 lines
drivers/acpi/arm64/mpam.c +6
-0+/* +* If any of the reserved fields are set, make no attempt to +* parse the msc structure. This will prevent the driver from +* probing all the MSC, meaning it can't discover the system +* wide supported partid and pmg ranges. This avoids whatever +* this MSC is truncating the partids and creating a screaming Mangled sentence?
Quoted 7 lines
just hard to parse:
"whatever this MSC is" "truncating the parids..."Fixed as
| This avoids this MSC truncating the partids and creating a screaming error interrupt.(although its not strictly the MSC that does that)
The critical thing here seems to be that we prevent the driver from ever being enabled. The comment doesn't seem to say that we are actually disabling the MPAM driver by giving up here (it only explains why).
For the benefit of people who are less familiar with the code, Would something like this be better: [*]
--8<--
If the reserved fields are set then the meaning of the rest of the entry is unknown, so leave the MSC marked as unprobed and give up. This means that the MPAM driver will never be enabled. There is no way to enable it safely, because we cannot determine safe system-wide partid and pmg ranges in this situation.
-->8--
Quoted 10 lines
Quoted 6 lines
I have not so far found any reference in [2] to the reset value of the MPAMF_ECR.INTEN bit. Do we rely on the error interrupt(s) for all MSCs to be disabled at the interrupt controller? If the same interrupt may be shared by multiple MSCs, that's bad.
They are anticipated as shared because the ESR register lets us know if this MSC triggered the error.
Quoted 6 lines
The interrupt won't be delivered until the driver requests it - it'll just stay pending.
Yes - if some firwmare component used an out-of-range PARTID then linux will believe that this was caused by linux and disable MPAM. Not much we can do about that.
(I've even seen a platform where this happens!)
Ah, right -- I'd missed the ESR.
This is inherently racy even without interrupt sharing, so I guess sharing the interrupt doesn't make things worse (except for having to poll multiple ESRs on interrupt -- but if that hardware vendor made us do that, it's on them.)
Quoted 8 lines
Quoted 7 lines
drivers/acpi/arm64/mpam.c line 213
Quoted 4 lines
drivers/acpi/arm64/mpam.c +4
-0+* error interrupt. +*/ +if (tbl_msc->reserved || tbl_msc->reserved1 || tbl_msc->reserved2) +continue ;The specs are not clear about how backwards compatibility is supposed to work.
Quoted 8 lines
How about that!
The spec people thought this would allow them to add things that aren't MSC in, re-using one of these reserved fields as a type.
But - the architecture spec mandates the OS run around the MSC and collect the minimum PARTID/PMG values (so people don't have to build an SoC where everything fits together nicely). Because of that, the OS needs to know how many MSC there are, so that it knows it has probed them all.
See my suggestion at [*], above.
The reader doesn't need to understand about future architecture directions, here. The only thing that matters is that the kernel does not make any assumptions about the meaning of the MSC entry if these fields are set. I'm not sure that we need to justify this position -- simply stating it is probably enough.
Quoted 16 lines
Quoted 3 lines
I would feel a bit uneasy about silently throwing away MSCs based on critera that may not indicate incompatibility, and without even a diagnostic.
Yes the MSC gets thrown away - but it was still counted by
acpi_mpam_count_msc(). The MPAM driver will never probe all the MSC, so it will never callmpam_enable().(you used to be able to see which cpuhp callback was registeered, to know it was stuck in discovery, but now a helper does that which always uses the same name. I may fix that)
I think its fine for MPAM to silently fail to probe like this. The system still works, all the known hardware gets probed and none of the unknown hardware is touched. (so it can't blow us up). You don't get resctrl - but we can't know if the unknown hardware was going to be important to supporting MPAM. e.g. it lowers the minimum usable PARTID/PMG.
Quoted 5 lines
It's between a rock and a hard place - I think this behaviour is the best we can do.
Regarding it being silent - whatever happens it needs a kernel upgrade to learn about the newly described hardware. I don't think its worth printing anything out in this case.
I would still vote for a basic
"Unrecognised MSC, MPAM not usable"
message or similar. This costs us nothing, and at least gives a clue that missing kernel support or a corrupt ACPI table are likely causes for resctrl not showing up.
It partly depends on how likely we think this scenario os -- or how likely it is that people will insist on using an old kernel with a botched stack of backports on new hardware. I don't have a strong sense on this.
Quoted 9 lines
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 219
Quoted 6 lines
drivers/acpi/arm64/mpam.c +6
-0+ +if (!tbl_msc->mmio_size) +continue ; + +if (decode_interface_type (tbl_msc, &iface)) +continue ;Ditto regarding diagnostics.
Quoted 4 lines
A third interface type will result in those MSC being untouched, and MPAM unavailable. I don't think printing "maybe upgrade your kernel?" is going to help anyone. I'll chuck some
pr_debug()in here so someone could find out which of these it is.
pr_debug() for sub-diagnosing the error may be helpful, but these feel like an addition to the basic "MPAM unusable" error rather than as a replacement for it...
Quoted 12 lines
Quoted 11 lines
drivers/acpi/arm64/mpam.c line 226
Quoted 7 lines
drivers/acpi/arm64/mpam.c +7
-0+ + next_res =0 ; + next_prop =0 ; +memset (res,0 ,sizeof (res)); +memset (props,0 ,sizeof (props)); + + pdev =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);If the
tbl_msc->identifiervalues contain duplicates, we will get a platform device with a duplicate name here. I don't know whether it matters.
Quoted 2 lines
The identifiers must be unique. I don't think this is the sort of table validation we need to do for the firmware vendor.
My question was really: "what goes wrong?" I suppose that if platform_device_alloc() doesn't like this then
Quoted 6 lines
Quoted 6 lines
drivers/acpi/arm64/mpam.c line 230
Quoted 6 lines
drivers/acpi/arm64/mpam.c +4
-0+if (!pdev) { + err = -ENOMEM ; +break ; + }
we call the caller we ran out of memory. While not exactly helpful, this is better than blowing up. If this is the only consequence of duplicate IDs here (and this anyway Should Not Happen), then I guess this is adequate.
If platform_device_alloc() doesn't care about duplicate names, then I guess we can just trust it to have done something sensible.
Quoted 11 lines
Quoted 7 lines
drivers/acpi/arm64/mpam.c line 235
Quoted 5 lines
drivers/acpi/arm64/mpam.c +5
-0+ +if (tbl_msc->length <sizeof (*tbl_msc)) { + err = -EINVAL ; +break ; + }No check for oversized
tbl_msc->length?You want it to parse the whole thing to check if firmware got the length field right? I don't think we can know that. If its wrong, firmware may just have easily got the number of RIS, or some other field wrong.
We can't know that the table is "wrong" (since it is authoritative), but we can know whether it so internally inconsistent that it is not parseable.
Quoted 4 lines
Historically x86 using to rummage around in memory to see if it could find something that looked like ACPI tables. I'd imagine it was important to parse those carefully, as they may never have been APCI table. We don't have this problem on arm64 - the ACPI tables were handed over from UEFI. If the value is wrong - the firmware vendor meant it to be wrong.
See above for my rationale.
This is more about maintainability and cleanliness than trying to defend ourselves against broken firmware.
Note: if the ACPI code in general doesn't bother to do these kind of checks then we can probably follow the established culture. I'm not so familiar with this codebase.
[...]
Quoted 8 lines
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 240
Quoted 5 lines
drivers/acpi/arm64/mpam.c +5
-0+ +/* Some power management is described in the namespace: */ + err =snprintf (uid,sizeof (uid),"%u" , tbl_msc->identifier); +if (err >0 && err <sizeof (uid)) { + companion =acpi_dev_get_first_match_dev ("ARMHAA5C" , uid,-1 );Diagnostic?
Ping
[...]
Quoted 15 lines
Quoted 6 lines
drivers/acpi/arm64/mpam.c line 296
Quoted 3 lines
drivers/acpi/arm64/mpam.c +3
-0+int acpi_mpam_count_msc (void ) +{ + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);checkpatch.pl says:
Quoted 8 lines
| ERROR: code indent should use tabs where possible | #359: FILE: drivers/acpi/arm64/mpam.c:296: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$ | | WARNING: please, no spaces at the start of a line | #359: FILE: drivers/acpi/arm64/mpam.c:296: | + struct acpi_table_header *table __free(acpi_table) = acpi_get_table_ret(ACPI_SIG_MPAM, 0);$
Quoted 1 line
Yup, runes copied from Jonathan's email.
Ack
Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
drivers/acpi/arm64/mpam.c +7
-0+char *table_end, *table_offset = (char *)(table +1 ); +struct acpi_mpam_msc_node *tbl_msc; +int count =0 ; + +if (IS_ERR (table)) +return 0 ; +Quoted 8 lines
drivers/acpi/arm64/mpam.c line 201
Quoted 4 lines
drivers/acpi/arm64/mpam.c +4
-0+if (table->revision <1 ) +return 0 ; + + tbl_msc = (struct acpi_mpam_msc_node *)table_offset;Can this be moved into the loop? It looks like it just duplicates the update to
tbl_mscat the end of the loop; the loop termination condition does not depend on this variable.
Quoted 1 line
Yup - looks like this used to be a do {} while () ; loop.
OK
Quoted 11 lines
Quoted 11 lines
drivers/acpi/arm64/mpam.c line 216
Quoted 5 lines
drivers/acpi/arm64/mpam.c +5
-0+ table_end = (char *)table + table->length; + +while (table_offset < table_end) { +if (!tbl_msc->mmio_size) +continue ;This is 0 for non-usable PCC-based MSCs, right?
(Why explicitly unusable MSCs are listed in the table at all is a mystery to me, but that's what the spec says. I guess there must be a reason.)
(I guess PCC was added after the size field was named. I suppose we don't really need a comment for that -- it's fairly clear why this bit of code is here. We could rename the field in our struct, but this would probably just move confusion around rather than solving it.)
Quoted 2 lines
Some firmwware stacks have pre-built ACPI tables, then go knock out the enabled bits based on what the platform actually has. This is easier than rebuilding the table...
Right, that makes sense.
Quoted 9 lines
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 315
Quoted 3 lines
drivers/acpi/arm64/mpam.c +3
-0+ +if (tbl_msc->length <sizeof (*tbl_msc)) +return -EINVAL ;Should we also have something like (not tested):
if (
tbl_msc->length>table_end-table_offset)return -EINVAL;
Quoted 8 lines
Sure,
Quoted 1 line
Also, is it an error if a length is not a multiple of four bytes?
Hmmm, because all the subtables in the sepc are multiples of four bytes? I think this falls in the bucket of 'table validation the OS shouldn't have to do'. If the ACPI tables are that wrong, we're going to have bigger problems.
So long as we don't fault on unaligned access in the kernel (which I'm pretty sure we don't do), then nothing goes wrong in the unaligned case -- so, fine. Otherwise we'll get a clean Oops, which would at least be a decent place to start debugging.
Quoted 4 lines
Quoted 2 lines
(I'm guessing that the core ACPI code doesn't try to understand the contents of the MPAM table and so doesn't check this.)
It doesn't.
Ack.
Quoted 35 lines
Quoted 29 lines
Quoted 16 lines
drivers/acpi/arm64/mpam.c +9
-0+ + count++; + + table_offset += tbl_msc->length; + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + } + +return count; +}include/linux/arm_mpam.h new file +0
-0include/linux/arm_mpam.h line 2
Quoted 2 lines
include/linux/arm_mpam.h new file +2
-0@@ -0,0 +1,46 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2025 Arm Ltd. */ checkpatch.pl says:
| WARNING: Improper SPDX comment style for 'include/linux/arm_mpam.h', please use '/*' instead | #414: FILE: include/linux/arm_mpam.h:1: | +// SPDX-License-Identifier: GPL-2.0 | | WARNING: Missing or malformed SPDX-License-Identifier tag in line 1 | #414: FILE: include/linux/arm_mpam.h:1: | +// SPDX-License-Identifier: GPL-2.0Quoted 5 lines
(That's probably the same error twice.)
(I never understood why the SPDX folks couldn't have allowed either type of comment -- or at least, the same style in .c and .h files. But I'm sure they had a reason that they believed was good.)
Quoted 6 lines
Fixed,
Thanks,
James
OK
Cheers
---Dave
James Morse
maintainerRe: [PATCH 08/33] ACPI / MPAM: Parse the MPAM table
Hi Dave,
Quoted 23 lines · Dave Martin
Quoted 11 lines · James Morse
Quoted 6 lines · Dave Martin
Quoted 2 lines · James Morse
Add code to parse the arm64 specific MPAM table, looking up the cache level from the PPTT and feeding the end result into the MPAM driver.
Might be worth mentioning that the hook for feeding the parsed factoids into the driver (
mpam_ris_create()) is not implemented for now.Sure,
Quoted 1 line
Quoted 1 line
drivers/acpi/arm64/mpam.c +0
-0[...]
Quoted 7 lines
Quoted 4 lines
drivers/acpi/arm64/mpam.c line 28
Quoted 2 lines
drivers/acpi/arm64/mpam.c +2
-0+static bool frob_irq (struct platform_device *pdev,int intid,u32 flags, +int *irq,u32 processor_container_uid)Can this have a name, please?
parse_irq()? (but not all of it - only the bits that are duplicated) How aboutparse_irq_common().
Quoted 7 lines
That is inoffensive, but since this is where the interrupt is validated as usable and an attempt is made to reqister it -- and this is basically all the function seems to do right now, would
acpi_mpam_register_irq()or similar make sense, here?
Sure,
Quoted 31 lines
Quoted 31 lines
Quoted 5 lines
Quoted 5 lines
drivers/acpi/arm64/mpam.c +5
-0+{ +int sense; + +if (!intid) +return false;Quoted 8 lines
drivers/acpi/arm64/mpam.c line 39
Quoted 6 lines
drivers/acpi/arm64/mpam.c +6
-0+ +if (FIELD_GET (ACPI_MPAM_MSC_IRQ_TYPE_MASK , flags) != +ACPI_MPAM_MSC_IRQ_TYPE_WIRED ) +return false; + + sense =FIELD_GET (ACPI_MPAM_MSC_IRQ_MODE_MASK , flags);Do we handle cross-endian ACPI tables?
ACPI depends on UEFI, which is little endian only.
Quoted 14 lines
ACPI defers to the relevant specification regarding the endianness of externally defined tables, but as of v3.0 (beta) of for MPAM ACPI spec [1], no statement is made about this.
Following the spirit of the ACPI core specs, I suspect that the "correct" answer is that MPAM tables are always little-endian, even if it not written down anywhere.
If the kernel is big-endian, we lose.
Maybe it is sufficient to make
CONFIG_ACPI_MPAMdepend on !CONFIG_CPU_BIG_ENDIAN for now.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
I haven't tried to understand how this is handled for other tables.
There is no way to hand the tables to the OS. I don't think this needs explict handling, it all just falls out in the wash.
(I'm not sure what you mean by not being able to hand tables to the OS. How did Linux get them?)
Your hypothetical cross-endian machine would have to be a big-endian kernel running with little-endian ACPI tables. UEFI is little-endian, UEFI is the only mechanism for handing ACPI tables to the kernel ... there is no mechanism for a big-endian kernel to get hold of ACPI tables.
Quoted 6 lines
But in any case, if the BE case is not handled by the Linux ACPI code in general then it almost certainly doesn't make sense to handle it here.
(There are a lot of BE-handling macros in the imported ACPICA headers in the kernel tree, which made me wonder whether this was a thing.)
Nothing defines ACPI_BIG_ENDIAN ... I don't know why acpica supports it, but linux doesn't use it.
[...]
Quoted 53 lines
Quoted 39 lines
Quoted 7 lines
Quoted 7 lines
drivers/acpi/arm64/mpam.c +7
-0+return false; + } + +return true; +} + +static void acpi_mpam_parse_irqs(struct platform_device *pdev,Quoted 8 lines
drivers/acpi/arm64/mpam.c line 65
Quoted 5 lines
drivers/acpi/arm64/mpam.c +5
-0+struct acpi_mpam_msc_node *tbl_msc, + struct resource *res, int *res_idx) +{ + u32 flags, aff; +int irq;We may still get in here if MPAMF_IDR.HAS_ERR_MSI and/or MPAMF_MSMON_IDR.HAS_OFLW_MSI is set.
(this code can't know that)
Quoted 3 lines
If so, there is no wired
interrupt. Does it matter if we still parse and allocate the wired interrupts here?I don't think we do -
frob_irq()will return false if the 'wired' flag is not set in the table, so the 'res[(*res_idx)++]' step will be skipped, and the callers array is left unmodified. The structproperty_entryarray is null-terminated, and gets copied anyway.Quoted 10 lines
drivers/acpi/arm64/mpam.c line 72
Quoted 6 lines
drivers/acpi/arm64/mpam.c +6
-0+ flags = tbl_msc->overflow_interrupt_flags; +if (flags &ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER ) + aff = tbl_msc->overflow_interrupt_affinity; +else + aff = ~0 ;(u32)~0 is used as an exceptional UID all over the place. If this is not a pre-existing convention, it could be worth having a #define for this. (grep '~0'
drivers/acpi/suggests that this is new.)Quoted 5 lines
It's not normal to describe the affinity of an interrupt in tables like this... The MPAM ACPI spec allows you to describe the PPI affinity because the MSC can be local to a CPU, (e.g. the L2 cache), and the MPAM architecture spec says it can be a PPI. We have support for this on DT systems, so it wasn't possible to argue "no one would ever do that"!
I guess this is OK.
(My misgivings about ~0 are partly due to the way C evaluates expressions. The conversion to the destination type occus only after the ~ is evaulated, so if the affected type is changed to a 64-bit type during maintenance (or by copy-pasting into another context), then it's easy to end up with with wrong values in the high bits. The definition
Quoted 5 lines
u64 x = ~0;does indeed set x to all ones, but I find the reason _why_ this works counterintuitive, and superficially similar expressions can go wrong. Having a #define only requires this to be got right in one place.)
Thanks for the background.
Quoted 30 lines
Quoted 30 lines
Quoted 5 lines
drivers/acpi/arm64/mpam.c line 74
Quoted 2 lines
drivers/acpi/arm64/mpam.c +2
-0+if (frob_irq (pdev, tbl_msc->overflow_interrupt, flags, &irq, aff)) + res[(*res_idx)++] =DEFINE_RES_IRQ_NAMED (irq,"overflow" );I couldn't find a statement in the spec of how the table can specify that there is no interrupt.
With zero in the GSI field, from the spec:
| If this MSC does not support overflow interrupts or monitors, this field must be set | to 0.Which is detected in
frob_irq()as:
| if (!intid)
| return false;Quoted 1 line
Are the interrupts always required for ACPI-based MPAM systems?
In practice they're optional.
Quoted 2 lines
overflow_interruptanderror_interruptare GSIVs, which seems to be an ACPI thing.It's a flat space of interrupt IDs - no need to shift the values around for the SGI/PPI/SPI range. (aka IPI, percpu interrupt or plain old wired interrupt)
Quoted 3 lines
The examples in the ACPI spec suggest that 0 can be a valid value. No exceptional value seems to be defined.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
The flags fields have some invalid encodings, but no explicit "no interrupt" encoding that I can see.
I think you missed it hidden as 'and another thing' in the description of the field.
I was being diplomatic. I meant: "explicitly uses this as an example of a valid value."
I can ask the spec people to clarify things - but I'm not entirely sure what the request is. An example entry in the appendix for a system with no interrupts?
Quoted 12 lines
Quoted 1 line
Pretty sure its not - it would be a secure SGI. The MPAM spec uses it as 'invalid'.
I guess that's is a Arm-ism, then. So long as this is standard for ACPI on Arm systems, then I guess there is no problem -- and anyway, the ACPI MPAM spec would take precedence for interpreting this field.
Now I look more closely, you're right: the ACPI MPAM spec says, e.g.:
"If the MSC supports MSI, as indicated by the MPAMF_MSMON_IDR.HAS_OFLW_MSI bit, then this field must be set to 0 and ignored by the OS. If this MSC does not support overflow interrupts or monitors, this field must be set to 0".
Quoted 5 lines
This does not say that the value 0 means there is no wired interrupt, but that seems to be the intention.
It could be better worded, but the intention does seem to be that 0 means "no (wired) interrupt", here...
I'll copy you on the thread to the spec people.
Quoted 14 lines
Quoted 9 lines
Quoted 9 lines
drivers/acpi/arm64/mpam.c line 83
Quoted 9 lines
drivers/acpi/arm64/mpam.c +9
-0+ + flags = tbl_msc->error_interrupt_flags; +if (flags &ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER ) + aff = tbl_msc->error_interrupt_affinity; +else + aff = ~0 ; +if (frob_irq (pdev, tbl_msc->error_interrupt, flags, &irq, aff)) + res[(*res_idx)++] =DEFINE_RES_IRQ_NAMED (irq,"error" );intid 0 gets ignored by
frob_irq(), so I guess we are OK in the MSI case (the spec says we must ignore it, but also says that it must be zero).
Quoted 3 lines
It would be onerous to have to map the MSC and examine its ID regs here assuming that the intid really is 0 in the MSI case seems reasonable.
Registering+Requesting MSI would be up to the driver to do, there is no reason for this ACPI parsing code to get involved with that.
Quoted 45 lines
Quoted 45 lines
Quoted 39 lines
Quoted 25 lines
drivers/acpi/arm64/mpam.c +25
-0+static int acpi_mpam_parse_resource (struct mpam_msc *msc, +struct acpi_mpam_resource_node *res) +{ +int level, nid; +u32 cache_id; + +switch (res->locator_type) { +case ACPI_MPAM_LOCATION_TYPE_PROCESSOR_CACHE : + cache_id = res->locator.cache_locator.cache_reference; + level =find_acpi_cache_level_from_id (cache_id); +if (level <=0 ) { +pr_err_once ("Bad level (%u) for cache with id %u\n" , level, cache_id); +return -EINVAL ; + } +return mpam_ris_create (msc, res->ris_index,MPAM_CLASS_CACHE , + level, cache_id); +case ACPI_MPAM_LOCATION_TYPE_MEMORY : + nid =pxm_to_node (res->locator.memory_locator.proximity_domain); +if (nid ==NUMA_NO_NODE ) + nid =0 ; +return mpam_ris_create (msc, res->ris_index,MPAM_CLASS_MEMORY , +255 , nid); +default : +/* These get discovered later and treated as unknown */ +return 0 ;drivers/acpi/arm64/mpam.c line 120
Quoted 10 lines
drivers/acpi/arm64/mpam.c +10
-0+ } +} + +int acpi_mpam_parse_resources (struct mpam_msc *msc, +struct acpi_mpam_msc_node *tbl_msc) +{ +int i, err; +struct acpi_mpam_resource_node *resources; + + resources = (struct acpi_mpam_resource_node *)(tbl_msc +1 );Should we check that we don't go out of the bounds of the MSC node (or, at the very least, of the MPAM table)?
Quoted 2 lines
If
tbl_msc->lengthwas already validated, that can be used for the bounds check.I'm not a fan of trying to validate the APCI tables - its extra work for something that just has to be right. e.g. we can't check the base-address, we just have to trust its correct.
Quoted 8 lines
Quoted 4 lines
I didn't want to pass the table in here, and grabbing another reference means I'd have to work out if its the same mapping...
But as
tbl_msc->lengthis to hand...That gets my vote, except that it would be trivial to validate length in length in
acpi_mpam_parse(). Checking against length without first validating length woule be completely pointless.
I've added something to do that...
Quoted 48 lines
I agree that there are things that we can't validate and things that it is redundant or pointless to check. But it feels a bit lazy not to care whether the parser wanders out of the table while trying to parse it.
If the rest of the ACPI code takes a similarly relaxed attitude (I've not checked) then I guess it is pointless to be more careful here.
But otherwise, I would prefer to see all the bounds checks done. This is not hard, and there are not many things to check. (In my experience, it requires far fewer brain cycles to just make these things correct by construction than to try to figure out the situations in which the corner-cutting might or might not matter.)
Quoted 10 lines
Quoted 7 lines
drivers/acpi/arm64/mpam.c line 122
Quoted 2 lines
drivers/acpi/arm64/mpam.c +2
-0+for (i =0 ; i < tbl_msc->num_resource_nodes; i++) { + err =acpi_mpam_parse_resource (msc, &resources[i]);Isn't the length of each resource node variable? According to [2], the length depends on the
num_functional_depsfield. It looks like the functional dependency descriptors (if any) are appended contiguously to the resource node, unless I've misunderstood something.No - it proably is that broken. They forgot to include a length field, painting themselves into a corner if they ever want to modify this!
The spec doesn't look particularly broken -- i.e., it can be compatibly extended, albeit in a more awkward way that would be ideal. The main impact is that the list of resources has to be parsed sequentially, but that's all code is ever likely to do with the table anyway.
Quoted 8 lines
I was ignoring the 'functional dependencies' until someone comes up with a platform that needs it. But yes, this assumes that field is zero.
I'm not entirely clear what the MPAM driver is supposed to do with functional
dependencies: 'make up a configuration' ... but when the controls formats are different, its going to get exciting.I'ved fixed that up:
Quoted 9 lines
| int acpi_mpam_parse_resources(struct mpam_msc *msc, | struct acpi_mpam_msc_node *tbl_msc) | { | int i, err; | char *ptr, *last_byte; | struct acpi_mpam_resource_node *resource; | | ptr = (char *)(tbl_msc + 1); | last_byte = ptr + tbl_msc->length;
Quoted 3 lines
Nit: misnomer. This is not the address of the last byte.
(I tend to call similar pointers something like "limit".)
It's called table_end elsewhere - I've gone with that.
Quoted 20 lines
Quoted 11 lines
| for (i = 0; i < tbl_msc->num_resource_nodes; i++) { | if (ptr + sizeof(*resource) > last_byte) | return -EINVAL; | | resource = (struct acpi_mpam_resource_node *)ptr; | err = acpi_mpam_parse_resource(msc, resource); | if (err) | return err; | | ptr += sizeof(*resource); | ptr += resource->num_functional_deps * sizeof(struct acpi_mpam_func_deps);This mostly looks sane.
But unless ptr is guaranteed to be less than
0xffffffff00000002(?), then the bounds check at the start of the loop may bogusly pass.Would a check like this before advancing ptr across the functional deps be reasonable? (untested)
Quoted 6 lines
if (resource->num_functional_deps > (last_byte - ptr) / sizeof(struct acpi_mpam_func_deps) return -EINVAL;(Why the table allows more functional dependencies to be declared than could possibly fit in an MSC node is a mystery.)
Giving bits of that names made it easier for me to read. (and fit on one line)
| remaining_table = table_end - ptr;
| max_deps = remaining_table / sizeof(struct acpi_mpam_func_deps);
| if (resource->num_functional_deps > max_deps) {
| pr_debug("MSC has impossible number of functional dependencies\n");
| return -EINVAL;
| }Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 9 lines
drivers/acpi/arm64/mpam.c +9
-0+if (err) +return err; + } + +return 0 ; +} + +static bool __init parse_msc_pm_link(struct acpi_mpam_msc_node *tbl_msc, + struct platform_device *pdev,drivers/acpi/arm64/mpam.c line 141
Quoted 10 lines
drivers/acpi/arm64/mpam.c +10
-0+ u32 *acpi_id) +{ +bool acpi_id_valid = false; +struct acpi_device *buddy; +char hid[16 ], uid[16 ]; +int err; + +memset (&hid,0 ,sizeof (hid)); +memcpy (hid, &tbl_msc->hardware_id_linked_device, +sizeof (tbl_msc->hardware_id_linked_device));This is safe by semi-accident, since 16 > 8.
It might be cleaner to declare
char hid[sizeof(tbl_msc->hardware_id_linked_device)];which can never be wrong.
Quoted 8 lines
Quoted 1 line
But can be too precise!
But only when the correct length was unknown. So you are saying that this can only be correct when it is not known to be correct? ;)
Sometimes an oversized array has useful defensive value against future mis-maintenance, but since the ACPI fields are all fixed-size, I don't see any benefit here.
The field in the table is eight bytes, but the string on the stack must be nine because the table isn't null terminated. I went for the next power of two.
Quoted 15 lines
Quoted 8 lines
Quoted 4 lines
memset()+memcpy() might be better replaced with
strscpy()(or just use snprintf again, since this avoids having to think about multiple different ways of avoiding buffer overflows at the same time. This is not a fast path.)
snprintf()demands space for a NULL byte - but these APCI tables fields are fixed width, so don't have them.Quoted 6 lines
The array sizes picked are the next larger power of 2, and the memset is ensure there is a NULL byte after the fixed size region is copied in.
Where
snprintf()is being used - its to convert from integer to string. For the hid, what we really want is a fixed sizememcpy()... its possible to cook up a format string that will do that, but its going to be an eye sore.
Quoted 7 lines
Well, it's a matter of style.
Quoted 3 lines
I can change this to;
| char hid[sizeof(tbl_msc->hardware_id_linked_device) + 1];Can do (and this would probably have shut me up), but:
fine, lets do that.
Quoted 32 lines
Quoted 7 lines
But I don't think this is better... How about a comment?!
| /* | * tbl_msc->hardware_id_linked_device is an 8 byte fixed width string. | * hid[] is the next larger power of 2, and is zero'd to give us a | * null terminated string for acpi_dev_get_first_match_dev(). | */this doesn't explain why the size needs to be a power of two, or what the wasted bytes are for.
(Either way, this is obviously no big deal...)
Quoted 19 lines
This is all because the MPAM stuff was forced into a static table, even though it needs to refer to stuff in the namespace.
Quoted 12 lines
drivers/acpi/arm64/mpam.c line 148
Quoted 7 lines
drivers/acpi/arm64/mpam.c +7
-0+ +if (!strcmp (hid,ACPI_PROCESSOR_CONTAINER_HID )) { + *acpi_id = tbl_msc->instance_id_linked_device; + acpi_id_valid = true; + } + + err = snprintf(uid, sizeof(uid), "%u",char uid[11]; would be sufficient, here. The instance ID is strictly 32-bit. Adding a safety margin is worthless here, since
snprintf()checks the bounds -- either the size is sufficient for all possible u32 values, or it isn't.
Quoted 6 lines
Quoted 1 line
I just plumped for the next largest power of 2.
This is harmless, but since the only way to know that this is sufficient is to know what the correct value would have been, it seems pointless (as above) to lie to the compiler about how much space can be used.
It's on the stack, the compiler is almost certain to align to register size anyway. Making it obviously big enough saves any time thinking about whether it should be 11 or 12.
I've changed it to 11. (I'll take your word for it!)
Quoted 34 lines
Quoted 5 lines
Quoted 3 lines
drivers/acpi/arm64/mpam.c line 149
Quoted 1 line
drivers/acpi/arm64/mpam.c +1
-0+ tbl_msc->instance_id_linked_device);Can
snprintf()return < 0 on error?Documented as returning a positive number.
Not in C23, and never standard for the
printf()family in general. But it does appear that this is true for the kernel'ssnprintf(), and this assmuption is widely relied upon in the kernel. So, a check here would be pointless compilexity after all.(The only documented error condition in C23 seems the case of a multibyte encoding error, which is not relevant here although it does not rule out other error conditions -- the wording is characteristically ambiguous.)
[...]
Quoted 16 lines
Quoted 4 lines
drivers/acpi/arm64/mpam.c
Quoted 2 lines
drivers/acpi/arm64/mpam.c +2
-0+if (err >=sizeof (uid)) +return acpi_id_valid;Possibly return true on error? Why?
(this is an error case that will never happen - but error handling is important!)
The
acpi_idwas parsed out of the table and returned to the caller. Hence true is returned. This gets used to find the CPU-affinity, which is how the driver avoids accessing caches that might not be on. There are sanity checks around this value. It can't be present for some caches and not others.Due to this error, the link to the buddy device was not added - which is silently ignored. If it matters, it will show up as an access to a device that didn't get turned on by the power-management code, because of that missing link.
Quoted 6 lines
Quoted 2 lines
Because I don't think this can happpen, I didn't try to handle it explicitly - just carry on like nothing is wrong.
What I meant was, if the MPAM table contains garbage, why not just flag this and give up? That feels semantically simpler than trying to limp on, with possibly undiagnosed invalid results.
Not junk in the table - but the kernel was unable to convert a known-size integer to a string in a correctly sized array.
I've added a debug statement for it:
| pr_debug("Failed to convert uid of device for power management.");
Quoted 26 lines
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
drivers/acpi/arm64/mpam.c +4
-0+while (table_offset < table_end) { + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + table_offset += tbl_msc->length; +Quoted 8 lines
drivers/acpi/arm64/mpam.c line 209
Quoted 6 lines
drivers/acpi/arm64/mpam.c +6
-0+/* +* If any of the reserved fields are set, make no attempt to +* parse the msc structure. This will prevent the driver from +* probing all the MSC, meaning it can't discover the system +* wide supported partid and pmg ranges. This avoids whatever +* this MSC is truncating the partids and creating a screaming Mangled sentence?
Quoted 7 lines
just hard to parse:
"whatever this MSC is" "truncating the parids..."Fixed as
| This avoids this MSC truncating the partids and creating a screaming error interrupt.(although its not strictly the MSC that does that)
The critical thing here seems to be that we prevent the driver from ever being enabled. The comment doesn't seem to say that we are actually disabling the MPAM driver by giving up here (it only explains why).
Quoted 7 lines
For the benefit of people who are less familiar with the code, Would something like this be better: [*]
--8<--
If the reserved fields are set then the meaning of the rest of the entry is unknown,
I'm not sure that is true - use of the reserved fields should be backward compatible, and there is no top level type because this thing isn't a subtable, so it isn't possible to add other kinds of MSC - which is where you need the other fields to change meaning.
Quoted 1 line
so leave the MSC marked as unprobed and give up.
It skips creating the platform devices, so there is nothing for the driver to probe against.
Quoted 5 lines
This means that the MPAM driver will never be enabled. There is no way to enable it safely, because we cannot determine safe system-wide partid and pmg ranges in this situation.
-->8--
Combined as:
| * If any of the reserved fields are set, make no attempt to | * parse the MSC structure. This MSC will still be counted, | * meaning the MPAM driver can't probe against all MSC, and | * will never be enabled. There is no way to enable it safely, | * because we cannot determine safe system-wide partid and pmg | * ranges in this situation.
Quoted 13 lines
Quoted 13 lines
Quoted 4 lines
I have not so far found any reference in [2] to the reset value of the MPAMF_ECR.INTEN bit. Do we rely on the error interrupt(s) for all MSCs to be disabled at the interrupt controller? If the same interrupt may be shared by multiple MSCs, that's bad.
They are anticipated as shared because the ESR register lets us know if this MSC triggered the error.
The interrupt won't be delivered until the driver requests it - it'll just stay pending.
Yes - if some firwmare component used an out-of-range PARTID then linux will believe that this was caused by linux and disable MPAM. Not much we can do about that.
Quoted 8 lines
Quoted 1 line
(I've even seen a platform where this happens!)
Ah, right -- I'd missed the ESR.
This is inherently racy even without interrupt sharing, so I guess sharing the interrupt doesn't make things worse (except for having to poll multiple ESRs on interrupt -- but if that hardware vendor made us do that, it's on them.)
The irqchip core will call every handler for a shared interrupt because of that race.
Quoted 69 lines
Quoted 8 lines
Quoted 7 lines
drivers/acpi/arm64/mpam.c line 213
Quoted 4 lines
drivers/acpi/arm64/mpam.c +4
-0+* error interrupt. +*/ +if (tbl_msc->reserved || tbl_msc->reserved1 || tbl_msc->reserved2) +continue ;The specs are not clear about how backwards compatibility is supposed to work.
Quoted 8 lines
How about that!
The spec people thought this would allow them to add things that aren't MSC in, re-using one of these reserved fields as a type.
But - the architecture spec mandates the OS run around the MSC and collect the minimum PARTID/PMG values (so people don't have to build an SoC where everything fits together nicely). Because of that, the OS needs to know how many MSC there are, so that it knows it has probed them all.See my suggestion at [*], above.
The reader doesn't need to understand about future architecture directions, here. The only thing that matters is that the kernel does not make any assumptions about the meaning of the MSC entry if these fields are set. I'm not sure that we need to justify this position -- simply stating it is probably enough.
Quoted 16 lines
Quoted 3 lines
I would feel a bit uneasy about silently throwing away MSCs based on critera that may not indicate incompatibility, and without even a diagnostic.
Yes the MSC gets thrown away - but it was still counted by
acpi_mpam_count_msc(). The MPAM driver will never probe all the MSC, so it will never callmpam_enable().(you used to be able to see which cpuhp callback was registeered, to know it was stuck in discovery, but now a helper does that which always uses the same name. I may fix that)
I think its fine for MPAM to silently fail to probe like this. The system still works, all the known hardware gets probed and none of the unknown hardware is touched. (so it can't blow us up). You don't get resctrl - but we can't know if the unknown hardware was going to be important to supporting MPAM. e.g. it lowers the minimum usable PARTID/PMG.
Quoted 5 lines
It's between a rock and a hard place - I think this behaviour is the best we can do.
Regarding it being silent - whatever happens it needs a kernel upgrade to learn about the newly described hardware. I don't think its worth printing anything out in this case.
I would still vote for a basic
"Unrecognised MSC, MPAM not usable"
message or similar. This costs us nothing, and at least gives a clue that missing kernel support or a corrupt ACPI table are likely causes for resctrl not showing up.
It partly depends on how likely we think this scenario os -- or how likely it is that people will insist on using an old kernel with a botched stack of backports on new hardware. I don't have a strong sense on this.
Quoted 9 lines
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 219
Quoted 6 lines
drivers/acpi/arm64/mpam.c +6
-0+ +if (!tbl_msc->mmio_size) +continue ; + +if (decode_interface_type (tbl_msc, &iface)) +continue ;Ditto regarding diagnostics.
Quoted 8 lines
Quoted 4 lines
A third interface type will result in those MSC being untouched, and MPAM unavailable. I don't think printing "maybe upgrade your kernel?" is going to help anyone. I'll chuck some
pr_debug()in here so someone could find out which of these it is.
pr_debug()for sub-diagnosing the error may be helpful, but these feel like an addition to the basic "MPAM unusable" error rather than as a replacement for it...
Sure, printing multiple levels of error message looks strange to me. But whatever.
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
drivers/acpi/arm64/mpam.c line 226
Quoted 7 lines
drivers/acpi/arm64/mpam.c +7
-0+ + next_res =0 ; + next_prop =0 ; +memset (res,0 ,sizeof (res)); +memset (props,0 ,sizeof (props)); + + pdev =platform_device_alloc ("mpam_msc" , tbl_msc->identifier);If the
tbl_msc->identifiervalues contain duplicates, we will get a platform device with a duplicate name here. I don't know whether it matters.
Quoted 5 lines
Quoted 2 lines
The identifiers must be unique. I don't think this is the sort of table validation we need to do for the firmware vendor.
My question was really: "what goes wrong?" I suppose that if
platform_device_alloc()doesn't like this then
True - you'd get an error from the plafrom device allocation code about the duplicate id.
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
drivers/acpi/arm64/mpam.c line 230
Quoted 5 lines
drivers/acpi/arm64/mpam.c +4
-0+if (!pdev) { + err = -ENOMEM ; +break ; + }we call the caller we ran out of memory. While not exactly helpful, this is better than blowing up. If this is the only consequence of duplicate IDs here (and this anyway Should Not Happen), then I guess this is adequate.
Quoted 2 lines
If
platform_device_alloc()doesn't care about duplicate names, then I guess we can just trust it to have done something sensible.
I'm pretty sure it cares because the names have to be unique under sysfs.
[..]
Quoted 10 lines
Quoted 8 lines
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 240
Quoted 5 lines
drivers/acpi/arm64/mpam.c +5
-0+ +/* Some power management is described in the namespace: */ + err =snprintf (uid,sizeof (uid),"%u" , tbl_msc->identifier); +if (err >0 && err <sizeof (uid)) { + companion =acpi_dev_get_first_match_dev ("ARMHAA5C" , uid,-1 );Diagnostic?
Ping
In line with what I did for the previous ones, I shoved something here.
[...]
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 216
Quoted 5 lines
drivers/acpi/arm64/mpam.c +5
-0+ table_end = (char *)table + table->length; + +while (table_offset < table_end) { +if (!tbl_msc->mmio_size) +continue ;This is 0 for non-usable PCC-based MSCs, right?
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
(Why explicitly unusable MSCs are listed in the table at all is a mystery to me, but that's what the spec says. I guess there must be a reason.)
(I guess PCC was added after the size field was named. I suppose we don't really need a comment for that -- it's fairly clear why this bit of code is here. We could rename the field in our struct, but this
(its imported from acpica - it would take quite a long time to change)
Quoted 1 line
would probably just move confusion around rather than solving it.)
[...]
Quoted 10 lines
Quoted 5 lines
Quoted 1 line
Also, is it an error if a length is not a multiple of four bytes?
Hmmm, because all the subtables in the sepc are multiples of four bytes? I think this falls in the bucket of 'table validation the OS shouldn't have to do'. If the ACPI tables are that wrong, we're going to have bigger problems.
So long as we don't fault on unaligned access in the kernel (which I'm pretty sure we don't do), then nothing goes wrong in the unaligned case -- so, fine. Otherwise we'll get a clean Oops, which would at least be a decent place to start debugging.
UEFI gets to choose the attibutes, but if it choses Device, things will blow up well before we get in here.
Thanks,
James
James Morse
maintainerPATCH 8/33 ACPI / MPAM: Parse the MPAM table
In reply to James MorseAdd code to parse the arm64 specific MPAM table, looking up the cache level from the PPTT and feeding the end result into the MPAM driver.
---
Changes since RFC:
* Used DEFINE_RES_IRQ_NAMED() and friends macros.
* Additional error handling.
* Check for zero sized MSC.
* Allow table revisions greater than 1. (no spec for revision 0!)
* Use cleanup helpers to retrive ACPI tables, which allows some functions to be folded together.
--- arch/arm64/Kconfig | 1 + drivers/acpi/arm64/Kconfig | 3 + drivers/acpi/arm64/Makefile | 1 + drivers/acpi/arm64/mpam.c | 331 ++++++++++++++++++++++++++++++++++++ drivers/acpi/tables.c | 2 +- include/linux/arm_mpam.h | 46 +++++ 6 files changed, 383 insertions(+), 1 deletion(-) create mode 100644 drivers/acpi/arm64/mpam.c create mode 100644 include/linux/arm_mpam.h
arch/arm64/Kconfig +1 -0
@@ -2062,6 +2062,7 @@ config ARM64_TLB_RANGE config ARM64_MPAM bool "Enable support for MPAM" + select ACPI_MPAM if ACPI help Memory Partitioning and Monitoring is an optional extension that allows the CPUs to mark load and store transactions with
drivers/acpi/arm64/Kconfig +3 -0
@@ -21,3 +21,6 @@ config ACPI_AGDI config ACPI_APMT bool + +config ACPI_MPAM + bool
drivers/acpi/arm64/Makefile +1 -0
@@ -4,6 +4,7 @@ obj-$(CONFIG_ACPI_APMT) += apmt.o obj-$(CONFIG_ACPI_FFH) += ffh.o obj-$(CONFIG_ACPI_GTDT) += gtdt.o obj-$(CONFIG_ACPI_IORT) += iort.o +obj-$(CONFIG_ACPI_MPAM) += mpam.o obj-$(CONFIG_ACPI_PROCESSOR_IDLE) += cpuidle.o obj-$(CONFIG_ARM_AMBA) += amba.o obj-y += dma.o init.o
drivers/acpi/arm64/mpam.c new file +331 -0
331 lines not on this page · raw message
drivers/acpi/tables.c +1 -1
@@ -408,7 +408,7 @@ static const char table_sigs[][ACPI_NAMESEG_SIZE] __nonstring_array __initconstACPI_SIG_PSDT ,ACPI_SIG_RSDT ,ACPI_SIG_XSDT ,ACPI_SIG_SSDT ,ACPI_SIG_IORT ,ACPI_SIG_NFIT ,ACPI_SIG_HMAT ,ACPI_SIG_PPTT ,ACPI_SIG_NHLT ,ACPI_SIG_AEST ,ACPI_SIG_CEDT ,ACPI_SIG_AGDI , - ACPI_SIG_NBFT }; + ACPI_SIG_NBFT, ACPI_SIG_MPAM };#define ACPI_HEADER_SIZE sizeof(struct acpi_table_header)
include/linux/arm_mpam.h new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.20.1
RE: [PATCH 08/33] ACPI / MPAM: Parse the MPAM table
Hello James,
Quoted 258 lines
Add code to parse the arm64 specific MPAM table, looking up the cache level from the PPTT and feeding the end result into the MPAM driver.
CC Carl Worth <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* UsedDEFINE_RES_IRQ_NAMED()and friends macros.
* Additional error handling.
* Check for zero sized MSC.
* Allow table revisions greater than 1. (no spec for revision 0!)
* Use cleanup helpers to retrive ACPI tables, which allows some functions to be folded together.--- arch/arm64/Kconfig | 1 + drivers/acpi/arm64/Kconfig | 3 + drivers/acpi/arm64/Makefile | 1 + drivers/acpi/arm64/mpam.c | 331 ++++++++++++++++++++++++++++++++++++ drivers/acpi/tables.c | 2 +- include/linux/arm_mpam.h | 46 +++++ 6 files changed, 383 insertions(+), 1 deletion(-) create mode 100644
drivers/acpi/arm64/mpam.ccreate mode 100644include/linux/arm_mpam.harch/arm64/Kconfig index renamed from arch/arm64/Kconfig +0
-0
658e47fc0c5a..e51ccf1da102100644arch/arm64/Kconfig +1
-1@@ -2062,6 +2062,7 @@ config ARM64_TLB_RANGE config ARM64_MPAM bool "Enable support for MPAM" + select ACPI_MPAM if ACPI help Memory Partitioning and Monitoring is an optional extension that allows the CPUs to mark load and store transactions with diff --git a/drivers/acpi/arm64/Kconfig b/drivers/acpi/arm64/Kconfig index
b3ed6212244c..f2fd79f22e7d100644drivers/acpi/arm64/Kconfig +3
-0@@ -21,3 +21,6 @@ config ACPI_AGDI config ACPI_APMT bool + +config ACPI_MPAM + booldrivers/acpi/arm64/Makefile index renamed from drivers/acpi/arm64/Makefile +0
-0
05ecde9eaabe..9390b57cb564100644drivers/acpi/arm64/Makefile +1
-0@@ -4,6 +4,7 @@ obj-$(CONFIG_ACPI_APMT) += apmt.o obj-$(CONFIG_ACPI_FFH) += ffh.o obj-$(CONFIG_ACPI_GTDT) += gtdt.o obj-$(CONFIG_ACPI_IORT) += iort.o +obj-$(CONFIG_ACPI_MPAM) += mpam.o obj-$(CONFIG_ACPI_PROCESSOR_IDLE) += cpuidle.o obj-$(CONFIG_ARM_AMBA) += amba.o obj-y += dma.o init.odrivers/acpi/arm64/mpam.c new renamed from drivers/acpi/arm64/mpam.c +0
-0file mode 100644 index 000000000000..e55fc2729ac5
drivers/acpi/arm64/mpam.c new file +23
-0@@ -0,0 +1,331 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2025 Arm Ltd. + +/* Parse the MPAM ACPI table feeding the discovered nodes into the +driver */ + +#define pr_fmt (fmt) "ACPI MPAM: " fmt + +#include <linux/acpi.h> +#include <linux/arm_mpam.h> +#include <linux/bits.h> +#include <linux/cpu.h> +#include <linux/cpumask.h> +#include <linux/platform_device.h> + +#include <acpi/processor.h> + +/* +* Flags for acpi_table_mpam_msc.*_interrupt_flags. +* See 2.1.1 Interrupt Flags, Table 5, of DEN0065B_MPAM_ACPI_3.0-bet. +*/ +#define ACPI_MPAM_MSC_IRQ_MODE_MASK BIT(0) +#define ACPI_MPAM_MSC_IRQ_TYPE_MASK
GENMASK(2, 1)drivers/acpi/arm64/mpam.c new file +2
-0+#define ACPI_MPAM_MSC_IRQ_TYPE_WIRED 0 +#define ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER BIT(3)
drivers/acpi/arm64/mpam.c new file +25
-0+#define ACPI_MPAM_MSC_IRQ_AFFINITY_VALID BIT(4) + +static bool frob_irq (struct platform_device *pdev,int intid,u32 flags, +int *irq,u32 processor_container_uid) { +int sense; + +if (!intid) +return false; + +if (FIELD_GET (ACPI_MPAM_MSC_IRQ_TYPE_MASK , flags) != +ACPI_MPAM_MSC_IRQ_TYPE_WIRED ) +return false; + + sense =FIELD_GET (ACPI_MPAM_MSC_IRQ_MODE_MASK , flags); + +/* +* If the GSI is in the GIC's PPI range, try and create a partitioned +* percpu interrupt. +*/ +if (16 <= intid && intid <32 && processor_container_uid != ~0 ) { +pr_err_once ("Partitioned interrupts not supported\n" ); +return false; + } + + *irq = acpi_register_gsi(&pdev->dev, intid, sense,
ACPI_ACTIVE_HIGH);drivers/acpi/arm64/mpam.c new file +18
-0+if (*irq <=0 ) { +pr_err_once ("Failed to register interrupt 0x%x with ACPI\n" , + intid); +return false; + } + +return true; +} + +static void acpi_mpam_parse_irqs(struct platform_device *pdev, + struct acpi_mpam_msc_node *tbl_msc, + struct resource *res, int *res_idx) { + u32 flags, aff; + int irq; + + flags = tbl_msc->overflow_interrupt_flags; + if (flags & ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &
ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER)drivers/acpi/arm64/mpam.c new file +5
-0+ aff = tbl_msc->overflow_interrupt_affinity; + else + aff = ~0 ; +if (frob_irq (pdev, tbl_msc->overflow_interrupt, flags, &irq, aff)) + res[(*res_idx)++] = DEFINE_RES_IRQ_NAMED(irq,"overflow");
drivers/acpi/arm64/mpam.c new file +4
-0+ + flags = tbl_msc->error_interrupt_flags; +if (flags &ACPI_MPAM_MSC_IRQ_AFFINITY_VALID && + flags &
ACPI_MPAM_MSC_IRQ_AFFINITY_PROCESSOR_CONTAINER)drivers/acpi/arm64/mpam.c new file +17
-0+ aff = tbl_msc->error_interrupt_affinity; + else + aff = ~0 ; +if (frob_irq (pdev, tbl_msc->error_interrupt, flags, &irq, aff)) + res[(*res_idx)++] =DEFINE_RES_IRQ_NAMED (irq,"error" ); } + +static int acpi_mpam_parse_resource (struct mpam_msc *msc, +struct acpi_mpam_resource_node *res) { +int level, nid; +u32 cache_id; + +switch (res->locator_type) { +case ACPI_MPAM_LOCATION_TYPE_PROCESSOR_CACHE : + cache_id = res->locator.cache_locator.cache_reference; + level =find_acpi_cache_level_from_id (cache_id); +if (level <=0 ) { + pr_err_once("Bad level (%u) for cache with id %u\n",level,
cache_id);drivers/acpi/arm64/mpam.c new file +3
-0+return -EINVAL ; + } + return mpam_ris_create(msc, res->ris_index,
MPAM_CLASS_CACHE,drivers/acpi/arm64/mpam.c new file +3
-0+ level, cache_id); +case ACPI_MPAM_LOCATION_TYPE_MEMORY : + nid =
pxm_to_node(res->locator.memory_locator.proximity_domain);drivers/acpi/arm64/mpam.c new file +3
-0+if (nid ==NUMA_NO_NODE ) + nid =0 ; + return mpam_ris_create(msc, res->ris_index,
MPAM_CLASS_MEMORY,drivers/acpi/arm64/mpam.c new file +23
-0+ 255, nid); +default : +/* These get discovered later and treated as unknown */ +return 0 ; + } +} + +int acpi_mpam_parse_resources (struct mpam_msc *msc, +struct acpi_mpam_msc_node *tbl_msc) { +int i, err; +struct acpi_mpam_resource_node *resources; + + resources = (struct acpi_mpam_resource_node *)(tbl_msc +1 ); +for (i =0 ; i < tbl_msc->num_resource_nodes; i++) { + err =acpi_mpam_parse_resource (msc, &resources[i]); +if (err) +return err; + } + +return 0 ; +} + +static bool __init parse_msc_pm_link(struct acpi_mpam_msc_node
*tbl_msc,drivers/acpi/arm64/mpam.c new file +25
-0+struct platform_device *pdev, + u32 *acpi_id) +{ + bool acpi_id_valid = false; +struct acpi_device *buddy; +char hid[16 ], uid[16 ]; +int err; + +memset (&hid,0 ,sizeof (hid)); +memcpy (hid, &tbl_msc->hardware_id_linked_device, +sizeof (tbl_msc->hardware_id_linked_device)); + +if (!strcmp (hid,ACPI_PROCESSOR_CONTAINER_HID )) { + *acpi_id = tbl_msc->instance_id_linked_device; + acpi_id_valid = true; + } + + err =snprintf (uid,sizeof (uid),"%u" , + tbl_msc->instance_id_linked_device); +if (err >=sizeof (uid)) +return acpi_id_valid; + + buddy =acpi_dev_get_first_match_dev (hid, uid,-1 ); + if (buddy) + device_link_add(&pdev->dev, &buddy->dev,
DL_FLAG_STATELESS);drivers/acpi/arm64/mpam.c new file +21
-0+ +return acpi_id_valid; +} + +static int decode_interface_type (struct acpi_mpam_msc_node *tbl_msc, +enum mpam_msc_iface *iface) +{ +switch (tbl_msc->interface_type) { +case 0 : + *iface =MPAM_IFACE_MMIO ; +return 0 ; +case 0xa : + *iface =MPAM_IFACE_PCC ; +return 0 ; +default : +return -EINVAL ; + } +} + +static int __init acpi_mpam_parse (void ) { + struct acpi_table_header *table __free(acpi_table) =
acpi_get_table_ret(ACPI_SIG_MPAM, 0);drivers/acpi/arm64/mpam.c new file +6
-0+char *table_end, *table_offset = (char *)(table +1 ); +struct property_entry props[4 ];/* needs a sentinel */ +struct acpi_mpam_msc_node *tbl_msc; +int next_res, next_prop, err =0 ; +struct acpi_device *companion; +struct platform_device *pdev;
drivers/acpi/arm64/mpam.c line 193
Quoted 10 lines
drivers/acpi/arm64/mpam.c new file +10
-0+enum mpam_msc_iface iface; +struct resource res[3 ]; +char uid[16 ]; +u32 acpi_id; + +if (acpi_disabled || !system_supports_mpam () ||IS_ERR (table)) +return 0 ; + +if (IS_ERR (table)) +return 0 ;
This is redundant, it's the same as the previous line.
Best regards,
Shaopeng TAN
Quoted 223 lines
drivers/acpi/arm64/mpam.c new file +14
-0+if (table->revision <1 ) +return 0 ; + + table_end = (char *)table + table->length; + +while (table_offset < table_end) { + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + table_offset += tbl_msc->length; + +/* +* If any of the reserved fields are set, make no attempt to +* parse the msc structure. This will prevent the driver from +* probing all the MSC, meaning it can't discover the system +* wide supported partid and pmg ranges. This avoids whatever
drivers/acpi/arm64/mpam.c new file +4
-0+* this MSC is truncating the partids and creating a screaming +* error interrupt. +*/ + if (tbl_msc->reserved || tbl_msc->reserved1 ||
tbl_msc->reserved2)drivers/acpi/arm64/mpam.c new file +14
-0+continue ; + +if (!tbl_msc->mmio_size) +continue ; + +if (decode_interface_type (tbl_msc, &iface)) +continue ; + + next_res =0 ; + next_prop =0 ; +memset (res,0 ,sizeof (res)); +memset (props,0 ,sizeof (props)); + + pdev = platform_device_alloc("mpam_msc",
tbl_msc->identifier);drivers/acpi/arm64/mpam.c new file +11
-0+if (!pdev) { + err = -ENOMEM ; +break ; + } + +if (tbl_msc->length <sizeof (*tbl_msc)) { + err = -EINVAL ; +break ; + } + +/* Some power management is described in the namespace: */
drivers/acpi/arm64/mpam.c new file +3
-0+ err =snprintf (uid,sizeof (uid),"%u" , tbl_msc->identifier); +if (err >0 && err <sizeof (uid)) { + companion =
acpi_dev_get_first_match_dev("ARMHAA5C", uid, -1);drivers/acpi/arm64/mpam.c new file +2
-0+if (companion) + ACPI_COMPANION_SET(&pdev->dev,companion);
drivers/acpi/arm64/mpam.c new file +4
-0+ } + +if (iface ==MPAM_IFACE_MMIO ) { + res[next_res++] =DEFINE_RES_MEM_NAMED(tbl_msc->base_address,
+tbl_msc->mmio_size,
+
"MPAM:MSC");drivers/acpi/arm64/mpam.c new file +2
-0+ }else if (iface ==MPAM_IFACE_PCC ) { + props[next_prop++] =PROPERTY_ENTRY_U32("pcc-channel",
+tbl_msc->base_address);drivers/acpi/arm64/mpam.c new file +9
-0+ next_prop++; + } + +acpi_mpam_parse_irqs (pdev, tbl_msc, res, &next_res); + err =platform_device_add_resources (pdev, res, next_res); +if (err) +break ; + + props[next_prop++] =PROPERTY_ENTRY_U32("arm,not-ready-us",
+tbl_msc->max_nrdy_usec);drivers/acpi/arm64/mpam.c new file +8
-0+ +/* +* The MSC's CPU affinity is described via its linked power +* management device, but only if it points at a Processor or +* Processor Container. +*/ +if (parse_msc_pm_link (tbl_msc, pdev, &acpi_id)) { + props[next_prop++] =PROPERTY_ENTRY_U32("cpu_affinity",
drivers/acpi/arm64/mpam.c new file +4
-0+ acpi_id); + } + + err = device_create_managed_software_node(&pdev->dev,props,
drivers/acpi/arm64/mpam.c new file +6
-0+ NULL); +if (err) +break ; + +/* Come back later if you want the RIS too */ + err = platform_device_add_data(pdev, tbl_msc,
tbl_msc->length);drivers/acpi/arm64/mpam.c new file +17
-0+if (err) +break ; + + err =platform_device_add (pdev); +if (err) +break ; + } + +if (err) +platform_device_put (pdev); + +return err; +} + +int acpi_mpam_count_msc (void ) +{ + struct acpi_table_header *table __free(acpi_table) =
acpi_get_table_ret(ACPI_SIG_MPAM, 0);drivers/acpi/arm64/mpam.c new file +37
-0+char *table_end, *table_offset = (char *)(table +1 ); +struct acpi_mpam_msc_node *tbl_msc; +int count =0 ; + +if (IS_ERR (table)) +return 0 ; + +if (table->revision <1 ) +return 0 ; + + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + table_end = (char *)table + table->length; + +while (table_offset < table_end) { +if (!tbl_msc->mmio_size) +continue ; + +if (tbl_msc->length <sizeof (*tbl_msc)) +return -EINVAL ; + + count++; + + table_offset += tbl_msc->length; + tbl_msc = (struct acpi_mpam_msc_node *)table_offset; + } + +return count; +} + +/* +* Call after ACPI devices have been created, which happens behind +acpi_scan_init() +* called from subsys_initcall(). PCC requires the mailbox driver, +which is +* initialised from postcore_initcall(). +*/ +subsys_initcall_sync (acpi_mpam_parse);drivers/acpi/tables.c index renamed from drivers/acpi/tables.c +0
-0
fa9bb8c8ce95..835e3795ede3100644drivers/acpi/tables.c +0
-0@@ -408,7 +408,7 @@ static const char table_sigs[][ACPI_NAMESEG_SIZE]
__nonstring_array__initconstdrivers/acpi/tables.c +1
-1ACPI_SIG_PSDT ,ACPI_SIG_RSDT ,ACPI_SIG_XSDT ,ACPI_SIG_SSDT ,ACPI_SIG_IORT ,ACPI_SIG_NFIT ,ACPI_SIG_HMAT ,ACPI_SIG_PPTT ,ACPI_SIG_NHLT ,ACPI_SIG_AEST ,ACPI_SIG_CEDT ,ACPI_SIG_AGDI , - ACPI_SIG_NBFT }; + ACPI_SIG_NBFT, ACPI_SIG_MPAM };#define ACPI_HEADER_SIZE sizeof(struct acpi_table_header)include/linux/arm_mpam.h new file renamed from include/linux/arm_mpam.h +0
-0mode 100644 index 000000000000..0edefa6ba019
include/linux/arm_mpam.h new file +31
-0@@ -0,0 +1,46 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2025 Arm Ltd. */ + +#ifndef __LINUX_ARM_MPAM_H +#define __LINUX_ARM_MPAM_H + +#include <linux/acpi.h> +#include <linux/types.h> + +struct mpam_msc ; + +enum mpam_msc_iface { +MPAM_IFACE_MMIO ,/* a real MPAM MSC */ +MPAM_IFACE_PCC ,/* a fake MPAM MSC */ +}; + +enum mpam_class_types { +MPAM_CLASS_CACHE ,/* Well known caches, e.g. L2 */ +MPAM_CLASS_MEMORY ,/* Main memory */ +MPAM_CLASS_UNKNOWN ,/* Everything else, e.g. SMMU */ +}; + +#ifdef CONFIG_ACPI_MPAM +/* Parse the ACPI description of resources entries for this MSC. */ int +acpi_mpam_parse_resources (struct mpam_msc *msc, +struct acpi_mpam_msc_node *tbl_msc); + +int acpi_mpam_count_msc (void ); +#else +static inline int acpi_mpam_parse_resources (struct mpam_msc *msc, + struct acpi_mpam_msc_node
*tbl_msc) {include/linux/arm_mpam.h new file +7
-0+return -EINVAL ; +} + +static inline int acpi_mpam_count_msc (void ) {return -EINVAL ; }#endif + +static inline int mpam_ris_create (struct mpam_msc *msc,u8 ris_idx, + enum mpam_class_types type, u8
class_id,include/linux/arm_mpam.h new file +6
-1+ int component_id) +{ +return -EINVAL ; +} + +#endif /* __LINUX_ARM_MPAM_H */ --2.20.1
James Morse
maintainerRe: [PATCH 08/33] ACPI / MPAM: Parse the MPAM table
Hi Shaopeng,
Quoted 22 lines · Shaopeng Tan (Fujitsu)
Quoted 22 lines
Add code to parse the arm64 specific MPAM table, looking up the cache level from the PPTT and feeding the end result into the MPAM driver.
drivers/acpi/arm64/mpam.c new renamed from drivers/acpi/arm64/mpam.c +0
-0file mode 100644 index 000000000000..e55fc2729ac5
drivers/acpi/arm64/mpam.c new file +2
-0@@ -0,0 +1,331 @@ +static int __init acpi_mpam_parse (void ) { + struct acpi_table_header *table __free(acpi_table) =
acpi_get_table_ret(ACPI_SIG_MPAM, 0);drivers/acpi/arm64/mpam.c new file +10
-0+char *table_end, *table_offset = (char *)(table +1 ); +struct property_entry props[4 ];/* needs a sentinel */ +struct acpi_mpam_msc_node *tbl_msc; +int next_res, next_prop, err =0 ; +struct acpi_device *companion; +struct platform_device *pdev; +enum mpam_msc_iface iface; +struct resource res[3 ]; +char uid[16 ]; +u32 acpi_id;
Quoted 8 lines
drivers/acpi/arm64/mpam.c line 193
Quoted 6 lines
drivers/acpi/arm64/mpam.c new file +6
-0+ +if (acpi_disabled || !system_supports_mpam () ||IS_ERR (table)) +return 0 ; + +if (IS_ERR (table)) +return 0 ;This is redundant, it's the same as the previous line.
Fixed, thanks.
James
James Morse
maintainerPATCH 9/33 dt-bindings: arm: Add MPAM MSC binding
In reply to James MorseFrom: Rob Herring <[email protected]>
The binding is designed around the assumption that an MSC will be a sub-block of something else such as a memory controller, cache controller, or IOMMU. However, it's certainly possible a design does not have that association or has a mixture of both, so the binding illustrates how we can support that with RIS child nodes.
A key part of MPAM is we need to know about all of the MSCs in the system before it can be enabled. This drives the need for the genericish 'arm,mpam-msc' compatible. Though we can't assume an MSC is accessible until a h/w specific driver potentially enables the h/w.
---
Changes since RFC:
* Syntax(?) corrections supplied by Rob.
* Culled some context in the example.
--- .../devicetree/bindings/arm/arm,mpam-msc.yaml | 200 ++++++++++++++++++ 1 file changed, 200 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml new file +200 -0
@@ -0,0 +1,200 @@ +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/arm/arm,mpam-msc.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Arm Memory System Resource Partitioning and Monitoring (MPAM) + +description: | + The Arm MPAM specification can be found here: + + https://developer.arm.com/documentation/ddi0598/latest + +maintainers: + - Rob Herring <[email protected]> + +properties: + compatible: + items: + - const: arm,mpam-msc # Further details are discoverable + - const: arm,mpam-memory-controller-msc + + reg: + maxItems: 1 + description: A memory region containing registers as defined in the MPAM + specification. + + interrupts: + minItems: 1 + items: + - description: error (optional) + - description: overflow (optional, only for monitoring) + + interrupt-names: + oneOf: + - items: + - enum: [ error, overflow ] + - items: + - const: error + - const: overflow + + arm,not-ready-us: + description: The maximum time in microseconds for monitoring data to be + accurate after a settings change. For more information, see the + Not-Ready (NRDY) bit description in the MPAM specification. + + numa-node-id: true # see NUMA binding + + '#address-cells': + const: 1 + + '#size-cells': + const: 0 + +patternProperties: + '^ris@[0-9a-f]$': + type: object + additionalProperties: false + description: + RIS nodes for each RIS in an MSC. These nodes are required for each RIS + implementing known MPAM controls + + properties: + compatible: + enum: + # Bulk storage for cache + - arm,mpam-cache + # Memory bandwidth + - arm,mpam-memory + + reg: + minimum: 0 + maximum: 0xf + + cpus: + description: + Phandle(s) to the CPU node(s) this RIS belongs to. By default, the parent + device's affinity is used. + + arm,mpam-device: + $ref: /schemas/types.yaml#/definitions/phandle + description: + By default, the MPAM enabled device associated with a RIS is the MSC's + parent node. It is possible for each RIS to be associated with different + devices in which case 'arm,mpam-device' should be used. + + required: + - compatible + - reg + +required: + - compatible + - reg + +dependencies: + interrupts: [ interrupt-names ] + +additionalProperties: false + +examples: + - | + L3: cache-controller@30000000 { + compatible = "arm,dsu-l3-cache", "cache"; + cache-level = <3>; + cache-unified; + + ranges = <0x0 0x30000000 0x800000>; + #address-cells = <1>; + #size-cells = <1>; + + msc@10000 { + compatible = "arm,mpam-msc"; + + /* CPU affinity implied by parent cache node's */ + reg = <0x10000 0x2000>; + interrupts = <1>, <2>; + interrupt-names = "error", "overflow"; + arm,not-ready-us = <1>; + }; + }; + + mem: memory-controller@20000 { + compatible = "foo,a-memory-controller"; + reg = <0x20000 0x1000>; + + #address-cells = <1>; + #size-cells = <1>; + ranges; + + msc@21000 { + compatible = "arm,mpam-memory-controller-msc", "arm,mpam-msc"; + reg = <0x21000 0x1000>; + interrupts = <3>; + interrupt-names = "error"; + arm,not-ready-us = <1>; + numa-node-id = <1>; + }; + }; + + iommu@40000 { + reg = <0x40000 0x1000>; + + ranges; + #address-cells = <1>; + #size-cells = <1>; + + msc@41000 { + compatible = "arm,mpam-msc"; + reg = <0 0x1000>; + interrupts = <5>, <6>; + interrupt-names = "error", "overflow"; + arm,not-ready-us = <1>; + + #address-cells = <1>; + #size-cells = <0>; + + ris@2 { + compatible = "arm,mpam-cache"; + reg = <0>; + // TODO: How to map to device(s)? + }; + }; + }; + + msc@80000 { + compatible = "foo,a-standalone-msc"; + reg = <0x80000 0x1000>; + + clocks = <&clks 123>; + + ranges; + #address-cells = <1>; + #size-cells = <1>; + + msc@10000 { + compatible = "arm,mpam-msc"; + + reg = <0x10000 0x2000>; + interrupts = <7>; + interrupt-names = "overflow"; + arm,not-ready-us = <1>; + + #address-cells = <1>; + #size-cells = <0>; + + ris@0 { + compatible = "arm,mpam-cache"; + reg = <0>; + arm,mpam-device = <&L2_0>; + }; + + ris@1 { + compatible = "arm,mpam-memory"; + reg = <1>; + arm,mpam-device = <&mem>; + }; + }; + }; + +...
Signature · 1 line
2.20.1
Re: [PATCH 09/33] dt-bindings: arm: Add MPAM MSC binding
Hi James,
Quoted 8 lines · James Morse
From: Rob Herring <[email protected]>
The binding is designed around the assumption that an MSC will be a sub-block of something else such as a memory controller, cache controller, or IOMMU. However, it's certainly possible a design does not have that association or has a mixture of both, so the binding illustrates how we can support that with RIS child nodes.
Quoted 4 lines
A key part of MPAM is we need to know about all of the MSCs in the system before it can be enabled. This drives the need for the genericish 'arm,mpam-msc' compatible. Though we can't assume an MSC is accessible until a h/w specific driver potentially enables the h/w.
I'll leave detailed review to other people for now, since I'm not so up to speed on all things DT.
A few random comments, below.
[...]
Quoted 1 line
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +0
-0
[...]
Quoted 1 line
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +0
-0@@ -0,0 +1,200 @@
[...]
Quoted 5 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +5
-0+title: Arm Memory System Resource Partitioning and Monitoring (MPAM) + +description: | + The Arm MPAM specification can be found here: +
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 21
Quoted 10 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +10
-0+ https://developer.arm.com/documentation/ddi0598/latest + +maintainers: + - Rob Herring <[email protected]> + +properties: + compatible: + items: + - const: arm,mpam-msc # Further details are discoverable + - const: arm,mpam-memory-controller-msc
There seems to be no clear statement about how these differ.
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 26
Quoted 4 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +4
-0+ reg: + maxItems: 1 + description: A memory region containing registers as defined in the MPAM + specification.
There seems to be no handling of PCC-based MSCs here. Should there be?
If this can be added later in a backwards-compatible way, I guess that's not a problem (and this is what compatible strings are for, if all else fails.)
An explicit statement that PCC is not supported here might be helpful, though.
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 40
Quoted 13 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +13
-0+ interrupts: + minItems: 1 + items: + - description: error (optional) + - description: overflow (optional, only for monitoring) + + interrupt-names: + oneOf: + - items: + - enum: [ error, overflow ] + - items: + - const: error + - const: overflow
Yeugh. Is this really the only way to say "one or both of foo"?
(I don't know the answer to this -- though I can believe that it's true. Perhaps just not describing this property is another option. Many bindings seem not to bother.)
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ + arm,not-ready-us: + description: The maximum time in microseconds for monitoring data to be + accurate after a settings change. For more information, see the + Not-Ready (NRDY) bit description in the MPAM specification. +
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 56
Quoted 10 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +10
-0+ numa-node-id: true # see NUMA binding + + '#address-cells': + const: 1 + + '#size-cells': + const: 0 + +patternProperties: + '^ris@[0-9a-f]$':
It this supposed to be '^ris@[0-9a-f]+$' ?
Currently MPAMF_IDR.RIS_MAX is only 4 bits in size and so cannot be greater than 0xf. But it is not inconceivable that a future revision of the architecture might enable more -- and the are 4 RES0 bits looming over the RIS_MAX field, just waiting to be used...
(In any case, it feels wrong to try to enforce numeric bounds with a regex, even in the cases where it happens to work straightforwardly.)
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 60
Quoted 4 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +4
-0+ type: object + additionalProperties: false + description: + RIS nodes for each RIS in an MSC. These nodes are required for each RIS
The architectural term is "resource instance", not "RIS".
But "RIS nodes" is fine for describing the DT nodes, since we can call them what we like, and "ris" is widely used inside the MPAM driver.
People writing DTs should not need to be familiar with the driver's internal naming conventions, though.
(There are other instances, but I won't comment on them all individually.)
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 66
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ implementing known MPAM controls + + properties: + compatible: + enum: + # Bulk storage for cache
Nit: What is "bulk storage"?
The MPAM spec just refers to "cache" or "cache memory".
Quoted 7 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +7
-0+ - arm,mpam-cache + # Memory bandwidth + - arm,mpam-memory + + reg: + minimum: 0 + maximum: 0xf
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 83
Quoted 10 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +10
-0+ + cpus: + description: + Phandle(s) to the CPU node(s) this RIS belongs to. By default, the parent + device's affinity is used. + + arm,mpam-device: + $ref: /schemas/types.yaml#/definitions/phandle + description: + By default, the MPAM enabled device associated with a RIS is the MSC's
Associated how? Is this the device where the physical resources managed by the MSC are located?
Quoted 2 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +2
-0+ parent node. It is possible for each RIS to be associated with different + devices in which case 'arm,mpam-device' should be used.
[...]
Quoted 5 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +5
-0+examples: + - | + L3: cache-controller@30000000 { + compatible = "arm,dsu-l3-cache", "cache"; + cache-level = <3>;
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 114
Quoted 10 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +10
-0+ cache-unified; + + ranges = <0x0 0x30000000 0x800000>; + #address-cells = <1>; + #size-cells = <1>; + + msc@10000 { + compatible = "arm,mpam-msc"; + + /* CPU affinity implied by parent cache node's */
"node's" -> "nodes".
(or it this supposed to be in the singular -- i.e., the immediately parent cache node only?)
Anyway, it looks like this is commenting on the "reg" property, which doesn't seem right.
Is this commnent supposed instead to explain the omission of the "cpus" property? If so, that should be made clearer.
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 120
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ reg = <0x10000 0x2000>; + interrupts = <1>, <2>; + interrupt-names = "error", "overflow"; + arm,not-ready-us = <1>; + }; + };
[...]
(Examples otherwise not reviewed in detail.)
Cheers
---Dave
James Morse
maintainerRe: [PATCH 09/33] dt-bindings: arm: Add MPAM MSC binding
Hi Dave,
Quoted 9 lines · Dave Martin
Quoted 8 lines · James Morse
From: Rob Herring <[email protected]>
The binding is designed around the assumption that an MSC will be a sub-block of something else such as a memory controller, cache controller, or IOMMU. However, it's certainly possible a design does not have that association or has a mixture of both, so the binding illustrates how we can support that with RIS child nodes.
Quoted 7 lines
Quoted 4 lines
A key part of MPAM is we need to know about all of the MSCs in the system before it can be enabled. This drives the need for the genericish 'arm,mpam-msc' compatible. Though we can't assume an MSC is accessible until a h/w specific driver potentially enables the h/w.
I'll leave detailed review to other people for now, since I'm not so up to speed on all things DT.
Me neither!
Quoted 17 lines
Quoted 1 line
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +0
-0[...]
Quoted 1 line
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +0
-0@@ -0,0 +1,200 @@[...]
Quoted 9 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +9
-0+title: Arm Memory System Resource Partitioning and Monitoring (MPAM) + +description: | + The Arm MPAM specification can be found here: + + https://developer.arm.com/documentation/ddi0598/latest + +maintainers: + - Rob Herring <[email protected]>
Quoted 8 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 21
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ +properties: + compatible: + items: + - const: arm,mpam-msc # Further details are discoverable + - const: arm,mpam-memory-controller-mscThere seems to be no clear statement about how these differ.
It's a more-specific compatible, I think these are usually things like:
| compatible = "acme,mega-cache-9000", "arm,mpam-msc"
Where the driver can key errata-workaround on the vendor specific bit when needed.
In this case - I think they're examples, but Rob said they were supposed to be in some other list of compatible. (not sure what/where that is)
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 26
Quoted 4 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +4
-0+ reg: + maxItems: 1 + description: A memory region containing registers as defined in the MPAM + specification.There seems to be no handling of PCC-based MSCs here. Should there be?
That is newer than this document. On DT platforms PCC is spelled SCMI, and is discoverable. Andre P prototyped this, (patches in the extras branch) but no-one has come out of the woodwork to say they actually need it yet.
ACPI PCC is a definite maybe.
Quoted 6 lines
If this can be added later in a backwards-compatible way, I guess that's not a problem (and this is what compatible strings are for, if all else fails.)
An explicit statement that PCC is not supported here might be helpful, though.
I'm pretty sure its discoverable on DT/SCMI platforms.
Quoted 30 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 40
Quoted 13 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +13
-0+ interrupts: + minItems: 1 + items: + - description: error (optional) + - description: overflow (optional, only for monitoring) + + interrupt-names: + oneOf: + - items: + - enum: [ error, overflow ] + - items: + - const: error + - const: overflowYeugh. Is this really the only way to say "one or both of foo"?
(I don't know the answer to this -- though I can believe that it's true. Perhaps just not describing this property is another option. Many bindings seem not to bother.)
Quoted 10 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +10
-0+ + arm,not-ready-us: + description: The maximum time in microseconds for monitoring data to be + accurate after a settings change. For more information, see the + Not-Ready (NRDY) bit description in the MPAM specification. + + numa-node-id: true # see NUMA binding + + '#address-cells': + const: 1
Quoted 8 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 56
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ + '#size-cells': + const: 0 + +patternProperties: + '^ris@[0-9a-f]$':It this supposed to be '^ris@[0-9a-f]+$' ?
Looks like yes. Fixed.
Quoted 13 lines
Currently MPAMF_IDR.RIS_MAX is only 4 bits in size and so cannot be greater than
0xf. But it is not inconceivable that a future revision of the architecture might enable more -- and the are 4 RES0 bits looming over theRIS_MAXfield, just waiting to be used...(In any case, it feels wrong to try to enforce numeric bounds with a regex, even in the cases where it happens to work straightforwardly.)
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 60
Quoted 4 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +4
-0+ type: object + additionalProperties: false + description: + RIS nodes for each RIS in an MSC. These nodes are required for each RIS
Quoted 8 lines
The architectural term is "resource instance", not "RIS".
But "RIS nodes" is fine for describing the DT nodes, since we can call them what we like, and "ris" is widely used inside the MPAM driver.
People writing DTs should not need to be familiar with the driver's internal naming conventions, though.
What about the architecture's name for fields?
This number goes in MPAMCFG_PART_SEL.RIS.
Quoted 11 lines
(There are other instances, but I won't comment on them all individually.)
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 66
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ implementing known MPAM controls + + properties: + compatible: + enum: + # Bulk storage for cacheNit: What is "bulk storage"?
Probably to distinguish it from other storage a cache may have, such as tag-ram.
Quoted 1 line
The MPAM spec just refers to "cache" or "cache memory".
I figure these are comments, I'll remove them...
Quoted 11 lines
Quoted 11 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +11
-0+ - arm,mpam-cache + # Memory bandwidth + - arm,mpam-memory + + reg: + minimum: 0 + maximum: 0xf + + cpus: + description: + Phandle(s) to the CPU node(s) this RIS belongs to. By default, the parent
Quoted 8 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 83
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ device's affinity is used. + + arm,mpam-device: + $ref: /schemas/types.yaml#/definitions/phandle + description: + By default, the MPAM enabled device associated with a RIS is the MSC'sAssociated how?
By the phandle this is a description for.
Quoted 1 line
Is this the device where the physical resources managed by the MSC are located?
Yes,
Quoted 23 lines
Quoted 2 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +2
-0+ parent node. It is possible for each RIS to be associated with different + devices in which case 'arm,mpam-device' should be used.[...]
Quoted 5 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +5
-0+examples: + - | + L3: cache-controller@30000000 { + compatible = "arm,dsu-l3-cache", "cache"; + cache-level = <3>;Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 114
Quoted 10 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +10
-0+ cache-unified; + + ranges = <0x0 0x30000000 0x800000>; + #address-cells = <1>; + #size-cells = <1>; + + msc@10000 { + compatible = "arm,mpam-msc"; + + /* CPU affinity implied by parent cache node's */"node's" -> "nodes".
Quoted 2 lines
(or it this supposed to be in the singular -- i.e., the immediately parent cache node only?)
The MSC's parent cache node can be used to find the affinity. I'll make it singular and drop the 's
Quoted 5 lines
Anyway, it looks like this is commenting on the "reg" property, which doesn't seem right.
Is this commnent supposed instead to explain the omission of the "cpus" property? If so, that should be made clearer.
I'll move it to the end of the list of properties so it doesn't look like it belongs to the one below it.
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 120
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ reg = <0x10000 0x2000>; + interrupts = <1>, <2>; + interrupt-names = "error", "overflow"; + arm,not-ready-us = <1>; + }; + };
Thanks,
James
Re: [PATCH 09/33] dt-bindings: arm: Add MPAM MSC binding
Hi,
Quoted 49 lines · James Morse
Hi Dave,
Quoted 9 lines · Dave Martin
Quoted 8 lines · James Morse
From: Rob Herring <[email protected]>
The binding is designed around the assumption that an MSC will be a sub-block of something else such as a memory controller, cache controller, or IOMMU. However, it's certainly possible a design does not have that association or has a mixture of both, so the binding illustrates how we can support that with RIS child nodes.
Quoted 7 lines
Quoted 4 lines
A key part of MPAM is we need to know about all of the MSCs in the system before it can be enabled. This drives the need for the genericish 'arm,mpam-msc' compatible. Though we can't assume an MSC is accessible until a h/w specific driver potentially enables the h/w.
I'll leave detailed review to other people for now, since I'm not so up to speed on all things DT.
Me neither!
Quoted 17 lines
Quoted 1 line
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +0
-0[...]
Quoted 1 line
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +0
-0@@ -0,0 +1,200 @@[...]
Quoted 9 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +9
-0+title: Arm Memory System Resource Partitioning and Monitoring (MPAM) + +description: | + The Arm MPAM specification can be found here: + + https://developer.arm.com/documentation/ddi0598/latest + +maintainers: + - Rob Herring <[email protected]>Quoted 8 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 21
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ +properties: + compatible: + items: + - const: arm,mpam-msc # Further details are discoverable + - const: arm,mpam-memory-controller-mscThere seems to be no clear statement about how these differ.
Quoted 7 lines
It's a more-specific compatible, I think these are usually things like:
| compatible = "acme,mega-cache-9000", "arm,mpam-msc"Where the driver can key errata-workaround on the vendor specific bit when needed.
In this case - I think they're examples, but Rob said they were supposed to be in some other list of compatible. (not sure what/where that is)
I guess I'll defer to the DT folks about how this ought to be presented.
The DT bindings are a weird hybrid of informal and formal that I'm not really used to.
Quoted 13 lines
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 26
Quoted 4 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +4
-0+ reg: + maxItems: 1 + description: A memory region containing registers as defined in the MPAM + specification.There seems to be no handling of PCC-based MSCs here. Should there be?
That is newer than this document. On DT platforms PCC is spelled SCMI, and is discoverable. Andre P prototyped this, (patches in the extras branch) but no-one has come out of the woodwork to say they actually need it yet.
ACPI PCC is a definite maybe.
Quoted 8 lines
Quoted 6 lines
If this can be added later in a backwards-compatible way, I guess that's not a problem (and this is what compatible strings are for, if all else fails.)
An explicit statement that PCC is not supported here might be helpful, though.
I'm pretty sure its discoverable on DT/SCMI platforms.
OK. If this may not be needed, is discoverable and/or can be bolted on in a compatible way later, I guess we wouldn't need to panic about it just now.
(At least we can do that much more easily than promulgating an update to the ACPI tables.)
Quoted 39 lines
Quoted 30 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 40
Quoted 13 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +13
-0+ interrupts: + minItems: 1 + items: + - description: error (optional) + - description: overflow (optional, only for monitoring) + + interrupt-names: + oneOf: + - items: + - enum: [ error, overflow ] + - items: + - const: error + - const: overflowYeugh. Is this really the only way to say "one or both of foo"?
(I don't know the answer to this -- though I can believe that it's true. Perhaps just not describing this property is another option. Many bindings seem not to bother.)
Quoted 10 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +10
-0+ + arm,not-ready-us: + description: The maximum time in microseconds for monitoring data to be + accurate after a settings change. For more information, see the + Not-Ready (NRDY) bit description in the MPAM specification. + + numa-node-id: true # see NUMA binding + + '#address-cells': + const: 1Quoted 8 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 56
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ + '#size-cells': + const: 0 + +patternProperties: + '^ris@[0-9a-f]$':It this supposed to be '^ris@[0-9a-f]+$' ?
Quoted 1 line
Looks like yes. Fixed.
OK
Quoted 19 lines
Quoted 17 lines
Currently MPAMF_IDR.RIS_MAX is only 4 bits in size and so cannot be greater than
0xf. But it is not inconceivable that a future revision of the architecture might enable more -- and the are 4 RES0 bits looming over theRIS_MAXfield, just waiting to be used...(In any case, it feels wrong to try to enforce numeric bounds with a regex, even in the cases where it happens to work straightforwardly.)
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 60
Quoted 4 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +4
-0+ type: object + additionalProperties: false + description: + RIS nodes for each RIS in an MSC. These nodes are required for each RISThe architectural term is "resource instance", not "RIS".
But "RIS nodes" is fine for describing the DT nodes, since we can call them what we like, and "ris" is widely used inside the MPAM driver.
Quoted 5 lines
Quoted 2 lines
People writing DTs should not need to be familiar with the driver's internal naming conventions, though.
What about the architecture's name for fields?
This number goes in MPAMCFG_PART_SEL.RIS.
That's the identifier for the resource instance (= "Resource Instance Selector", see e.g., ARM IHI 0099A.a Section 9.4.14 "MPAMCFG_PART_SEL, MPAM Partition Configuration Selection Register"). The way I read this, the contents of MPAMCFG_PART_SEL.RIS is just a numeric identifier identifier, rather than the thing being identified.
(I guess I am bikeshedding, here. The chance for actual confusion remains low. I just find this use of "RIS" a bit dissonant.)
Quoted 11 lines
Quoted 11 lines
(There are other instances, but I won't comment on them all individually.)
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ implementing known MPAM controls + + properties: + compatible: + enum: + # Bulk storage for cache
Quoted 12 lines
Quoted 1 line
Nit: What is "bulk storage"?
Probably to distinguish it from other storage a cache may have, such as tag-ram.
Quoted 1 line
The MPAM spec just refers to "cache" or "cache memory".
I figure these are comments, I'll remove them...
Quoted 3 lines
Quoted 3 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +3
-0+ - arm,mpam-cache + # Memory bandwidth + - arm,mpam-memory
I think that the meaning of "mpam-cache" is pretty obvious without benefiting from a comment, but "mpam-memory" is not an obvious name for memory _bandwidth_. That probably still wants clarification.
Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +8
-0+ + reg: + minimum: 0 + maximum: 0xf + + cpus: + description: + Phandle(s) to the CPU node(s) this RIS belongs to. By default, the parentQuoted 8 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 83
Quoted 6 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +6
-0+ device's affinity is used. + + arm,mpam-device: + $ref: /schemas/types.yaml#/definitions/phandle + description: + By default, the MPAM enabled device associated with a RIS is the MSC'sAssociated how?
Quoted 6 lines
By the phandle this is a description for.
Quoted 1 line
Is this the device where the physical resources managed by the MSC are located?
Yes,
OK, that's not "associated by the phandle". It's a physical hardware property.
[...]
Quoted 18 lines
Quoted 18 lines
Quoted 5 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +5
-0+examples: + - | + L3: cache-controller@30000000 { + compatible = "arm,dsu-l3-cache", "cache"; + cache-level = <3>;Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml line 114
Quoted 10 lines
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml +10
-0+ cache-unified; + + ranges = <0x0 0x30000000 0x800000>; + #address-cells = <1>; + #size-cells = <1>; + + msc@10000 { + compatible = "arm,mpam-msc"; + + /* CPU affinity implied by parent cache node's */"node's" -> "nodes".
Quoted 5 lines
Quoted 2 lines
(or it this supposed to be in the singular -- i.e., the immediately parent cache node only?)
The MSC's parent cache node can be used to find the affinity. I'll make it singular and drop the 's
OK
Quoted 9 lines
Quoted 5 lines
Anyway, it looks like this is commenting on the "reg" property, which doesn't seem right.
Is this commnent supposed instead to explain the omission of the "cpus" property? If so, that should be made clearer.
I'll move it to the end of the list of properties so it doesn't look like it belongs to the one below it.
Ack, that works.
Cheers
---Dave
James Morse
maintainerPATCH 9/33 dt-bindings: arm: Add MPAM MSC binding
In reply to James MorseFrom: Rob Herring <[email protected]>
The binding is designed around the assumption that an MSC will be a sub-block of something else such as a memory controller, cache controller, or IOMMU. However, it's certainly possible a design does not have that association or has a mixture of both, so the binding illustrates how we can support that with RIS child nodes.
A key part of MPAM is we need to know about all of the MSCs in the system before it can be enabled. This drives the need for the genericish 'arm,mpam-msc' compatible. Though we can't assume an MSC is accessible until a h/w specific driver potentially enables the h/w.
---
Changes since RFC:
* Syntax(?) corrections supplied by Rob.
* Culled some context in the example.
--- .../devicetree/bindings/arm/arm,mpam-msc.yaml | 200 ++++++++++++++++++ 1 file changed, 200 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml
Documentation/devicetree/bindings/arm/arm,mpam-msc.yaml new file +200 -0
@@ -0,0 +1,200 @@ +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/arm/arm,mpam-msc.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Arm Memory System Resource Partitioning and Monitoring (MPAM) + +description: | + The Arm MPAM specification can be found here: + + https://developer.arm.com/documentation/ddi0598/latest + +maintainers: + - Rob Herring <[email protected]> + +properties: + compatible: + items: + - const: arm,mpam-msc # Further details are discoverable + - const: arm,mpam-memory-controller-msc + + reg: + maxItems: 1 + description: A memory region containing registers as defined in the MPAM + specification. + + interrupts: + minItems: 1 + items: + - description: error (optional) + - description: overflow (optional, only for monitoring) + + interrupt-names: + oneOf: + - items: + - enum: [ error, overflow ] + - items: + - const: error + - const: overflow + + arm,not-ready-us: + description: The maximum time in microseconds for monitoring data to be + accurate after a settings change. For more information, see the + Not-Ready (NRDY) bit description in the MPAM specification. + + numa-node-id: true # see NUMA binding + + '#address-cells': + const: 1 + + '#size-cells': + const: 0 + +patternProperties: + '^ris@[0-9a-f]$': + type: object + additionalProperties: false + description: + RIS nodes for each RIS in an MSC. These nodes are required for each RIS + implementing known MPAM controls + + properties: + compatible: + enum: + # Bulk storage for cache + - arm,mpam-cache + # Memory bandwidth + - arm,mpam-memory + + reg: + minimum: 0 + maximum: 0xf + + cpus: + description: + Phandle(s) to the CPU node(s) this RIS belongs to. By default, the parent + device's affinity is used. + + arm,mpam-device: + $ref: /schemas/types.yaml#/definitions/phandle + description: + By default, the MPAM enabled device associated with a RIS is the MSC's + parent node. It is possible for each RIS to be associated with different + devices in which case 'arm,mpam-device' should be used. + + required: + - compatible + - reg + +required: + - compatible + - reg + +dependencies: + interrupts: [ interrupt-names ] + +additionalProperties: false + +examples: + - | + L3: cache-controller@30000000 { + compatible = "arm,dsu-l3-cache", "cache"; + cache-level = <3>; + cache-unified; + + ranges = <0x0 0x30000000 0x800000>; + #address-cells = <1>; + #size-cells = <1>; + + msc@10000 { + compatible = "arm,mpam-msc"; + + /* CPU affinity implied by parent cache node's */ + reg = <0x10000 0x2000>; + interrupts = <1>, <2>; + interrupt-names = "error", "overflow"; + arm,not-ready-us = <1>; + }; + }; + + mem: memory-controller@20000 { + compatible = "foo,a-memory-controller"; + reg = <0x20000 0x1000>; + + #address-cells = <1>; + #size-cells = <1>; + ranges; + + msc@21000 { + compatible = "arm,mpam-memory-controller-msc", "arm,mpam-msc"; + reg = <0x21000 0x1000>; + interrupts = <3>; + interrupt-names = "error"; + arm,not-ready-us = <1>; + numa-node-id = <1>; + }; + }; + + iommu@40000 { + reg = <0x40000 0x1000>; + + ranges; + #address-cells = <1>; + #size-cells = <1>; + + msc@41000 { + compatible = "arm,mpam-msc"; + reg = <0 0x1000>; + interrupts = <5>, <6>; + interrupt-names = "error", "overflow"; + arm,not-ready-us = <1>; + + #address-cells = <1>; + #size-cells = <0>; + + ris@2 { + compatible = "arm,mpam-cache"; + reg = <0>; + // TODO: How to map to device(s)? + }; + }; + }; + + msc@80000 { + compatible = "foo,a-standalone-msc"; + reg = <0x80000 0x1000>; + + clocks = <&clks 123>; + + ranges; + #address-cells = <1>; + #size-cells = <1>; + + msc@10000 { + compatible = "arm,mpam-msc"; + + reg = <0x10000 0x2000>; + interrupts = <7>; + interrupt-names = "overflow"; + arm,not-ready-us = <1>; + + #address-cells = <1>; + #size-cells = <0>; + + ris@0 { + compatible = "arm,mpam-cache"; + reg = <0>; + arm,mpam-device = <&L2_0>; + }; + + ris@1 { + compatible = "arm,mpam-memory"; + reg = <1>; + arm,mpam-device = <&mem>; + }; + }; + }; + +...
Signature · 1 line
2.20.1
James Morse
maintainerPATCH 10/33 arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
In reply to James MorseProbing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for num_partid and num_pmg have been discovered.
Start with driver probe/remove and mapping the MSC.
---
Changes since RFC:
* Check for status=broken DT devices.
* Moved all the files around.
* Made Kconfig symbols depend on EXPERT
--- arch/arm64/Kconfig | 1 + drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/resctrl/Kconfig | 11 ++ drivers/resctrl/Makefile | 4 + drivers/resctrl/mpam_devices.c | 336 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 62 ++++++ 7 files changed, 417 insertions(+) create mode 100644 drivers/resctrl/Kconfig create mode 100644 drivers/resctrl/Makefile create mode 100644 drivers/resctrl/mpam_devices.c create mode 100644 drivers/resctrl/mpam_internal.h
arch/arm64/Kconfig +1 -0
@@ -2062,6 +2062,7 @@ config ARM64_TLB_RANGE config ARM64_MPAM bool "Enable support for MPAM" + select ARM64_MPAM_DRIVER select ACPI_MPAM if ACPI help Memory Partitioning and Monitoring is an optional extension
drivers/Kconfig +2 -0
@@ -251,4 +251,6 @@ source "drivers/hte/Kconfig" source "drivers/cdx/Kconfig" +source "drivers/resctrl/Kconfig" + endmenu
drivers/Makefile +1 -0
@@ -194,5 +194,6 @@ obj-$(CONFIG_HTE) += hte/ obj-$(CONFIG_DRM_ACCEL) += accel/ obj-$(CONFIG_CDX_BUS) += cdx/ obj-$(CONFIG_DPLL) += dpll/ +obj-y += resctrl/ obj-$(CONFIG_S390) += s390/
drivers/resctrl/Kconfig new file +11 -0
@@ -0,0 +1,11 @@ +# Confusingly, this is everything but the CPU bits of MPAM. CPU here means +# CPU resources, not containers or cgroups etc. +config ARM64_MPAM_DRIVER + bool "MPAM driver for System IP, e,g. caches and memory controllers" + depends on ARM64_MPAM && EXPERT + +config ARM64_MPAM_DRIVER_DEBUG + bool "Enable debug messages from the MPAM driver." + depends on ARM64_MPAM_DRIVER + help + Say yes here to enable debug messages from the MPAM driver.
drivers/resctrl/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o +mpam-y += mpam_devices.o + +cflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUG
drivers/resctrl/mpam_devices.c new file +336 -0
336 lines not on this page · raw message
drivers/resctrl/mpam_internal.h new file +62 -0
62 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
…
Quoted 2 lines
+++
b/drivers/resctrl/mpam_devices.c(quoted diff) +0
-0@@ -0,0 +1,336 @@
…
Quoted 2 lines
(quoted diff) +2
-0+static void mpam_msc_drv_remove(struct platform_device *pdev) +{
…
Quoted 2 lines
drivers/resctrl/mpam_devices.c new file +2
-0+mutex_lock (&mpam_list_lock); + mpam_num_msc--;
…
drivers/resctrl/mpam_devices.c line 190
Quoted 3 lines
drivers/resctrl/mpam_devices.c new file +3
-0+devm_kfree (&pdev->dev, msc); +mutex_unlock (&mpam_list_lock); +}
…
Under which circumstances would you become interested to apply a statement like “guard(mutex)(&mpam_list_lock);”?
https://elixir.bootlin.com/linux/v6.17-rc2/source/include/linux/mutex.h#L228
Regards,
Markus
Re: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
In reply to James Morse…
…
Quoted 2 lines
drivers/resctrl/mpam_devices.c new file +2
-0+static int mpam_msc_drv_probe (struct platform_device *pdev) +{
…
Quoted 9 lines
drivers/resctrl/mpam_devices.c new file +9
-0+ }while (0 ); +mutex_unlock (&mpam_list_lock); + +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled) + err =acpi_mpam_parse_resources (msc, plat_data); +else + err =mpam_dt_parse_resources (msc, plat_data);
drivers/resctrl/mpam_devices.c line 280
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ } + +if (!err && fw_num_msc == mpam_num_msc) +mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err; +}
…
* Would you like to integrate anything from the following source code variant?
if (!err)
/* Create RIS entries described by firmware */
err = acpi_disabled
? mpam_dt_parse_resources(msc, plat_data)
: acpi_mpam_parse_resources(msc, plat_data);
if (err) {
if (msc)
mpam_msc_drv_remove(pdev);
} else {
if (fw_num_msc == mpam_num_msc)
mpam_discovery_complete();
}* How do you think about to increase the application of scope-based resource management at further places?
Regards,
Markus
Greg Kroah-Hartman
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
Quoted 42 lines · Markus Elfring
…
…Quoted 2 lines
drivers/resctrl/mpam_devices.c new file +2
-0+static int mpam_msc_drv_probe (struct platform_device *pdev) +{…
Quoted 9 lines
drivers/resctrl/mpam_devices.c new file +9
-0+ }while (0 ); +mutex_unlock (&mpam_list_lock); + +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled) + err =acpi_mpam_parse_resources (msc, plat_data); +else + err =mpam_dt_parse_resources (msc, plat_data);drivers/resctrl/mpam_devices.c line 280
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ } + +if (!err && fw_num_msc == mpam_num_msc) +mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err; +}…
* Would you like to integrate anything from the following source code variant?
if (!err) /* Create RIS entries described by firmware */ err = acpi_disabled ? mpam_dt_parse_resources(msc, plat_data) : acpi_mpam_parse_resources(msc, plat_data); if (err) { if (msc) mpam_msc_drv_remove(pdev); } else { if (fw_num_msc == mpam_num_msc) mpam_discovery_complete(); }
Quoted 6 lines
* How do you think about to increase the application of scope-based resource management at further places?
Regards,
Markus
Hi,
This is the semi-friendly patch-bot of Greg Kroah-Hartman.
Markus, you seem to have sent a nonsensical or otherwise pointless review comment to a patch submission on a Linux kernel developer mailing list. I strongly suggest that you not do this anymore. Please do not bother developers who are actively working to produce patches and features with comments that, in the end, are a waste of time.
Patch submitter, please ignore Markus's suggestion; you do not need to follow it at all. The person/bot/AI that sent it is being ignored by almost all Linux kernel maintainers for having a persistent pattern of behavior of producing distracting and pointless commentary, and inability to adapt to feedback. Please feel free to also ignore emails from them.
thanks,
greg k-h's patch email bot
Ben Horgan
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
In reply to James MorseHi James,
Quoted 161 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.Start with driver probe/remove and mapping the MSC.
CC Carl Worth <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* Check for status=broken DT devices.
* Moved all the files around.
* MadeKconfigsymbols depend on EXPERT--- arch/arm64/Kconfig | 1 + drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/resctrl/Kconfig | 11 ++ drivers/resctrl/Makefile | 4 + drivers/resctrl/mpam_devices.c | 336 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 62 ++++++ 7 files changed, 417 insertions(+) create mode 100644 drivers/resctrl/Kconfig create mode 100644 drivers/resctrl/Makefile create mode 100644 drivers/resctrl/mpam_devices.c create mode 100644 drivers/resctrl/mpam_internal.harch/arm64/Kconfig +1
-0@@ -2062,6 +2062,7 @@ config ARM64_TLB_RANGE config ARM64_MPAM bool "Enable support for MPAM" + select ARM64_MPAM_DRIVER select ACPI_MPAM if ACPI help Memory Partitioning and Monitoring is an optional extensiondrivers/Kconfig +2
-0@@ -251,4 +251,6 @@ source "drivers/hte/Kconfig" source "drivers/cdx/Kconfig" +source "drivers/resctrl/Kconfig" + endmenudrivers/Makefile +1
-0@@ -194,5 +194,6 @@ obj-$(CONFIG_HTE) += hte/ obj-$(CONFIG_DRM_ACCEL) += accel/ obj-$(CONFIG_CDX_BUS) += cdx/ obj-$(CONFIG_DPLL) += dpll/ +obj-y += resctrl/ obj-$(CONFIG_S390) += s390/drivers/resctrl/Kconfig new file +11
-0@@ -0,0 +1,11 @@ +# Confusingly, this is everything but the CPU bits of MPAM. CPU here means +# CPU resources, not containers or cgroups etc. +config ARM64_MPAM_DRIVER + bool "MPAM driver for System IP, e,g. caches and memory controllers" + depends on ARM64_MPAM && EXPERT + +config ARM64_MPAM_DRIVER_DEBUG + bool "Enable debug messages from the MPAM driver." + depends on ARM64_MPAM_DRIVER + help + Say yes here to enable debug messages from the MPAM driver.drivers/resctrl/Makefile new file +4
-0@@ -0,0 +1,4 @@ +obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o +mpam-y += mpam_devices.o + +cflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUGdrivers/resctrl/mpam_devices.c new file +65
-0@@ -0,0 +1,336 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2025 Arm Ltd. + +#define pr_fmt (fmt) "%s:%s: " fmt, KBUILD_MODNAME, __func__ + +#include <linux/acpi.h> +#include <linux/arm_mpam.h> +#include <linux/cacheinfo.h> +#include <linux/cpu.h> +#include <linux/cpumask.h> +#include <linux/device.h> +#include <linux/errno.h> +#include <linux/gfp.h> +#include <linux/list.h> +#include <linux/lockdep.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/of_platform.h> +#include <linux/platform_device.h> +#include <linux/printk.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/srcu.h> +#include <linux/types.h> + +#include <acpi/pcc.h> + +#include "mpam_internal.h" + +/* +* mpam_list_lock protects the SRCU lists when writing. Once the +* mpam_enabled key is enabled these lists are read-only, +* unless the error interrupt disables the driver. +*/ +static DEFINE_MUTEX (mpam_list_lock); +static LIST_HEAD (mpam_all_msc); + +static struct srcu_struct mpam_srcu; + +/* MPAM isn't available until all the MSC have been probed. */ +static u32 mpam_num_msc; + +static void mpam_discovery_complete (void ) +{ +pr_err ("Discovered all MSC\n" ); +} + +static int mpam_dt_count_msc (void ) +{ +int count =0 ; +struct device_node *np; + +for_each_compatible_node (np,NULL ,"arm,mpam-msc" ) { +if (of_device_is_available (np)) + count++; + } + +return count; +} + +static int mpam_dt_parse_resource (struct mpam_msc *msc,struct device_node *np, +u32 ris_idx) +{ +int err =0 ; +u32 level =0 ;
drivers/resctrl/mpam_devices.c line 75
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+unsigned long cache_id; +struct device_node *cache; + + do { + if (of_device_is_compatible(np, "arm,mpam-cache")) { + cache = of_parse_phandle(np, "arm,mpam-device", 0); + if (!cache) { + pr_err("Failed to read phandle\n"); + break; + }
This looks like this allows "arm,mpam-cache" and "arm,mpam-device" to be used on an msc node when there are no ris children. This usage could be reasonable but doesn't match the schema in the previous patch. Should this usage be rejected or the schema extended?
Quoted 172 lines
drivers/resctrl/mpam_devices.c new file +172
-0+ }else if (of_device_is_compatible (np->parent,"cache" )) { + cache =of_node_get (np->parent); + }else { +/* For now, only caches are supported */ + cache =NULL ; +break ; + } + + err =of_property_read_u32 (cache,"cache-level" , &level); +if (err) { +pr_err ("Failed to read cache-level\n" ); +break ; + } + + cache_id =cache_of_calculate_id (cache); +if (cache_id == ~0UL ) { + err = -ENOENT ; +break ; + } + + err =mpam_ris_create (msc, ris_idx,MPAM_CLASS_CACHE , level, + cache_id); + }while (0 ); +of_node_put (cache); + +return err; +} + +static int mpam_dt_parse_resources (struct mpam_msc *msc,void *ignored) +{ +int err, num_ris =0 ; +const u32 *ris_idx_p; +struct device_node *iter, *np; + + np = msc->pdev->dev.of_node; +for_each_child_of_node (np, iter) { + ris_idx_p =of_get_property (iter,"reg" ,NULL ); +if (ris_idx_p) { + num_ris++; + err =mpam_dt_parse_resource (msc, iter, *ris_idx_p); +if (err) { +of_node_put (iter); +return err; + } + } + } + +if (!num_ris) +mpam_dt_parse_resource (msc, np,0 ); + +return err; +} + +/* +* An MSC can control traffic from a set of CPUs, but may only be accessible +* from a (hopefully wider) set of CPUs. The common reason for this is power +* management. If all the CPUs in a cluster are in PSCI:CPU_SUSPEND, the +* the corresponding cache may also be powered off. By making accesses from +* one of those CPUs, we ensure this isn't the case. +*/ +static int update_msc_accessibility (struct mpam_msc *msc) +{ +struct device_node *parent; +u32 affinity_id; +int err; + +if (!acpi_disabled) { + err =device_property_read_u32 (&msc->pdev->dev,"cpu_affinity" , + &affinity_id); +if (err) +cpumask_copy (&msc->accessibility, cpu_possible_mask); +else +acpi_pptt_get_cpus_from_container (affinity_id, + &msc->accessibility); + +return 0 ; + } + +/* This depends on the path to of_node */ + parent =of_get_parent (msc->pdev->dev.of_node); +if (parent == of_root) { +cpumask_copy (&msc->accessibility, cpu_possible_mask); + err =0 ; + }else { + err = -EINVAL ; +pr_err ("Cannot determine accessibility of MSC: %s\n" , +dev_name (&msc->pdev->dev)); + } +of_node_put (parent); + +return err; +} + +static int fw_num_msc; + +static void mpam_pcc_rx_callback (struct mbox_client *cl,void *msg) +{ +/* TODO: wake up tasks blocked on this MSC's PCC channel */ +} + +static void mpam_msc_drv_remove (struct platform_device *pdev) +{ +struct mpam_msc *msc =platform_get_drvdata (pdev); + +if (!msc) +return ; + +mutex_lock (&mpam_list_lock); + mpam_num_msc--; +platform_set_drvdata (pdev,NULL ); +list_del_rcu (&msc->glbl_list); +synchronize_srcu (&mpam_srcu); +devm_kfree (&pdev->dev, msc); +mutex_unlock (&mpam_list_lock); +} + +static int mpam_msc_drv_probe (struct platform_device *pdev) +{ +int err; +struct mpam_msc *msc; +struct resource *msc_res; +void *plat_data = pdev->dev.platform_data; + +mutex_lock (&mpam_list_lock); + do { + msc = devm_kzalloc(&pdev->dev, sizeof(*msc), GFP_KERNEL); + if (!msc) { + err = -ENOMEM; + break; + } + + mutex_init(&msc->probe_lock); + mutex_init(&msc->part_sel_lock); + mutex_init(&msc->outer_mon_sel_lock); + raw_spin_lock_init(&msc->inner_mon_sel_lock); + msc->id = mpam_num_msc++; + msc->pdev = pdev; + INIT_LIST_HEAD_RCU(&msc->glbl_list); + INIT_LIST_HEAD_RCU(&msc->ris); + + err = update_msc_accessibility(msc); + if (err) + break; + if (cpumask_empty(&msc->accessibility)) { + pr_err_once("msc:%u is not accessible from any CPU!", + msc->id); + err = -EINVAL; + break; + } + + if (device_property_read_u32(&pdev->dev, "pcc-channel", + &msc->pcc_subspace_id)) + msc->iface = MPAM_IFACE_MMIO; + else + msc->iface = MPAM_IFACE_PCC; + + if (msc->iface == MPAM_IFACE_MMIO) { + void __iomem *io; + + io = devm_platform_get_and_ioremap_resource(pdev, 0, + &msc_res); + if (IS_ERR(io)) { + pr_err("Failed to map MSC base address\n"); + err = PTR_ERR(io); + break; + } + msc->mapped_hwpage_sz = msc_res->end - msc_res->start; + msc->mapped_hwpage = io; + } else if (msc->iface == MPAM_IFACE_PCC) { + msc->pcc_cl.dev = &pdev->dev; + msc->pcc_cl.rx_callback = mpam_pcc_rx_callback; + msc->pcc_cl.tx_block = false;
drivers/resctrl/mpam_devices.c line 257
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ msc->pcc_cl.tx_tout =1000 ;/* 1s */ + msc->pcc_cl.knows_txdone = false; + + msc->pcc_chan =pcc_mbox_request_channel (&msc->pcc_cl, + msc->pcc_subspace_id); +if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); + err =PTR_ERR (msc->pcc_chan); +break ; + }
I don't see pcc support added in this series. Should we fail the probe if this interface is specified?
(If keeping, there is a missing pcc_mbox_free_channel() on the error path.)
Quoted 106 lines
drivers/resctrl/mpam_devices.c new file +79
-0+ } + +list_add_rcu (&msc->glbl_list, &mpam_all_msc); +platform_set_drvdata (pdev, msc); + }while (0 ); +mutex_unlock (&mpam_list_lock); + +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled) + err =acpi_mpam_parse_resources (msc, plat_data); +else + err =mpam_dt_parse_resources (msc, plat_data); + } + +if (!err && fw_num_msc == mpam_num_msc) +mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err; +} + +static const struct of_device_id mpam_of_match[] = { + { .compatible ="arm,mpam-msc" , }, + {}, +}; +MODULE_DEVICE_TABLE (of, mpam_of_match); + +static struct platform_driver mpam_msc_driver = { + .driver = { + .name ="mpam_msc" , + .of_match_table =of_match_ptr (mpam_of_match), + }, + .probe = mpam_msc_drv_probe, + .remove = mpam_msc_drv_remove, +}; + +/* +* MSC that are hidden under caches are not created as platform devices +* as there is no cache driver. Caches are also special-cased in +* update_msc_accessibility(). +*/ +static void mpam_dt_create_foundling_msc (void ) +{ +int err; +struct device_node *cache; + +for_each_compatible_node (cache,NULL ,"cache" ) { + err =of_platform_populate (cache, mpam_of_match,NULL ,NULL ); +if (err) +pr_err ("Failed to create MSC devices under caches\n" ); + } +} + +static int __init mpam_msc_driver_init (void ) +{ +if (!system_supports_mpam ()) +return -EOPNOTSUPP ; + +init_srcu_struct (&mpam_srcu); + +if (!acpi_disabled) + fw_num_msc =acpi_mpam_count_msc (); +else + fw_num_msc =mpam_dt_count_msc (); + +if (fw_num_msc <=0 ) { +pr_err ("No MSC devices found in firmware\n" ); +return -EINVAL ; + } + +if (acpi_disabled) +mpam_dt_create_foundling_msc (); + +return platform_driver_register (&mpam_msc_driver); +} +subsys_initcall (mpam_msc_driver_init);drivers/resctrl/mpam_internal.h new file +21
-0@@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +// Copyright (C) 2024 Arm Ltd. + +#ifndef MPAM_INTERNAL_H +#define MPAM_INTERNAL_H + +#include <linux/arm_mpam.h> +#include <linux/cpumask.h> +#include <linux/io.h> +#include <linux/mailbox_client.h> +#include <linux/mutex.h> +#include <linux/resctrl.h> +#include <linux/sizes.h> + +struct mpam_msc { + /* member of mpam_all_msc */ + struct list_head glbl_list; + + int id; + struct platform_device *pdev; +
drivers/resctrl/mpam_internal.h line 31
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+/* Not modified after mpam_is_enabled() becomes true */ +enum mpam_msc_iface iface; +u32 pcc_subspace_id; +struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +u32 nrdy_usec; +cpumask_t accessibility; + +/* +* probe_lock is only take during discovery. After discovery these
nit: s/take/taken/
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+* properties become read-only and the lists are protected by SRCU. +*/ +struct mutex probe_lock; +unsigned long ris_idxs[128 /BITS_PER_LONG ]; +u32 ris_max; + +/* mpam_msc_ris of this component */ +struct list_head ris; + +/*
drivers/resctrl/mpam_internal.h line 51
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+* part_sel_lock protects access to the MSC hardware registers that are +* affected by MPAMCFG_PART_SEL. (including the ID registers that vary +* by RIS). +* If needed, take msc->lock first. +*/ +struct mutex part_sel_lock; + +/* +* mon_sel_lock protects access to the MSC hardware registers that are +* affeted by MPAMCFG_MON_SEL.
nit: s/affeted/affected/
drivers/resctrl/mpam_internal.h line 62
Quoted 11 lines
drivers/resctrl/mpam_internal.h new file +11
-0+* If needed, take msc->lock first. +*/ +struct mutex outer_mon_sel_lock; +raw_spinlock_t inner_mon_sel_lock; +unsigned long inner_mon_sel_flags; + +void __iomem *mapped_hwpage; + size_t mapped_hwpage_sz; +}; + +#endif /* MPAM_INTERNAL_H */
Thanks,
Ben
James Morse
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
Hi Ben,
Quoted 27 lines · Ben Horgan
Quoted 26 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.Start with driver probe/remove and mapping the MSC.
drivers/resctrl/mpam_devices.c new file +11
-0@@ -0,0 +1,336 @@ +static int mpam_dt_parse_resource (struct mpam_msc *msc,struct device_node *np, +u32 ris_idx) +{ +int err =0 ; +u32 level =0 ; +unsigned long cache_id; +struct device_node *cache; + + do { + if (of_device_is_compatible(np, "arm,mpam-cache")) { + cache = of_parse_phandle(np, "arm,mpam-device", 0);
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 75
Quoted 4 lines
drivers/resctrl/mpam_devices.c new file +4
-0+if (!cache) { +pr_err ("Failed to read phandle\n" ); +break ; + }This looks like this allows "arm,mpam-cache" and "arm,mpam-device" to be used on an msc node when there are no ris children. This usage could be reasonable but doesn't match the schema in the previous patch. Should this usage be rejected or the schema extended?
The DT/ACPI stuff is only going to describe the things that make sense at a high level, e.g. the controls for the L3. There may be other controls for stuff that doesn't make sense in the hardware - these get discovered, grouped as 'unknown' and left alone.
Another angle on this is where there is an MSC that the OS will never make use of, but needs to know about to find the system wide minimum value. (there is a comment about this in the ACPI spec...)
I don't think its a problem if the magic dt-binding machinery is overly restrictive, that is about validating DTB files...
Quoted 88 lines
Quoted 88 lines
drivers/resctrl/mpam_devices.c new file +87
-0+ }else if (of_device_is_compatible (np->parent,"cache" )) { + cache =of_node_get (np->parent); + }else { +/* For now, only caches are supported */ + cache =NULL ; +break ; + } + + err =of_property_read_u32 (cache,"cache-level" , &level); +if (err) { +pr_err ("Failed to read cache-level\n" ); +break ; + } + + cache_id =cache_of_calculate_id (cache); +if (cache_id == ~0UL ) { + err = -ENOENT ; +break ; + } + + err =mpam_ris_create (msc, ris_idx,MPAM_CLASS_CACHE , level, + cache_id); + }while (0 ); +of_node_put (cache); + +return err; +} +static int mpam_msc_drv_probe (struct platform_device *pdev) +{ +int err; +struct mpam_msc *msc; +struct resource *msc_res; +void *plat_data = pdev->dev.platform_data; + +mutex_lock (&mpam_list_lock); + do { + msc = devm_kzalloc(&pdev->dev, sizeof(*msc), GFP_KERNEL); + if (!msc) { + err = -ENOMEM; + break; + } + + mutex_init(&msc->probe_lock); + mutex_init(&msc->part_sel_lock); + mutex_init(&msc->outer_mon_sel_lock); + raw_spin_lock_init(&msc->inner_mon_sel_lock); + msc->id = mpam_num_msc++; + msc->pdev = pdev; + INIT_LIST_HEAD_RCU(&msc->glbl_list); + INIT_LIST_HEAD_RCU(&msc->ris); + + err = update_msc_accessibility(msc); + if (err) + break; + if (cpumask_empty(&msc->accessibility)) { + pr_err_once("msc:%u is not accessible from any CPU!", + msc->id); + err = -EINVAL; + break; + } + + if (device_property_read_u32(&pdev->dev, "pcc-channel", + &msc->pcc_subspace_id)) + msc->iface = MPAM_IFACE_MMIO; + else + msc->iface = MPAM_IFACE_PCC; + + if (msc->iface == MPAM_IFACE_MMIO) { + void __iomem *io; + + io = devm_platform_get_and_ioremap_resource(pdev, 0, + &msc_res); + if (IS_ERR(io)) { + pr_err("Failed to map MSC base address\n"); + err = PTR_ERR(io); + break; + } + msc->mapped_hwpage_sz = msc_res->end - msc_res->start; + msc->mapped_hwpage = io; + } else if (msc->iface == MPAM_IFACE_PCC) { + msc->pcc_cl.dev = &pdev->dev; + msc->pcc_cl.rx_callback = mpam_pcc_rx_callback; + msc->pcc_cl.tx_block = false; + msc->pcc_cl.tx_tout = 1000; /* 1s */ + msc->pcc_cl.knows_txdone = false; + + msc->pcc_chan = pcc_mbox_request_channel(&msc->pcc_cl,
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 257
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +6
-0+ msc->pcc_subspace_id); +if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); + err =PTR_ERR (msc->pcc_chan); +break ; + }I don't see pcc support added in this series. Should we fail the probe if this interface is specified?
I've got patches from Andre P to support it on DT - but the platforms that need it keeping popping in and out of existence. I'll pull these bits out - they were intended to check the ACPI table wasn't totally rotten...
Quoted 1 line
(If keeping, there is a missing
pcc_mbox_free_channel()on the error path.)
When pcc_mbox_request_channel() fails? It already called mbox_free_channel() itself.
Quoted 41 lines
Quoted 41 lines
drivers/resctrl/mpam_devices.c new file +23
-0+ } + +list_add_rcu (&msc->glbl_list, &mpam_all_msc); +platform_set_drvdata (pdev, msc); + }while (0 ); +mutex_unlock (&mpam_list_lock); + +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled) + err =acpi_mpam_parse_resources (msc, plat_data); +else + err =mpam_dt_parse_resources (msc, plat_data); + } + +if (!err && fw_num_msc == mpam_num_msc) +mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err; +}drivers/resctrl/mpam_internal.h new file +10
-0@@ -0,0 +1,62 @@ +struct mpam_msc { +/* member of mpam_all_msc */ +struct list_head glbl_list; + +int id; +struct platform_device *pdev; + +/* Not modified after mpam_is_enabled() becomes true */ +enum mpam_msc_iface iface; +u32 pcc_subspace_id;
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 31
Quoted 7 lines
drivers/resctrl/mpam_internal.h new file +7
-0+struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +u32 nrdy_usec; +cpumask_t accessibility; + +/* +* probe_lock is only take during discovery. After discovery these nit: s/take/taken/
Fixed,
Quoted 13 lines
Quoted 13 lines
drivers/resctrl/mpam_internal.h new file +13
-0+* properties become read-only and the lists are protected by SRCU. +*/ +struct mutex probe_lock; +unsigned long ris_idxs[128 /BITS_PER_LONG ]; +u32 ris_max; + +/* mpam_msc_ris of this component */ +struct list_head ris; + +/* +* part_sel_lock protects access to the MSC hardware registers that are +* affected by MPAMCFG_PART_SEL. (including the ID registers that vary +* by RIS).
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 51
Quoted 7 lines
drivers/resctrl/mpam_internal.h new file +7
-0+* If needed, take msc->lock first. +*/ +struct mutex part_sel_lock; + +/* +* mon_sel_lock protects access to the MSC hardware registers that are +* affeted by MPAMCFG_MON_SEL. nit: s/affeted/affected/
Fixed,
Quoted 11 lines
drivers/resctrl/mpam_internal.h line 62
Quoted 11 lines
drivers/resctrl/mpam_internal.h new file +11
-0+* If needed, take msc->lock first. +*/ +struct mutex outer_mon_sel_lock; +raw_spinlock_t inner_mon_sel_lock; +unsigned long inner_mon_sel_flags; + +void __iomem *mapped_hwpage; + size_t mapped_hwpage_sz; +}; + +#endif /* MPAM_INTERNAL_H */
Thanks,
James
Ben Horgan
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
Hi James,
Quoted 43 lines · James Morse
Hi Ben,
Quoted 27 lines · Ben Horgan
Quoted 26 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.Start with driver probe/remove and mapping the MSC.
drivers/resctrl/mpam_devices.c new file +11
-0@@ -0,0 +1,336 @@ +static int mpam_dt_parse_resource (struct mpam_msc *msc,struct device_node *np, +u32 ris_idx) +{ +int err =0 ; +u32 level =0 ; +unsigned long cache_id; +struct device_node *cache; + + do { + if (of_device_is_compatible(np, "arm,mpam-cache")) { + cache = of_parse_phandle(np, "arm,mpam-device", 0);Quoted 8 lines
drivers/resctrl/mpam_devices.c line 75
Quoted 4 lines
drivers/resctrl/mpam_devices.c new file +4
-0+if (!cache) { +pr_err ("Failed to read phandle\n" ); +break ; + }This looks like this allows "arm,mpam-cache" and "arm,mpam-device" to be used on an msc node when there are no ris children. This usage could be reasonable but doesn't match the schema in the previous patch. Should this usage be rejected or the schema extended?
The DT/ACPI stuff is only going to describe the things that make sense at a high level, e.g. the controls for the L3. There may be other controls for stuff that doesn't make sense in the hardware - these get discovered, grouped as 'unknown' and left alone.
Quoted 6 lines
Another angle on this is where there is an MSC that the OS will never make use of, but needs to know about to find the system wide minimum value. (there is a comment about this in the ACPI spec...)
I don't think its a problem if the magic dt-binding machinery is overly restrictive, that is about validating DTB files...
I agree with your points. However, I was rather thinking that the code allows more ways to describe the same thing than the schema does. In that, you could write something like:
msc@80000 {
compatible = "foo,a-standalone-msc";
reg = <0x80000 0x1000>;
...
msc@10000 {
compatible = "arm,mpam-msc arm,mpam-cache";
arm,mpam-device = <&mem>;
...
}
}Although, now I've written this out, it doesn't seem sensible to worry about this. Using ris compatibles on an msc, as in my example, is clearly an error.
Quoted 2 lines
[snip]
Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
drivers/resctrl/mpam_devices.c new file +8
-0+ }else if (msc->iface ==MPAM_IFACE_PCC ) { + msc->pcc_cl.dev = &pdev->dev; + msc->pcc_cl.rx_callback = mpam_pcc_rx_callback; + msc->pcc_cl.tx_block = false; + msc->pcc_cl.tx_tout =1000 ;/* 1s */ + msc->pcc_cl.knows_txdone = false; + + msc->pcc_chan = pcc_mbox_request_channel(&msc->pcc_cl,Quoted 8 lines
drivers/resctrl/mpam_devices.c line 257
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +6
-0+ msc->pcc_subspace_id); +if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); + err =PTR_ERR (msc->pcc_chan); +break ; + }I don't see pcc support added in this series. Should we fail the probe if this interface is specified?
Quoted 8 lines
I've got patches from Andre P to support it on DT - but the platforms that need it keeping popping in and out of existence. I'll pull these bits out - they were intended to check the ACPI table wasn't totally rotten...
Quoted 1 line
(If keeping, there is a missing
pcc_mbox_free_channel()on the error path.)When
pcc_mbox_request_channel()fails? It already calledmbox_free_channel()itself.
Apologies, this was relating to if the *_parse_resources() call below failed.
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
drivers/resctrl/mpam_devices.c new file +15
-0+ } + +list_add_rcu (&msc->glbl_list, &mpam_all_msc); +platform_set_drvdata (pdev, msc); + }while (0 ); +mutex_unlock (&mpam_list_lock); + +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled) + err =acpi_mpam_parse_resources (msc, plat_data); +else + err =mpam_dt_parse_resources (msc, plat_data); + } +
Quoted 8 lines
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 280
Quoted 8 lines
drivers/resctrl/mpam_devices.c new file +8
-0+if (!err && fw_num_msc == mpam_num_msc) +mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err; +}
[snip]>
Quoted 3 lines
Thanks,
James
Thanks,
Ben
Rob Herring
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
In reply to James MorseQuoted 7 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.
Quoted 7 lines
Start with driver probe/remove and mapping the MSC.
CC Carl Worth <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* Check for status=broken DT devices.
No such status... 'disabled' can be for a variety of reasons.
Quoted 83 lines
* Moved all the files around.
* MadeKconfigsymbols depend on EXPERT--- arch/arm64/Kconfig | 1 + drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/resctrl/Kconfig | 11 ++ drivers/resctrl/Makefile | 4 + drivers/resctrl/mpam_devices.c | 336 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 62 ++++++ 7 files changed, 417 insertions(+) create mode 100644 drivers/resctrl/Kconfig create mode 100644 drivers/resctrl/Makefile create mode 100644 drivers/resctrl/mpam_devices.c create mode 100644 drivers/resctrl/mpam_internal.harch/arm64/Kconfig +1
-0@@ -2062,6 +2062,7 @@ config ARM64_TLB_RANGE config ARM64_MPAM bool "Enable support for MPAM" + select ARM64_MPAM_DRIVER select ACPI_MPAM if ACPI help Memory Partitioning and Monitoring is an optional extensiondrivers/Kconfig +2
-0@@ -251,4 +251,6 @@ source "drivers/hte/Kconfig" source "drivers/cdx/Kconfig" +source "drivers/resctrl/Kconfig" + endmenudrivers/Makefile +1
-0@@ -194,5 +194,6 @@ obj-$(CONFIG_HTE) += hte/ obj-$(CONFIG_DRM_ACCEL) += accel/ obj-$(CONFIG_CDX_BUS) += cdx/ obj-$(CONFIG_DPLL) += dpll/ +obj-y += resctrl/ obj-$(CONFIG_S390) += s390/drivers/resctrl/Kconfig new file +11
-0@@ -0,0 +1,11 @@ +# Confusingly, this is everything but the CPU bits of MPAM. CPU here means +# CPU resources, not containers or cgroups etc. +config ARM64_MPAM_DRIVER + bool "MPAM driver for System IP, e,g. caches and memory controllers" + depends on ARM64_MPAM && EXPERT + +config ARM64_MPAM_DRIVER_DEBUG + bool "Enable debug messages from the MPAM driver." + depends on ARM64_MPAM_DRIVER + help + Say yes here to enable debug messages from the MPAM driver.drivers/resctrl/Makefile new file +4
-0@@ -0,0 +1,4 @@ +obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o +mpam-y += mpam_devices.o + +cflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUGdrivers/resctrl/mpam_devices.c new file +0
-0
drivers/resctrl/mpam_devices.c line 2
Quoted 2 lines
drivers/resctrl/mpam_devices.c new file +2
-0@@ -0,0 +1,336 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2025 Arm Ltd.
Given the 2024 below, should this be 2024-2025?
Quoted 33 lines
drivers/resctrl/mpam_devices.c new file +33
-0+ +#define pr_fmt (fmt) "%s:%s: " fmt, KBUILD_MODNAME, __func__ + +#include <linux/acpi.h> +#include <linux/arm_mpam.h> +#include <linux/cacheinfo.h> +#include <linux/cpu.h> +#include <linux/cpumask.h> +#include <linux/device.h> +#include <linux/errno.h> +#include <linux/gfp.h> +#include <linux/list.h> +#include <linux/lockdep.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/of_platform.h> +#include <linux/platform_device.h> +#include <linux/printk.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/srcu.h> +#include <linux/types.h> + +#include <acpi/pcc.h> + +#include "mpam_internal.h" + +/* +* mpam_list_lock protects the SRCU lists when writing. Once the +* mpam_enabled key is enabled these lists are read-only, +* unless the error interrupt disables the driver. +*/ +static DEFINE_MUTEX (mpam_list_lock);
drivers/resctrl/mpam_devices.c line 45
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+static LIST_HEAD (mpam_all_msc); + +static struct srcu_struct mpam_srcu; + +/* MPAM isn't available until all the MSC have been probed. */ +static u32 mpam_num_msc; + +static void mpam_discovery_complete (void ) +{ +pr_err ("Discovered all MSC\n" );
Perhaps print out how many MSCs.
Quoted 21 lines
drivers/resctrl/mpam_devices.c new file +21
-0+} + +static int mpam_dt_count_msc (void ) +{ +int count =0 ; +struct device_node *np; + +for_each_compatible_node (np,NULL ,"arm,mpam-msc" ) { +if (of_device_is_available (np)) + count++; + } + +return count; +} + +static int mpam_dt_parse_resource (struct mpam_msc *msc,struct device_node *np, +u32 ris_idx) +{ +int err =0 ; +u32 level =0 ; +unsigned long cache_id;
drivers/resctrl/mpam_devices.c line 76
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+struct device_node *cache; + + do { + if (of_device_is_compatible(np, "arm,mpam-cache")) { + cache = of_parse_phandle(np, "arm,mpam-device", 0); + if (!cache) { + pr_err("Failed to read phandle\n"); + break; + } + } else if (of_device_is_compatible(np->parent, "cache")) {
Don't access device_node members. I'm trying to make it opaque. And technically it can be racy to access parent ptr when/if nodes are dynamic. I think this should suffice:
else {
cache = of_get_parent(np);
if (!of_device_is_compatible(cache, "cache")) {
cache = NULL;
break; }
}
Quoted 25 lines
drivers/resctrl/mpam_devices.c new file +25
-0+ cache =of_node_get (np->parent); + }else { +/* For now, only caches are supported */ + cache =NULL ; +break ; + } + + err =of_property_read_u32 (cache,"cache-level" , &level); +if (err) { +pr_err ("Failed to read cache-level\n" ); +break ; + } + + cache_id =cache_of_calculate_id (cache); +if (cache_id == ~0UL ) { + err = -ENOENT ; +break ; + } + + err =mpam_ris_create (msc, ris_idx,MPAM_CLASS_CACHE , level, + cache_id); + }while (0 ); +of_node_put (cache); + +return err;
drivers/resctrl/mpam_devices.c line 111
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+} + +static int mpam_dt_parse_resources (struct mpam_msc *msc,void *ignored) +{ +int err, num_ris =0 ; +const u32 *ris_idx_p; +struct device_node *iter, *np; + + np = msc->pdev->dev.of_node; +for_each_child_of_node (np, iter) {
Use for_each_available_child_of_node_scoped()
drivers/resctrl/mpam_devices.c line 112
Quoted 1 line
drivers/resctrl/mpam_devices.c new file +1
-0+ ris_idx_p =of_get_property (iter,"reg" ,NULL );
This is broken on big endian and new users of of_get_property() are discouraged. Use of_property_read_reg().
drivers/resctrl/mpam_devices.c line 117
Quoted 5 lines
drivers/resctrl/mpam_devices.c new file +5
-0+if (ris_idx_p) { + num_ris++; + err =mpam_dt_parse_resource (msc, iter, *ris_idx_p); +if (err) { +of_node_put (iter);
And then drop the put.
Quoted 27 lines
drivers/resctrl/mpam_devices.c new file +27
-0+return err; + } + } + } + +if (!num_ris) +mpam_dt_parse_resource (msc, np,0 ); + +return err; +} + +/* +* An MSC can control traffic from a set of CPUs, but may only be accessible +* from a (hopefully wider) set of CPUs. The common reason for this is power +* management. If all the CPUs in a cluster are in PSCI:CPU_SUSPEND, the +* the corresponding cache may also be powered off. By making accesses from +* one of those CPUs, we ensure this isn't the case. +*/ +static int update_msc_accessibility (struct mpam_msc *msc) +{ +struct device_node *parent; +u32 affinity_id; +int err; + +if (!acpi_disabled) { + err =device_property_read_u32 (&msc->pdev->dev,"cpu_affinity" , + &affinity_id);
drivers/resctrl/mpam_devices.c line 154
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+if (err) +cpumask_copy (&msc->accessibility, cpu_possible_mask); +else +acpi_pptt_get_cpus_from_container (affinity_id, + &msc->accessibility); + +return 0 ; + } + +/* This depends on the path to of_node */
I'm failing to understand what has to be at the root node?
Quoted 24 lines
drivers/resctrl/mpam_devices.c new file +24
-0+ parent =of_get_parent (msc->pdev->dev.of_node); +if (parent == of_root) { +cpumask_copy (&msc->accessibility, cpu_possible_mask); + err =0 ; + }else { + err = -EINVAL ; +pr_err ("Cannot determine accessibility of MSC: %s\n" , +dev_name (&msc->pdev->dev)); + } +of_node_put (parent); + +return err; +} + +static int fw_num_msc; + +static void mpam_pcc_rx_callback (struct mbox_client *cl,void *msg) +{ +/* TODO: wake up tasks blocked on this MSC's PCC channel */ +} + +static void mpam_msc_drv_remove (struct platform_device *pdev) +{ +struct mpam_msc *msc =platform_get_drvdata (pdev);
drivers/resctrl/mpam_devices.c line 188
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ +if (!msc) +return ; + +mutex_lock (&mpam_list_lock); + mpam_num_msc--; +platform_set_drvdata (pdev,NULL ); +list_del_rcu (&msc->glbl_list); +synchronize_srcu (&mpam_srcu); +devm_kfree (&pdev->dev, msc);
This should happen automagically.
Quoted 13 lines
drivers/resctrl/mpam_devices.c new file +13
-0+mutex_unlock (&mpam_list_lock); +} + +static int mpam_msc_drv_probe (struct platform_device *pdev) +{ +int err; +struct mpam_msc *msc; +struct resource *msc_res; +void *plat_data = pdev->dev.platform_data; + +mutex_lock (&mpam_list_lock); + do { + msc = devm_kzalloc(&pdev->dev, sizeof(*msc), GFP_KERNEL);
drivers/resctrl/mpam_devices.c line 211
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+if (!msc) { + err = -ENOMEM ; +break ; + } + +mutex_init (&msc->probe_lock); +mutex_init (&msc->part_sel_lock); +mutex_init (&msc->outer_mon_sel_lock); +raw_spin_lock_init (&msc->inner_mon_sel_lock); + msc->id = mpam_num_msc++;
Multiple probe functions can run in parallel, so this needs to be atomic. Maybe it is with mpam_list_lock, but then the name of the mutex is misleading given this is not the list. It's not really clear to me what all needs the mutex here. Certainly a lot of it doesn't. Like everything else above here except the increment.
Quoted 5 lines
drivers/resctrl/mpam_devices.c new file +5
-0+ msc->pdev = pdev; +INIT_LIST_HEAD_RCU (&msc->glbl_list); +INIT_LIST_HEAD_RCU (&msc->ris); + + err =update_msc_accessibility (msc);
drivers/resctrl/mpam_devices.c line 226
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+if (err) +break ; +if (cpumask_empty (&msc->accessibility)) { +pr_err_once ("msc:%u is not accessible from any CPU!" , + msc->id); + err = -EINVAL ; +break ; + } + + if (device_property_read_u32(&pdev->dev, "pcc-channel",
Does this property apply to DT? It would as the code is written. It is not documented though.
Quoted 34 lines
drivers/resctrl/mpam_devices.c new file +34
-0+ &msc->pcc_subspace_id)) + msc->iface =MPAM_IFACE_MMIO ; + else + msc->iface = MPAM_IFACE_PCC; + +if (msc->iface ==MPAM_IFACE_MMIO ) { +void __iomem *io; + + io =devm_platform_get_and_ioremap_resource (pdev,0 , + &msc_res); +if (IS_ERR (io)) { +pr_err ("Failed to map MSC base address\n" ); + err =PTR_ERR (io); +break ; + } + msc->mapped_hwpage_sz = msc_res->end - msc_res->start; + msc->mapped_hwpage = io; + }else if (msc->iface ==MPAM_IFACE_PCC ) { + msc->pcc_cl.dev = &pdev->dev; + msc->pcc_cl.rx_callback = mpam_pcc_rx_callback; + msc->pcc_cl.tx_block = false; + msc->pcc_cl.tx_tout =1000 ;/* 1s */ + msc->pcc_cl.knows_txdone = false; + + msc->pcc_chan =pcc_mbox_request_channel (&msc->pcc_cl, + msc->pcc_subspace_id); +if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); + err =PTR_ERR (msc->pcc_chan); +break ; + } + } + +list_add_rcu (&msc->glbl_list, &mpam_all_msc);
drivers/resctrl/mpam_devices.c line 270
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+platform_set_drvdata (pdev, msc); + }while (0 ); +mutex_unlock (&mpam_list_lock); + +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled) + err =acpi_mpam_parse_resources (msc, plat_data); +else + err =mpam_dt_parse_resources (msc, plat_data);
Isn't there a race here if an error occurs since you already added the MSC to the list? Something like this sequence with 2 MSCs:
device 1 probe
device 1 added
device 2 probe
device 2 added
device 1 calls mpam_discovery_complete()
device 2 error on parse_resources
device 2 removed
Quoted 28 lines
drivers/resctrl/mpam_devices.c new file +28
-0+ } + +if (!err && fw_num_msc == mpam_num_msc) +mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err; +} + +static const struct of_device_id mpam_of_match[] = { + { .compatible ="arm,mpam-msc" , }, + {}, +}; +MODULE_DEVICE_TABLE (of, mpam_of_match); + +static struct platform_driver mpam_msc_driver = { + .driver = { + .name ="mpam_msc" , + .of_match_table =of_match_ptr (mpam_of_match), + }, + .probe = mpam_msc_drv_probe, + .remove = mpam_msc_drv_remove, +}; + +/* +* MSC that are hidden under caches are not created as platform devices
drivers/resctrl/mpam_devices.c line 308
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+* as there is no cache driver. Caches are also special-cased in +* update_msc_accessibility(). +*/ +static void mpam_dt_create_foundling_msc (void ) +{ +int err; +struct device_node *cache; + +for_each_compatible_node (cache,NULL ,"cache" ) { + err =of_platform_populate (cache, mpam_of_match,NULL ,NULL );
This is going to create platform devices for all caches (except L1) regardless of whether they support MPAM or not. Isn't it likely or possible that only L3 or SLC caches support MPAM?
Quoted 34 lines
drivers/resctrl/mpam_devices.c new file +28
-0+if (err) +pr_err ("Failed to create MSC devices under caches\n" ); + } +} + +static int __init mpam_msc_driver_init (void ) +{ +if (!system_supports_mpam ()) +return -EOPNOTSUPP ; + +init_srcu_struct (&mpam_srcu); + +if (!acpi_disabled) + fw_num_msc =acpi_mpam_count_msc (); +else + fw_num_msc =mpam_dt_count_msc (); + +if (fw_num_msc <=0 ) { +pr_err ("No MSC devices found in firmware\n" ); +return -EINVAL ; + } + +if (acpi_disabled) +mpam_dt_create_foundling_msc (); + +return platform_driver_register (&mpam_msc_driver); +} +subsys_initcall (mpam_msc_driver_init);drivers/resctrl/mpam_internal.h new file +0
-0
drivers/resctrl/mpam_internal.h line 2
Quoted 2 lines
drivers/resctrl/mpam_internal.h new file +2
-0@@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +// Copyright (C) 2024 Arm Ltd.
It's 2025.
Quoted 19 lines
drivers/resctrl/mpam_internal.h new file +19
-0+ +#ifndef MPAM_INTERNAL_H +#define MPAM_INTERNAL_H + +#include <linux/arm_mpam.h> +#include <linux/cpumask.h> +#include <linux/io.h> +#include <linux/mailbox_client.h> +#include <linux/mutex.h> +#include <linux/resctrl.h> +#include <linux/sizes.h> + +struct mpam_msc { + /* member of mpam_all_msc */ + struct list_head glbl_list; + + int id; + struct platform_device *pdev; +
drivers/resctrl/mpam_internal.h line 31
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+/* Not modified after mpam_is_enabled() becomes true */ +enum mpam_msc_iface iface; +u32 pcc_subspace_id; +struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +u32 nrdy_usec; +cpumask_t accessibility; + +/* +* probe_lock is only take during discovery. After discovery these
s/take/taken/
Quoted 4 lines
drivers/resctrl/mpam_internal.h new file +4
-0+* properties become read-only and the lists are protected by SRCU. +*/ +struct mutex probe_lock; +unsigned long ris_idxs[128 /BITS_PER_LONG ];
drivers/resctrl/mpam_internal.h line 45
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+u32 ris_max; + +/* mpam_msc_ris of this component */ +struct list_head ris; + +/* +* part_sel_lock protects access to the MSC hardware registers that are +* affected by MPAMCFG_PART_SEL. (including the ID registers that vary +* by RIS). +* If needed, take msc->lock first.
Stale comment? I don't see any 'lock' member.
Quoted 20 lines
drivers/resctrl/mpam_internal.h new file +17
-1+*/ +struct mutex part_sel_lock; + +/* +* mon_sel_lock protects access to the MSC hardware registers that are +* affeted by MPAMCFG_MON_SEL. +* If needed, take msc->lock first. +*/ +struct mutex outer_mon_sel_lock; +raw_spinlock_t inner_mon_sel_lock; +unsigned long inner_mon_sel_flags; + +void __iomem *mapped_hwpage; + size_t mapped_hwpage_sz; +}; + +#endif /* MPAM_INTERNAL_H */ --2.20.1
Rob Herring
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
Quoted 8 lines · Rob Herring
Quoted 6 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.
[...]
Quoted 11 lines
drivers/resctrl/mpam_devices.c line 111
Quoted 8 lines
drivers/resctrl/mpam_devices.c new file +8
-0+static int mpam_dt_parse_resources (struct mpam_msc *msc,void *ignored) +{ +int err, num_ris =0 ; +const u32 *ris_idx_p; +struct device_node *iter, *np; + + np = msc->pdev->dev.of_node; +for_each_child_of_node (np, iter) {Use
for_each_available_child_of_node_scoped()
Quoted 4 lines
Quoted 1 line
+
ris_idx_p=of_get_property(iter, "reg", NULL);This is broken on big endian and new users of
of_get_property()are discouraged. Useof_property_read_reg().
Err, this is broken on little endian as the DT is big endian.
So this was obviously not tested as I'm confident you didn't test on BE.
Rob
James Morse
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
Hi Rob,
Quoted 23 lines · Rob Herring
Quoted 8 lines · Rob Herring
Quoted 6 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.[...]
Quoted 11 lines
drivers/resctrl/mpam_devices.c line 111
Quoted 8 lines
drivers/resctrl/mpam_devices.c new file +8
-0+static int mpam_dt_parse_resources (struct mpam_msc *msc,void *ignored) +{ +int err, num_ris =0 ; +const u32 *ris_idx_p; +struct device_node *iter, *np; + + np = msc->pdev->dev.of_node; +for_each_child_of_node (np, iter) {Use
for_each_available_child_of_node_scoped()
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
+
ris_idx_p=of_get_property(iter, "reg", NULL);This is broken on big endian and new users of
of_get_property()are discouraged. Useof_property_read_reg().Err, this is broken on little endian as the DT is big endian.
So this was obviously not tested as I'm confident you didn't test on BE.
'not tested' is shades of grey. I fed the FVP ~6 different DTB files to hit the different paths through the driver. The FVP only has controls under RIS-0, so all of those only defined RIS-0, and unsurprisingly didn't notice the helper isn't endian safe.
Thanks,
James
James Morse
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
In reply to Rob HerringHi Rob,
Quoted 11 lines · Rob Herring
Quoted 8 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.Start with driver probe/remove and mapping the MSC.
Quoted 10 lines
drivers/resctrl/mpam_devices.c line 2
Quoted 8 lines
drivers/resctrl/mpam_devices.c new file +2
-0@@ -0,0 +1,336 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2025 Arm Ltd. Given the 2024 below, should this be 2024-2025?
I've never known what this year is really for. People clearly expect it to be this year ... I've evidently missed one somewhere. I'll fix that. ... I wrote some of this code in 2018, so there are a range of options on what it 'should' be ...
Quoted 37 lines
Quoted 37 lines
drivers/resctrl/mpam_devices.c new file +37
-0+ +#define pr_fmt (fmt) "%s:%s: " fmt, KBUILD_MODNAME, __func__ + +#include <linux/acpi.h> +#include <linux/arm_mpam.h> +#include <linux/cacheinfo.h> +#include <linux/cpu.h> +#include <linux/cpumask.h> +#include <linux/device.h> +#include <linux/errno.h> +#include <linux/gfp.h> +#include <linux/list.h> +#include <linux/lockdep.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/of_platform.h> +#include <linux/platform_device.h> +#include <linux/printk.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/srcu.h> +#include <linux/types.h> + +#include <acpi/pcc.h> + +#include "mpam_internal.h" + +/* +* mpam_list_lock protects the SRCU lists when writing. Once the +* mpam_enabled key is enabled these lists are read-only, +* unless the error interrupt disables the driver. +*/ +static DEFINE_MUTEX (mpam_list_lock); +static LIST_HEAD (mpam_all_msc); + +static struct srcu_struct mpam_srcu; +
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 45
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +6
-0+/* MPAM isn't available until all the MSC have been probed. */ +static u32 mpam_num_msc; + +static void mpam_discovery_complete (void ) +{ +pr_err ("Discovered all MSC\n" );Perhaps print out how many MSCs.
Once the whole thing is assembled the mpam_enable() call prints the number of PARTID/PMG, which is something user-space can do something with. I don't think the number of MSC is useful to anyone as some of them are grouped together and can't be configured independently.
Quoted 23 lines
Quoted 8 lines
drivers/resctrl/mpam_devices.c new file +7
-0+} +static int mpam_dt_parse_resource (struct mpam_msc *msc,struct device_node *np, +u32 ris_idx) +{ +int err =0 ; +u32 level =0 ; +unsigned long cache_id;drivers/resctrl/mpam_devices.c line 76
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+struct device_node *cache; + + do { + if (of_device_is_compatible(np, "arm,mpam-cache")) { + cache = of_parse_phandle(np, "arm,mpam-device", 0); + if (!cache) { + pr_err("Failed to read phandle\n"); + break; + } + } else if (of_device_is_compatible(np->parent, "cache")) {Don't access
device_nodemembers. I'm trying to make it opaque. And technically it can be racy to access parent ptr when/if nodes are dynamic. I think this should suffice:
Quoted 7 lines
else {
cache =of_get_parent(np);if (!of_device_is_compatible(cache, "cache")) { cache = NULL; break;}
}
Thanks!
The if/else-if ladder needs to stay, I've grabbed the parent earlier, and added a of_node_put() of it after the do/while.
Quoted 29 lines
Quoted 29 lines
drivers/resctrl/mpam_devices.c new file +29
-0+ cache =of_node_get (np->parent); + }else { +/* For now, only caches are supported */ + cache =NULL ; +break ; + } + + err =of_property_read_u32 (cache,"cache-level" , &level); +if (err) { +pr_err ("Failed to read cache-level\n" ); +break ; + } + + cache_id =cache_of_calculate_id (cache); +if (cache_id == ~0UL ) { + err = -ENOENT ; +break ; + } + + err =mpam_ris_create (msc, ris_idx,MPAM_CLASS_CACHE , level, + cache_id); + }while (0 ); +of_node_put (cache); + +return err; +} + +static int mpam_dt_parse_resources (struct mpam_msc *msc,void *ignored) +{
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 111
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +6
-0+int err, num_ris =0 ; +const u32 *ris_idx_p; +struct device_node *iter, *np; + + np = msc->pdev->dev.of_node; +for_each_child_of_node (np, iter) {Use
for_each_available_child_of_node_scoped()
Sure,
Quoted 4 lines
Quoted 1 line
+
ris_idx_p=of_get_property(iter, "reg", NULL);This is broken on big endian and new users of
of_get_property()are discouraged. Useof_property_read_reg().
Done,
Quoted 40 lines
drivers/resctrl/mpam_devices.c line 117
Quoted 5 lines
drivers/resctrl/mpam_devices.c new file +5
-0+if (ris_idx_p) { + num_ris++; + err =mpam_dt_parse_resource (msc, iter, *ris_idx_p); +if (err) { +of_node_put (iter);And then drop the put.
Quoted 32 lines
drivers/resctrl/mpam_devices.c new file +30
-0+return err; + } + } + } + +if (!num_ris) +mpam_dt_parse_resource (msc, np,0 ); + +return err; +} +/* +* An MSC can control traffic from a set of CPUs, but may only be accessible +* from a (hopefully wider) set of CPUs. The common reason for this is power +* management. If all the CPUs in a cluster are in PSCI:CPU_SUSPEND, the +* the corresponding cache may also be powered off. By making accesses from +* one of those CPUs, we ensure this isn't the case. +*/ +static int update_msc_accessibility (struct mpam_msc *msc) +{ +struct device_node *parent; +u32 affinity_id; +int err; + +if (!acpi_disabled) { + err =device_property_read_u32 (&msc->pdev->dev,"cpu_affinity" , + &affinity_id); +if (err) +cpumask_copy (&msc->accessibility, cpu_possible_mask); +else + acpi_pptt_get_cpus_from_container(affinity_id,
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 154
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +6
-0+ &msc->accessibility); + +return 0 ; + } + +/* This depends on the path to of_node */ I'm failing to understand what has to be at the root node?
The accessibility bitmap. If the MSC is at the root of the tree - its assumed to be global. If its buried in a device, its assumed to be in the same power domain as that device. Practically this only matters for caches where PSCI's CPU_SUSPEND can turn the cache off, so the cache MSC can only be accessed from the CPU's local to it, that way we know its not about to be turned off by PSCI.
I'll rephrase the comment - its trying to explain why its not explicitly encoded. | /* Where an MSC can be accessed from depends on the path to of_node. */
Quoted 50 lines
Quoted 16 lines
drivers/resctrl/mpam_devices.c new file +16
-0+ parent =of_get_parent (msc->pdev->dev.of_node); +if (parent == of_root) { +cpumask_copy (&msc->accessibility, cpu_possible_mask); + err =0 ; + }else { + err = -EINVAL ; +pr_err ("Cannot determine accessibility of MSC: %s\n" , +dev_name (&msc->pdev->dev)); + } +of_node_put (parent); + +return err; +} +static void mpam_msc_drv_remove (struct platform_device *pdev) +{ +struct mpam_msc *msc =platform_get_drvdata (pdev);drivers/resctrl/mpam_devices.c line 188
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ +if (!msc) +return ; + +mutex_lock (&mpam_list_lock); + mpam_num_msc--; +platform_set_drvdata (pdev,NULL ); +list_del_rcu (&msc->glbl_list); +synchronize_srcu (&mpam_srcu); +devm_kfree (&pdev->dev, msc);This should happen automagically.
Quoted 21 lines
drivers/resctrl/mpam_devices.c new file +21
-0+mutex_unlock (&mpam_list_lock); +} + +static int mpam_msc_drv_probe (struct platform_device *pdev) +{ +int err; +struct mpam_msc *msc; +struct resource *msc_res; +void *plat_data = pdev->dev.platform_data; + +mutex_lock (&mpam_list_lock); + do { + msc = devm_kzalloc(&pdev->dev, sizeof(*msc), GFP_KERNEL); + if (!msc) { + err = -ENOMEM; + break; + } + + mutex_init(&msc->probe_lock); + mutex_init(&msc->part_sel_lock); + mutex_init(&msc->outer_mon_sel_lock);
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 211
Quoted 2 lines
drivers/resctrl/mpam_devices.c new file +2
-0+raw_spin_lock_init (&msc->inner_mon_sel_lock); + msc->id = mpam_num_msc++;Multiple probe functions can run in parallel, so this needs to be atomic. Maybe it is with
mpam_list_lock, but then the name of the mutex is misleading given this is not the list. It's not really clear to me what all needs the mutex here. Certainly a lot of it doesn't. Like everything else above here except the increment.
It's more about the class/component/device lists that get added later - but more on this mpam_num_msc thing below...
Quoted 10 lines
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ msc->pdev = pdev; +INIT_LIST_HEAD_RCU (&msc->glbl_list); +INIT_LIST_HEAD_RCU (&msc->ris); + + err =update_msc_accessibility (msc); +if (err) +break ; +if (cpumask_empty (&msc->accessibility)) { +pr_err_once ("msc:%u is not accessible from any CPU!" , + msc->id);
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 226
Quoted 5 lines
drivers/resctrl/mpam_devices.c new file +5
-0+ err = -EINVAL ; +break ; + } + + if (device_property_read_u32(&pdev->dev, "pcc-channel",Does this property apply to DT? It would as the code is written. It is not documented though.
I don't think so - on DT PCC is going to be spelled SCMI which somes with some kind of discovery instead. This property is added by the ACPI table 'driver'.
Quoted 48 lines
Quoted 34 lines
drivers/resctrl/mpam_devices.c new file +34
-0+ &msc->pcc_subspace_id)) + msc->iface =MPAM_IFACE_MMIO ; + else + msc->iface = MPAM_IFACE_PCC; + +if (msc->iface ==MPAM_IFACE_MMIO ) { +void __iomem *io; + + io =devm_platform_get_and_ioremap_resource (pdev,0 , + &msc_res); +if (IS_ERR (io)) { +pr_err ("Failed to map MSC base address\n" ); + err =PTR_ERR (io); +break ; + } + msc->mapped_hwpage_sz = msc_res->end - msc_res->start; + msc->mapped_hwpage = io; + }else if (msc->iface ==MPAM_IFACE_PCC ) { + msc->pcc_cl.dev = &pdev->dev; + msc->pcc_cl.rx_callback = mpam_pcc_rx_callback; + msc->pcc_cl.tx_block = false; + msc->pcc_cl.tx_tout =1000 ;/* 1s */ + msc->pcc_cl.knows_txdone = false; + + msc->pcc_chan =pcc_mbox_request_channel (&msc->pcc_cl, + msc->pcc_subspace_id); +if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); + err =PTR_ERR (msc->pcc_chan); +break ; + } + } + +list_add_rcu (&msc->glbl_list, &mpam_all_msc);drivers/resctrl/mpam_devices.c line 270
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+platform_set_drvdata (pdev, msc); + }while (0 ); +mutex_unlock (&mpam_list_lock); + +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled) + err =acpi_mpam_parse_resources (msc, plat_data); +else + err =mpam_dt_parse_resources (msc, plat_data);Isn't there a race here if an error occurs since you already added the MSC to the list? Something like this sequence with 2 MSCs:
Quoted 7 lines
device 1 probe
device 1 added
device 2 probe
device 2 added
device 1 callsmpam_discovery_complete()
device 2 error onparse_resources
device 2 removed
By the time the whole thing is assembled the 'discovery complete' work is scheduled by a cpuhp callback and has better protection against this. That discovery-complete call is just to help illustrate that there is stuff that happens once all the MSC have been discovered, as that code gets much more complicated later in teh series.
Combined with your comment about the msc_id increment above - I'll stop using that as both a count and an id, re-using the pdev->id as its more likely to be stable from boot to boot. mpam_num_msc can then become an atomic_t that is incremented once we know we're not going to remove the MSC from the list.
Quoted 18 lines
Quoted 18 lines
drivers/resctrl/mpam_devices.c new file +18
-0+ } + +if (!err && fw_num_msc == mpam_num_msc) +mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err; +} +/* +* MSC that are hidden under caches are not created as platform devices +* as there is no cache driver. Caches are also special-cased in +* update_msc_accessibility(). +*/ +static void mpam_dt_create_foundling_msc (void ) +{ +int err;
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 308
Quoted 4 lines
drivers/resctrl/mpam_devices.c new file +4
-0+struct device_node *cache; + +for_each_compatible_node (cache,NULL ,"cache" ) { + err =of_platform_populate (cache, mpam_of_match,NULL ,NULL );This is going to create platform devices for all caches (except L1) regardless of whether they support MPAM or not. Isn't it likely or possible that only L3 or SLC caches support MPAM?
Likely - but all things are possible. You could put an MPAM MSC in your L1-I cache. (or even in the CPU - but lets not go there)
I'll attempt to fix that up, what I have doesn't quite work, but I'll keep picking at it:
| for_each_compatible_node(cache, NULL, "cache") {
| struct device_node *cache_device;
|
| if (of_node_check_flag(cache, OF_POPULATED))
| continue;
|
| cache_device = of_find_matching_node_and_match(cache, mpam_of_match, NULL);
| if (!cache_device)
| continue;
| of_node_put(cache_device);
|
| pdev = of_platform_device_create(cache, "cache", NULL);
| if (!pdev)
| pr_err_once("Failed to create MSC devices under caches\n");
| }Quoted 6 lines
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +4
-0+if (err) +pr_err ("Failed to create MSC devices under caches\n" ); + } +}drivers/resctrl/mpam_internal.h new file +0
-0
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 2
Quoted 6 lines
index 000000000000..07e0f240eaca
drivers/resctrl/mpam_internal.h new file +2
-0@@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +// Copyright (C) 2024 Arm Ltd. It's 2025.
... and I wrote this in 2018, but missed the annual update. I'll fix it.
Quoted 12 lines
Quoted 12 lines
drivers/resctrl/mpam_internal.h new file +12
-0+ +struct mpam_msc { +/* member of mpam_all_msc */ +struct list_head glbl_list; + +int id; +struct platform_device *pdev; + +/* Not modified after mpam_is_enabled() becomes true */ +enum mpam_msc_iface iface; +u32 pcc_subspace_id; +struct mbox_client pcc_cl;
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 31
Quoted 6 lines
drivers/resctrl/mpam_internal.h new file +6
-0+struct pcc_mbox_chan *pcc_chan; +u32 nrdy_usec; +cpumask_t accessibility; + +/* +* probe_lock is only take during discovery. After discovery these s/take/taken/
Fixed,
Quoted 8 lines
Quoted 8 lines
drivers/resctrl/mpam_internal.h new file +8
-0+* properties become read-only and the lists are protected by SRCU. +*/ +struct mutex probe_lock; +unsigned long ris_idxs[128 /BITS_PER_LONG ]; +u32 ris_max; + +/* mpam_msc_ris of this component */ +struct list_head ris;
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 45
Quoted 6 lines
drivers/resctrl/mpam_internal.h new file +6
-0+ +/* +* part_sel_lock protects access to the MSC hardware registers that are +* affected by MPAMCFG_PART_SEL. (including the ID registers that vary +* by RIS). +* If needed, take msc->lock first. Stale comment? I don't see any 'lock' member.
Yes, it should say probe_lock .. renamed after more locks got added in here, and the comment got fixed up in the wrong patch. Fixed.
Quoted 12 lines
Quoted 12 lines
drivers/resctrl/mpam_internal.h new file +12
-0+*/ +struct mutex part_sel_lock; + +/* +* mon_sel_lock protects access to the MSC hardware registers that are +* affeted by MPAMCFG_MON_SEL. +* If needed, take msc->lock first. +*/ +struct mutex outer_mon_sel_lock; +raw_spinlock_t inner_mon_sel_lock; +unsigned long inner_mon_sel_flags; +
Quoted 8 lines
Quoted 8 lines
drivers/resctrl/mpam_internal.h new file +5
-1+void __iomem *mapped_hwpage; + size_t mapped_hwpage_sz; +}; + +#endif /* MPAM_INTERNAL_H */ --2.20.1
Thanks,
James
Ben Horgan
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
In reply to James MorseHi James,
Quoted 463 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.Start with driver probe/remove and mapping the MSC.
CC Carl Worth <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* Check for status=broken DT devices.
* Moved all the files around.
* MadeKconfigsymbols depend on EXPERT--- arch/arm64/Kconfig | 1 + drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/resctrl/Kconfig | 11 ++ drivers/resctrl/Makefile | 4 + drivers/resctrl/mpam_devices.c | 336 ++++++++++++++++++++++++++++++++ drivers/resctrl/mpam_internal.h | 62 ++++++ 7 files changed, 417 insertions(+) create mode 100644 drivers/resctrl/Kconfig create mode 100644 drivers/resctrl/Makefile create mode 100644 drivers/resctrl/mpam_devices.c create mode 100644 drivers/resctrl/mpam_internal.harch/arm64/Kconfig +1
-0@@ -2062,6 +2062,7 @@ config ARM64_TLB_RANGE config ARM64_MPAM bool "Enable support for MPAM" + select ARM64_MPAM_DRIVER select ACPI_MPAM if ACPI help Memory Partitioning and Monitoring is an optional extensiondrivers/Kconfig +2
-0@@ -251,4 +251,6 @@ source "drivers/hte/Kconfig" source "drivers/cdx/Kconfig" +source "drivers/resctrl/Kconfig" + endmenudrivers/Makefile +1
-0@@ -194,5 +194,6 @@ obj-$(CONFIG_HTE) += hte/ obj-$(CONFIG_DRM_ACCEL) += accel/ obj-$(CONFIG_CDX_BUS) += cdx/ obj-$(CONFIG_DPLL) += dpll/ +obj-y += resctrl/ obj-$(CONFIG_S390) += s390/drivers/resctrl/Kconfig new file +11
-0@@ -0,0 +1,11 @@ +# Confusingly, this is everything but the CPU bits of MPAM. CPU here means +# CPU resources, not containers or cgroups etc. +config ARM64_MPAM_DRIVER + bool "MPAM driver for System IP, e,g. caches and memory controllers" + depends on ARM64_MPAM && EXPERT + +config ARM64_MPAM_DRIVER_DEBUG + bool "Enable debug messages from the MPAM driver." + depends on ARM64_MPAM_DRIVER + help + Say yes here to enable debug messages from the MPAM driver.drivers/resctrl/Makefile new file +4
-0@@ -0,0 +1,4 @@ +obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o +mpam-y += mpam_devices.o + +cflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUGdrivers/resctrl/mpam_devices.c new file +336
-0@@ -0,0 +1,336 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2025 Arm Ltd. + +#define pr_fmt (fmt) "%s:%s: " fmt, KBUILD_MODNAME, __func__ + +#include <linux/acpi.h> +#include <linux/arm_mpam.h> +#include <linux/cacheinfo.h> +#include <linux/cpu.h> +#include <linux/cpumask.h> +#include <linux/device.h> +#include <linux/errno.h> +#include <linux/gfp.h> +#include <linux/list.h> +#include <linux/lockdep.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/of_platform.h> +#include <linux/platform_device.h> +#include <linux/printk.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/srcu.h> +#include <linux/types.h> + +#include <acpi/pcc.h> + +#include "mpam_internal.h" + +/* +* mpam_list_lock protects the SRCU lists when writing. Once the +* mpam_enabled key is enabled these lists are read-only, +* unless the error interrupt disables the driver. +*/ +static DEFINE_MUTEX (mpam_list_lock); +static LIST_HEAD (mpam_all_msc); + +static struct srcu_struct mpam_srcu; + +/* MPAM isn't available until all the MSC have been probed. */ +static u32 mpam_num_msc; + +static void mpam_discovery_complete (void ) +{ +pr_err ("Discovered all MSC\n" ); +} + +static int mpam_dt_count_msc (void ) +{ +int count =0 ; +struct device_node *np; + +for_each_compatible_node (np,NULL ,"arm,mpam-msc" ) { +if (of_device_is_available (np)) + count++; + } + +return count; +} + +static int mpam_dt_parse_resource (struct mpam_msc *msc,struct device_node *np, +u32 ris_idx) +{ +int err =0 ; +u32 level =0 ; +unsigned long cache_id; +struct device_node *cache; + +do { +if (of_device_is_compatible (np,"arm,mpam-cache" )) { + cache =of_parse_phandle (np,"arm,mpam-device" ,0 ); +if (!cache) { +pr_err ("Failed to read phandle\n" ); +break ; + } + }else if (of_device_is_compatible (np->parent,"cache" )) { + cache =of_node_get (np->parent); + }else { +/* For now, only caches are supported */ + cache =NULL ; +break ; + } + + err =of_property_read_u32 (cache,"cache-level" , &level); +if (err) { +pr_err ("Failed to read cache-level\n" ); +break ; + } + + cache_id =cache_of_calculate_id (cache); +if (cache_id == ~0UL ) { + err = -ENOENT ; +break ; + } + + err =mpam_ris_create (msc, ris_idx,MPAM_CLASS_CACHE , level, + cache_id); + }while (0 ); +of_node_put (cache); + +return err; +} + +static int mpam_dt_parse_resources (struct mpam_msc *msc,void *ignored) +{ +int err, num_ris =0 ; +const u32 *ris_idx_p; +struct device_node *iter, *np; + + np = msc->pdev->dev.of_node; +for_each_child_of_node (np, iter) { + ris_idx_p =of_get_property (iter,"reg" ,NULL ); +if (ris_idx_p) { + num_ris++; + err =mpam_dt_parse_resource (msc, iter, *ris_idx_p); +if (err) { +of_node_put (iter); +return err; + } + } + } + +if (!num_ris) +mpam_dt_parse_resource (msc, np,0 ); + +return err; +} + +/* +* An MSC can control traffic from a set of CPUs, but may only be accessible +* from a (hopefully wider) set of CPUs. The common reason for this is power +* management. If all the CPUs in a cluster are in PSCI:CPU_SUSPEND, the +* the corresponding cache may also be powered off. By making accesses from +* one of those CPUs, we ensure this isn't the case. +*/ +static int update_msc_accessibility (struct mpam_msc *msc) +{ +struct device_node *parent; +u32 affinity_id; +int err; + +if (!acpi_disabled) { + err =device_property_read_u32 (&msc->pdev->dev,"cpu_affinity" , + &affinity_id); +if (err) +cpumask_copy (&msc->accessibility, cpu_possible_mask); +else +acpi_pptt_get_cpus_from_container (affinity_id, + &msc->accessibility); + +return 0 ; + } + +/* This depends on the path to of_node */ + parent =of_get_parent (msc->pdev->dev.of_node); +if (parent == of_root) { +cpumask_copy (&msc->accessibility, cpu_possible_mask); + err =0 ; + }else { + err = -EINVAL ; +pr_err ("Cannot determine accessibility of MSC: %s\n" , +dev_name (&msc->pdev->dev)); + } +of_node_put (parent); + +return err; +} + +static int fw_num_msc; + +static void mpam_pcc_rx_callback (struct mbox_client *cl,void *msg) +{ +/* TODO: wake up tasks blocked on this MSC's PCC channel */ +} + +static void mpam_msc_drv_remove (struct platform_device *pdev) +{ +struct mpam_msc *msc =platform_get_drvdata (pdev); + +if (!msc) +return ; + +mutex_lock (&mpam_list_lock); + mpam_num_msc--; +platform_set_drvdata (pdev,NULL ); +list_del_rcu (&msc->glbl_list); +synchronize_srcu (&mpam_srcu); +devm_kfree (&pdev->dev, msc); +mutex_unlock (&mpam_list_lock); +} + +static int mpam_msc_drv_probe (struct platform_device *pdev) +{ +int err; +struct mpam_msc *msc; +struct resource *msc_res; +void *plat_data = pdev->dev.platform_data; + +mutex_lock (&mpam_list_lock); +do { + msc =devm_kzalloc (&pdev->dev,sizeof (*msc),GFP_KERNEL ); +if (!msc) { + err = -ENOMEM ; +break ; + } + +mutex_init (&msc->probe_lock); +mutex_init (&msc->part_sel_lock); +mutex_init (&msc->outer_mon_sel_lock); +raw_spin_lock_init (&msc->inner_mon_sel_lock); + msc->id = mpam_num_msc++; + msc->pdev = pdev; +INIT_LIST_HEAD_RCU (&msc->glbl_list); +INIT_LIST_HEAD_RCU (&msc->ris); + + err =update_msc_accessibility (msc); +if (err) +break ; +if (cpumask_empty (&msc->accessibility)) { +pr_err_once ("msc:%u is not accessible from any CPU!" , + msc->id); + err = -EINVAL ; +break ; + } + +if (device_property_read_u32 (&pdev->dev,"pcc-channel" , + &msc->pcc_subspace_id)) + msc->iface =MPAM_IFACE_MMIO ; +else + msc->iface =MPAM_IFACE_PCC ; + +if (msc->iface ==MPAM_IFACE_MMIO ) { +void __iomem *io; + + io =devm_platform_get_and_ioremap_resource (pdev,0 , + &msc_res); +if (IS_ERR (io)) { +pr_err ("Failed to map MSC base address\n" ); + err =PTR_ERR (io); +break ; + } + msc->mapped_hwpage_sz = msc_res->end - msc_res->start; + msc->mapped_hwpage = io; + }else if (msc->iface ==MPAM_IFACE_PCC ) { + msc->pcc_cl.dev = &pdev->dev; + msc->pcc_cl.rx_callback = mpam_pcc_rx_callback; + msc->pcc_cl.tx_block = false; + msc->pcc_cl.tx_tout =1000 ;/* 1s */ + msc->pcc_cl.knows_txdone = false; + + msc->pcc_chan =pcc_mbox_request_channel (&msc->pcc_cl, + msc->pcc_subspace_id); +if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); + err =PTR_ERR (msc->pcc_chan); +break ; + } + } + +list_add_rcu (&msc->glbl_list, &mpam_all_msc); +platform_set_drvdata (pdev, msc); + }while (0 ); +mutex_unlock (&mpam_list_lock); + +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled) + err =acpi_mpam_parse_resources (msc, plat_data); +else + err =mpam_dt_parse_resources (msc, plat_data); + } + +if (!err && fw_num_msc == mpam_num_msc) +mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err; +} + +static const struct of_device_id mpam_of_match[] = { + { .compatible ="arm,mpam-msc" , }, + {}, +}; +MODULE_DEVICE_TABLE (of, mpam_of_match); + +static struct platform_driver mpam_msc_driver = { + .driver = { + .name ="mpam_msc" , + .of_match_table =of_match_ptr (mpam_of_match), + }, + .probe = mpam_msc_drv_probe, + .remove = mpam_msc_drv_remove, +}; + +/* +* MSC that are hidden under caches are not created as platform devices +* as there is no cache driver. Caches are also special-cased in +* update_msc_accessibility(). +*/ +static void mpam_dt_create_foundling_msc (void ) +{ +int err; +struct device_node *cache; + +for_each_compatible_node (cache,NULL ,"cache" ) { + err =of_platform_populate (cache, mpam_of_match,NULL ,NULL ); +if (err) +pr_err ("Failed to create MSC devices under caches\n" ); + } +} + +static int __init mpam_msc_driver_init (void ) +{ +if (!system_supports_mpam ()) +return -EOPNOTSUPP ; + +init_srcu_struct (&mpam_srcu); + +if (!acpi_disabled) + fw_num_msc =acpi_mpam_count_msc (); +else + fw_num_msc =mpam_dt_count_msc (); + +if (fw_num_msc <=0 ) { +pr_err ("No MSC devices found in firmware\n" ); +return -EINVAL ; + } + +if (acpi_disabled) +mpam_dt_create_foundling_msc (); + +return platform_driver_register (&mpam_msc_driver); +} +subsys_initcall (mpam_msc_driver_init);drivers/resctrl/mpam_internal.h new file +25
-0@@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +// Copyright (C) 2024 Arm Ltd. + +#ifndef MPAM_INTERNAL_H +#define MPAM_INTERNAL_H + +#include <linux/arm_mpam.h> +#include <linux/cpumask.h> +#include <linux/io.h> +#include <linux/mailbox_client.h> +#include <linux/mutex.h> +#include <linux/resctrl.h> +#include <linux/sizes.h> + +struct mpam_msc { + /* member of mpam_all_msc */ + struct list_head glbl_list; + + int id; + struct platform_device *pdev; + + /* Not modified after mpam_is_enabled() becomes true */ + enum mpam_msc_iface iface; + u32 pcc_subspace_id; + struct mbox_client pcc_cl;
drivers/resctrl/mpam_internal.h line 35
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+struct pcc_mbox_chan *pcc_chan; +u32 nrdy_usec; +cpumask_t accessibility; + +/* +* probe_lock is only take during discovery. After discovery these +* properties become read-only and the lists are protected by SRCU. +*/ +struct mutex probe_lock; +unsigned long ris_idxs[128 /BITS_PER_LONG ];
Why is this sized this way? RIS_MAX is 4 bits and so there are at most 16 RIS per msc.
Quoted 17 lines
drivers/resctrl/mpam_internal.h new file +17
-0+u32 ris_max; + +/* mpam_msc_ris of this component */ +struct list_head ris; + +/* +* part_sel_lock protects access to the MSC hardware registers that are +* affected by MPAMCFG_PART_SEL. (including the ID registers that vary +* by RIS). +* If needed, take msc->lock first. +*/ +struct mutex part_sel_lock; + +/* +* mon_sel_lock protects access to the MSC hardware registers that are +* affeted by MPAMCFG_MON_SEL. +* If needed, take msc->lock first.
drivers/resctrl/mpam_internal.h line 62
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+*/ +struct mutex outer_mon_sel_lock; +raw_spinlock_t inner_mon_sel_lock; +unsigned long inner_mon_sel_flags; + +void __iomem *mapped_hwpage; + size_t mapped_hwpage_sz; +}; + +#endif /* MPAM_INTERNAL_H */
Thanks,
Ben
James Morse
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
Hi Ben,
Quoted 32 lines · Ben Horgan
Quoted 31 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.Start with driver probe/remove and mapping the MSC.
drivers/resctrl/mpam_internal.h new file +16
-0@@ -0,0 +1,62 @@ +struct mpam_msc { +/* member of mpam_all_msc */ +struct list_head glbl_list; + +int id; +struct platform_device *pdev; + +/* Not modified after mpam_is_enabled() becomes true */ +enum mpam_msc_iface iface; +u32 pcc_subspace_id; +struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +u32 nrdy_usec; +cpumask_t accessibility; + +/*
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 35
Quoted 5 lines
drivers/resctrl/mpam_internal.h new file +5
-0+* probe_lock is only take during discovery. After discovery these +* properties become read-only and the lists are protected by SRCU. +*/ +struct mutex probe_lock; +unsigned long ris_idxs[128 /BITS_PER_LONG ];Why is this sized this way?
RIS_MAXis 4 bits and so there are at most 16 RIS per msc.
Hmmm, lost in time - I agree with the 16 reasoning. Fixed.
(It's likely due to RES0 space above the field - but that has been filled in with other
stuff since then. RIS was added as a 'backward compatible feature' - I was wary of them extending it)
Quoted 19 lines
Quoted 19 lines
drivers/resctrl/mpam_internal.h new file +19
-0+u32 ris_max; + +/* mpam_msc_ris of this component */ +struct list_head ris; + +/* +* part_sel_lock protects access to the MSC hardware registers that are +* affected by MPAMCFG_PART_SEL. (including the ID registers that vary +* by RIS). +* If needed, take msc->lock first. +*/ +struct mutex part_sel_lock; + +/* +* mon_sel_lock protects access to the MSC hardware registers that are +* affeted by MPAMCFG_MON_SEL. +* If needed, take msc->lock first. +*/ +struct mutex outer_mon_sel_lock;
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 62
Quoted 8 lines
drivers/resctrl/mpam_internal.h new file +8
-0+raw_spinlock_t inner_mon_sel_lock; +unsigned long inner_mon_sel_flags; + +void __iomem *mapped_hwpage; + size_t mapped_hwpage_sz; +}; + +#endif /* MPAM_INTERNAL_H */
Thanks,
James
Re: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
In reply to James MorseHi James,
Quoted 16 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.Start with driver probe/remove and mapping the MSC.
CC Carl Worth <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since RFC:
* Check for status=broken DT devices.
* Moved all the files around.
* MadeKconfigsymbols depend on EXPERT
---
[...]
drivers/resctrl/Kconfig line 2
Quoted 8 lines
drivers/resctrl/Kconfig new file +2
-0@@ -0,0 +1,11 @@ +# Confusingly, this is everything but the CPU bits of MPAM. CPU here means +# CPU resources, not containers or cgroups etc.
Drop confusing comment?
CPUs are not mentioned other than in the comment -- I think the descriptions are sufficiently self-explanatory that they don't read onto CPUs.
drivers/resctrl/Kconfig line 8
Quoted 6 lines
drivers/resctrl/Kconfig new file +6
-0+config ARM64_MPAM_DRIVER + bool "MPAM driver for System IP, e,g. caches and memory controllers" + depends on ARM64_MPAM && EXPERT + +config ARM64_MPAM_DRIVER_DEBUG + bool "Enable debug messages from the MPAM driver."
Nit: spurious full stop.
(i.e., people don't add one in these one-line descriptions. They are title-like and self-delimiting, even when the text is a valid sentence.)
Quoted 49 lines
drivers/resctrl/Kconfig new file +3
-0+ depends on ARM64_MPAM_DRIVER + help + Say yes here to enable debug messages from the MPAM driver.drivers/resctrl/Makefile new file +4
-0@@ -0,0 +1,4 @@ +obj-$(CONFIG_ARM64_MPAM_DRIVER) += mpam.o +mpam-y += mpam_devices.o + +cflags-$(CONFIG_ARM64_MPAM_DRIVER_DEBUG) += -DDEBUGdrivers/resctrl/mpam_devices.c new file +30
-0@@ -0,0 +1,336 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2025 Arm Ltd. + +#define pr_fmt (fmt) "%s:%s: " fmt, KBUILD_MODNAME, __func__ + +#include <linux/acpi.h> +#include <linux/arm_mpam.h> +#include <linux/cacheinfo.h> +#include <linux/cpu.h> +#include <linux/cpumask.h> +#include <linux/device.h> +#include <linux/errno.h> +#include <linux/gfp.h> +#include <linux/list.h> +#include <linux/lockdep.h> +#include <linux/mutex.h> +#include <linux/of.h> +#include <linux/of_platform.h> +#include <linux/platform_device.h> +#include <linux/printk.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/srcu.h> +#include <linux/types.h> + +#include <acpi/pcc.h> + +#include "mpam_internal.h" + +/*
drivers/resctrl/mpam_devices.c line 40
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+* mpam_list_lock protects the SRCU lists when writing. Once the +* mpam_enabled key is enabled these lists are read-only, +* unless the error interrupt disables the driver. +*/ +static DEFINE_MUTEX (mpam_list_lock); +static LIST_HEAD (mpam_all_msc); + +static struct srcu_struct mpam_srcu; + +/* MPAM isn't available until all the MSC have been probed. */
Comment doesn't really explain the variable.
Maybe something like "Number of MSCs that need to be probed for MPAM to be usable" ?
drivers/resctrl/mpam_devices.c line 41
Quoted 1 line
drivers/resctrl/mpam_devices.c new file +1
-0+static u32 mpam_num_msc;
Any particular reason this is u32 and not unsigned int?
How are accesses to this protected against data races?
If there are supposed to be locks to protect globals in the MPAM driver, is it worth wrapping them in access functions with a lockdep assert? Otherwise, it feels rather easy to get this wrong -- I think I've found at least one bug (see mpam_msc_drv_probe().)
drivers/resctrl/mpam_devices.c line 46
Quoted 5 lines
drivers/resctrl/mpam_devices.c new file +5
-0+ +static void mpam_discovery_complete (void ) +{ +pr_err ("Discovered all MSC\n" ); +}
As others have commented, if this is non-functional code that gets removed later on, it's probably best to drop this up-front?
[...]
Quoted 21 lines
drivers/resctrl/mpam_devices.c new file +21
-0+static int mpam_dt_parse_resource (struct mpam_msc *msc,struct device_node *np, +u32 ris_idx) +{ +int err =0 ; +u32 level =0 ; +unsigned long cache_id; +struct device_node *cache; + + do { + if (of_device_is_compatible(np, "arm,mpam-cache")) { + cache = of_parse_phandle(np, "arm,mpam-device", 0); + if (!cache) { + pr_err("Failed to read phandle\n"); + break; + } + } else if (of_device_is_compatible(np->parent, "cache")) { + cache = of_node_get(np->parent); + } else { + /* For now, only caches are supported */ + cache = NULL; + break;
drivers/resctrl/mpam_devices.c line 91
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ } + + err =of_property_read_u32 (cache,"cache-level" , &level); +if (err) { +pr_err ("Failed to read cache-level\n" ); +break ; + } + + cache_id =cache_of_calculate_id (cache); +if (cache_id == ~0UL ) {
The type of cache_id may change if the return type of cache_of_calculate_id() changes (see comments on patch 1).
Possible #define for the exceptional value.
drivers/resctrl/mpam_devices.c line 93
Quoted 2 lines
drivers/resctrl/mpam_devices.c new file +2
-0+ err = -ENOENT ; +break ;
The lack of a diagnostic here is inconsistent with the level of diagnostics in the rest of the loop.
drivers/resctrl/mpam_devices.c line 98
Quoted 5 lines
drivers/resctrl/mpam_devices.c new file +5
-0+ } + + err =mpam_ris_create (msc, ris_idx,MPAM_CLASS_CACHE , level, + cache_id); + }while (0 );
Abuse of do ... while () here?
There is no loop. The breaks are stealth "goto"s to this statement:
drivers/resctrl/mpam_devices.c line 99
Quoted 1 line
drivers/resctrl/mpam_devices.c new file +1
-0+of_node_put (cache);
(It works either way, but maybe gotos to an explicit label would be more readable, as well as avoiding an unnecessary level of indentation.)
Quoted 3 lines
drivers/resctrl/mpam_devices.c new file +3
-0+ +return err; +}
[...]
Quoted 5 lines
+/*
+ * An MSC can control traffic from a set of CPUs, but may only be accessible
+ * from a (hopefully wider) set of CPUs. The common reason for this is power
+ * management. If all the CPUs in a cluster are in PSCI:CPU_SUSPEND, the
+ * the corresponding cache may also be powered off. By making accesses from
Nit: the the
Quoted 26 lines
+ * one of those CPUs, we ensure this isn't the case.
+ */drivers/resctrl/mpam_devices.c new file +24
-0+static int update_msc_accessibility (struct mpam_msc *msc) +{ +struct device_node *parent; +u32 affinity_id; +int err; + +if (!acpi_disabled) { + err =device_property_read_u32 (&msc->pdev->dev,"cpu_affinity" , + &affinity_id); +if (err) +cpumask_copy (&msc->accessibility, cpu_possible_mask); +else +acpi_pptt_get_cpus_from_container (affinity_id, + &msc->accessibility); + +return 0 ; + } + +/* This depends on the path to of_node */ + parent =of_get_parent (msc->pdev->dev.of_node); +if (parent == of_root) { +cpumask_copy (&msc->accessibility, cpu_possible_mask); + err =0 ; + }else {
drivers/resctrl/mpam_devices.c line 169
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ err = -EINVAL ; +pr_err ("Cannot determine accessibility of MSC: %s\n" , +dev_name (&msc->pdev->dev)); + } +of_node_put (parent); + +return err; +} + +static int fw_num_msc;
Does this need to be protected against data races?
If individual mpam_msc_drv_probe() calls may execute on different CPUs from mpam_msc_driver_init(), then seem to be potential races here.
drivers/resctrl/mpam_devices.c line 173
Quoted 4 lines
drivers/resctrl/mpam_devices.c new file +4
-0+ +static void mpam_pcc_rx_callback (struct mbox_client *cl,void *msg) +{ +/* TODO: wake up tasks blocked on this MSC's PCC channel */
So, is this broken in this commit?
(If the series does not get broken up or applied piecemail, that's not such a concern, though.)
drivers/resctrl/mpam_devices.c line 177
Quoted 4 lines
drivers/resctrl/mpam_devices.c new file +4
-0+} + +static void mpam_msc_drv_remove (struct platform_device *pdev) +{
The MPAM driver cannot currently be built as a module.
Is it possible to exercise the driver remove paths, today?
Quoted 71 lines
drivers/resctrl/mpam_devices.c new file +71
-0+struct mpam_msc *msc =platform_get_drvdata (pdev); + +if (!msc) +return ; + +mutex_lock (&mpam_list_lock); + mpam_num_msc--; +platform_set_drvdata (pdev,NULL ); +list_del_rcu (&msc->glbl_list); +synchronize_srcu (&mpam_srcu); +devm_kfree (&pdev->dev, msc); +mutex_unlock (&mpam_list_lock); +} + +static int mpam_msc_drv_probe (struct platform_device *pdev) +{ +int err; +struct mpam_msc *msc; +struct resource *msc_res; +void *plat_data = pdev->dev.platform_data; + +mutex_lock (&mpam_list_lock); + do { + msc = devm_kzalloc(&pdev->dev, sizeof(*msc), GFP_KERNEL); + if (!msc) { + err = -ENOMEM; + break; + } + + mutex_init(&msc->probe_lock); + mutex_init(&msc->part_sel_lock); + mutex_init(&msc->outer_mon_sel_lock); + raw_spin_lock_init(&msc->inner_mon_sel_lock); + msc->id = mpam_num_msc++; + msc->pdev = pdev; + INIT_LIST_HEAD_RCU(&msc->glbl_list); + INIT_LIST_HEAD_RCU(&msc->ris); + + err = update_msc_accessibility(msc); + if (err) + break; + if (cpumask_empty(&msc->accessibility)) { + pr_err_once("msc:%u is not accessible from any CPU!", + msc->id); + err = -EINVAL; + break; + } + + if (device_property_read_u32(&pdev->dev, "pcc-channel", + &msc->pcc_subspace_id)) + msc->iface = MPAM_IFACE_MMIO; + else + msc->iface = MPAM_IFACE_PCC; + + if (msc->iface == MPAM_IFACE_MMIO) { + void __iomem *io; + + io = devm_platform_get_and_ioremap_resource(pdev, 0, + &msc_res); + if (IS_ERR(io)) { + pr_err("Failed to map MSC base address\n"); + err = PTR_ERR(io); + break; + } + msc->mapped_hwpage_sz = msc_res->end - msc_res->start; + msc->mapped_hwpage = io; + } else if (msc->iface == MPAM_IFACE_PCC) { + msc->pcc_cl.dev = &pdev->dev; + msc->pcc_cl.rx_callback = mpam_pcc_rx_callback; + msc->pcc_cl.tx_block = false; + msc->pcc_cl.tx_tout = 1000; /* 1s */
drivers/resctrl/mpam_devices.c line 258
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ msc->pcc_cl.knows_txdone = false; + + msc->pcc_chan =pcc_mbox_request_channel (&msc->pcc_cl, + msc->pcc_subspace_id); +if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); + err =PTR_ERR (msc->pcc_chan); +break ; + } + }
Should the lock be held across initialisation of the msc fields?
list_add_rcu() might imply sufficient barriers to ensure that the initialisations are visible to other threads that obtain the msc pointer by iterating over mpam_all_msc.
It's probably cleaner to hold the lock explicitly, though.
What other ways of obtaining the msc pointer exist?
Quoted 5 lines
drivers/resctrl/mpam_devices.c new file +5
-0+ +list_add_rcu (&msc->glbl_list, &mpam_all_msc); +platform_set_drvdata (pdev, msc); + }while (0 ); +mutex_unlock (&mpam_list_lock);
drivers/resctrl/mpam_devices.c line 273
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled) + err =acpi_mpam_parse_resources (msc, plat_data); +else + err =mpam_dt_parse_resources (msc, plat_data); + } + +if (!err && fw_num_msc == mpam_num_msc)
Unlocked read of mpam_num_msc?
Quoted 18 lines
drivers/resctrl/mpam_devices.c new file +18
-0+mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err; +} + +static const struct of_device_id mpam_of_match[] = { + { .compatible ="arm,mpam-msc" , }, + {}, +}; +MODULE_DEVICE_TABLE (of, mpam_of_match); + +static struct platform_driver mpam_msc_driver = { + .driver = { + .name ="mpam_msc" , + .of_match_table =of_match_ptr (mpam_of_match),
drivers/resctrl/mpam_devices.c line 301
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+ }, + .probe = mpam_msc_drv_probe, + .remove = mpam_msc_drv_remove, +}; + +/* +* MSC that are hidden under caches are not created as platform devices +* as there is no cache driver. Caches are also special-cased in +* update_msc_accessibility(). +*/
Can you elaborate? I don't understand quite what this is doing.
Quoted 11 lines
drivers/resctrl/mpam_devices.c new file +11
-0+static void mpam_dt_create_foundling_msc (void ) +{ +int err; +struct device_node *cache; + +for_each_compatible_node (cache,NULL ,"cache" ) { + err =of_platform_populate (cache, mpam_of_match,NULL ,NULL ); +if (err) +pr_err ("Failed to create MSC devices under caches\n" ); + } +}
[...]
Quoted 13 lines
drivers/resctrl/mpam_internal.h new file +7
-0@@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +// Copyright (C) 2024 Arm Ltd. + +#ifndef MPAM_INTERNAL_H +#define MPAM_INTERNAL_H + +#include <linux/arm_mpam.h>
drivers/resctrl/mpam_internal.h line 17
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+#include <linux/cpumask.h> +#include <linux/io.h> +#include <linux/mailbox_client.h> +#include <linux/mutex.h> +#include <linux/resctrl.h> +#include <linux/sizes.h> + +struct mpam_msc { +/* member of mpam_all_msc */ +struct list_head glbl_list;
It is worth making these names less mismatched?
Quoted 7 lines
drivers/resctrl/mpam_internal.h new file +7
-0+ +int id; +struct platform_device *pdev; + +/* Not modified after mpam_is_enabled() becomes true */ +enum mpam_msc_iface iface; +u32 pcc_subspace_id;
drivers/resctrl/mpam_internal.h line 34
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +u32 nrdy_usec; +cpumask_t accessibility; + +/* +* probe_lock is only take during discovery. After discovery these +* properties become read-only and the lists are protected by SRCU. +*/ +struct mutex probe_lock;
Can we have more clarify about the locking strategy, including details of which things each lock is supposed to apply to and when, and how (if at all) the locks are intended to nest?
(Similarly for the global locks.)
drivers/resctrl/mpam_internal.h line 36
Quoted 2 lines
drivers/resctrl/mpam_internal.h new file +2
-0+unsigned long ris_idxs[128 /BITS_PER_LONG ]; +u32 ris_max;
nrdy_usec, ris_idxs and ris_max appear unused in this patch (though I suppose they get initialised by virtue of kzalloc()). Is this intentional?
drivers/resctrl/mpam_internal.h line 46
Quoted 10 lines
drivers/resctrl/mpam_internal.h new file +10
-0+ +/* mpam_msc_ris of this component */ +struct list_head ris; + +/* +* part_sel_lock protects access to the MSC hardware registers that are +* affected by MPAMCFG_PART_SEL. (including the ID registers that vary +* by RIS). +* If needed, take msc->lock first. +*/
What's msc->lock ?
drivers/resctrl/mpam_internal.h line 53
Quoted 7 lines
drivers/resctrl/mpam_internal.h new file +7
-0+struct mutex part_sel_lock; + +/* +* mon_sel_lock protects access to the MSC hardware registers that are +* affeted by MPAMCFG_MON_SEL. +* If needed, take msc->lock first. +*/
Same here.
drivers/resctrl/mpam_internal.h line 62
Quoted 9 lines
drivers/resctrl/mpam_internal.h new file +9
-0+struct mutex outer_mon_sel_lock; +raw_spinlock_t inner_mon_sel_lock; +unsigned long inner_mon_sel_flags; + +void __iomem *mapped_hwpage; + size_t mapped_hwpage_sz; +}; + +#endif /* MPAM_INTERNAL_H */
[...]
Cheers
---Dave
James Morse
maintainerRe: [PATCH 10/33] arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate
Hi Dave,
Quoted 19 lines · Dave Martin
Quoted 13 lines · James Morse
Probing MPAM is convoluted. MSCs that are integrated with a CPU may only be accessible from those CPUs, and they may not be online. Touching the hardware early is pointless as MPAM can't be used until the system-wide common values for
num_partidandnum_pmghave been discovered.Start with driver probe/remove and mapping the MSC.
drivers/resctrl/Kconfig new file +0
-0drivers/resctrl/Kconfig line 2
Quoted 2 lines
drivers/resctrl/Kconfig new file +2
-0@@ -0,0 +1,11 @@ +# Confusingly, this is everything but the CPU bits of MPAM. CPU here means +# CPU resources, not containers or cgroups etc.Drop confusing comment?
Quoted 3 lines
CPUs are not mentioned other than in the comment -- I think the descriptions are sufficiently self-explanatory that they don't read onto CPUs.
This used to add ARM_CPU_RESCTRL, to mirror X86_CPU_RESCTRL. It's been tidied up since then, but the comment remains. I'll remove it.
Quoted 9 lines
drivers/resctrl/Kconfig line 8
Quoted 6 lines
drivers/resctrl/Kconfig new file +6
-0+config ARM64_MPAM_DRIVER + bool "MPAM driver for System IP, e,g. caches and memory controllers" + depends on ARM64_MPAM && EXPERT + +config ARM64_MPAM_DRIVER_DEBUG + bool "Enable debug messages from the MPAM driver."Nit: spurious full stop.
Quoted 3 lines
(i.e., people don't add one in these one-line descriptions. They are title-like and self-delimiting, even when the text is a valid sentence.)
/me waves hands around
Quoted 25 lines
Quoted 12 lines
drivers/resctrl/Kconfig new file +3
-0+ depends on ARM64_MPAM_DRIVER + help + Say yes here to enable debug messages from the MPAM driver.drivers/resctrl/mpam_devices.c new file +1
-0@@ -0,0 +1,336 @@ +/* drivers/resctrl/mpam_devices.c line 40
Quoted 10 lines
drivers/resctrl/mpam_devices.c new file +10
-0+* mpam_list_lock protects the SRCU lists when writing. Once the +* mpam_enabled key is enabled these lists are read-only, +* unless the error interrupt disables the driver. +*/ +static DEFINE_MUTEX (mpam_list_lock); +static LIST_HEAD (mpam_all_msc); + +static struct srcu_struct mpam_srcu; + +/* MPAM isn't available until all the MSC have been probed. */ Comment doesn't really explain the variable.
Quoted 2 lines
Maybe something like "Number of MSCs that need to be probed for MPAM to be usable" ?
Its the count not the remainder. I went with:
| * Number of MSCs that have been probed. Once all MSC have been probed MPAM | * can be enabled.
Quoted 3 lines
Quoted 1 line
+static u32
mpam_num_msc;Any particular reason this is u32 and not unsigned int?
u32 is less typing!
Quoted 1 line
How are accesses to this protected against data races?
It's under the list-lock, but after Rob's feedback I've made it an atomic_t and stopped using it as an id for all the print messages.
Quoted 4 lines
If there are supposed to be locks to protect globals in the MPAM driver, is it worth wrapping them in access functions with a lockdep assert? Otherwise, it feels rather easy to get this wrong -- I think I've found at least one bug (see
mpam_msc_drv_probe().)
Broadly: everything is protected by the list_lock when things are being discovered. Once everything has been discovered, these things can become read-only.
It's not until everything has been discovered that interrupts get registered, and things like a potential PMU driver could make calls in a strange context.
Adding helpers would need some global state variable, if (state == foo) lockdep_assert... I had that early on, but figured it was overkill.
Quoted 7 lines
drivers/resctrl/mpam_devices.c line 46
Quoted 4 lines
drivers/resctrl/mpam_devices.c new file +4
-0+static void mpam_discovery_complete (void ) +{ +pr_err ("Discovered all MSC\n" ); +}As others have commented, if this is non-functional code that gets removed later on, it's probably best to drop this up-front?
It's illustrating that something happens after all the MSC have been discovered. Knowing that from the beginning of the series is supposed to make the insertion of the cpuhp notifiers in the middle easier to think about...
Quoted 26 lines
Quoted 26 lines
drivers/resctrl/mpam_devices.c new file +26
-0+static int mpam_dt_parse_resource (struct mpam_msc *msc,struct device_node *np, +u32 ris_idx) +{ +int err =0 ; +u32 level =0 ; +unsigned long cache_id; +struct device_node *cache; + + do { + if (of_device_is_compatible(np, "arm,mpam-cache")) { + cache = of_parse_phandle(np, "arm,mpam-device", 0); + if (!cache) { + pr_err("Failed to read phandle\n"); + break; + } + } else if (of_device_is_compatible(np->parent, "cache")) { + cache = of_node_get(np->parent); + } else { + /* For now, only caches are supported */ + cache = NULL; + break; + } + + err = of_property_read_u32(cache, "cache-level", &level); + if (err) { + pr_err("Failed to read cache-level\n");
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 91
Quoted 5 lines
drivers/resctrl/mpam_devices.c new file +5
-0+break ; + } + + cache_id =cache_of_calculate_id (cache); +if (cache_id == ~0UL ) {The type of
cache_idmay change if the return type ofcache_of_calculate_id()changes (see comments on patch 1).
Yup,
Quoted 1 line
Possible #define for the exceptional value.
I don't think its any more surprising than '-1' as an error value, and its only got one caller, which is pretty obviously an error path.
Quoted 5 lines
drivers/resctrl/mpam_devices.c line 93
Quoted 2 lines
drivers/resctrl/mpam_devices.c new file +2
-0+ err = -ENOENT ; +break ;The lack of a diagnostic here is inconsistent with the level of diagnostics in the rest of the loop.
I've never needed to debug that one because its already visible to user-space. If the cache-id's are missing, you can tell that in sysfs, you don't need to instrument the kernel.
I'll add one if you think its important. They can all be _once, and as its related to the probing of a particular device, can use the dev_ print helpers.
Quoted 9 lines
drivers/resctrl/mpam_devices.c line 98
Quoted 5 lines
drivers/resctrl/mpam_devices.c new file +5
-0+ } + + err =mpam_ris_create (msc, ris_idx,MPAM_CLASS_CACHE , level, + cache_id); + }while (0 );Abuse of do ... while () here?
There is no loop. The breaks are stealth "goto"s to this statement:
Yes. The alternative would be actual gotos - which is surely worse! It just wasn't worth pulling this out as a separate function.
Quoted 4 lines
drivers/resctrl/mpam_devices.c line 99
Quoted 1 line
drivers/resctrl/mpam_devices.c new file +1
-0+of_node_put (cache);(It works either way, but maybe gotos to an explicit label would be more readable, as well as avoiding an unnecessary level of indentation.)
As the cleanup magic has become fashionable, I'll switch to using that...
Quoted 6 lines
Quoted 3 lines
drivers/resctrl/mpam_devices.c new file +3
-0+ +return err; +}[...]
Quoted 7 lines
Quoted 5 lines
+/*
+ * An MSC can control traffic from a set of CPUs, but may only be accessible
+ * from a (hopefully wider) set of CPUs. The common reason for this is power
+ * management. If all the CPUs in a cluster are in PSCI:CPU_SUSPEND, the
+ * the corresponding cache may also be powered off. By making accesses fromNit: the the
Fixed,
Quoted 9 lines
Quoted 4 lines
+ * one of those CPUs, we ensure this isn't the case.
+ */+static int
fw_num_msc;Does this need to be protected against data races?
If individual
mpam_msc_drv_probe()calls may execute on different CPUs frommpam_msc_driver_init(), then seem to be potential races here.
Incrementing was under the list-lock, but not the last 'are they all done' read. Following Rob's comments I've made this an atomic_t.
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 173
Quoted 4 lines
drivers/resctrl/mpam_devices.c new file +4
-0+ +static void mpam_pcc_rx_callback (struct mbox_client *cl,void *msg) +{ +/* TODO: wake up tasks blocked on this MSC's PCC channel */ So, is this broken in this commit?>
(If the series does not get broken up or applied piecemail, that's not such a concern, though.)
Unsupported - or at least only enough to not mistake them for MMIO devices. I've pulled this out to a later patch in the tree that isn't in this series.
The platforms that need this haven't yet materialised, (and may not!). There is a prototype for DT/SCMI, but nothing I've seen yet for ACPI.
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 177
Quoted 4 lines
drivers/resctrl/mpam_devices.c new file +4
-0+} + +static void mpam_msc_drv_remove (struct platform_device *pdev) +{The MPAM driver cannot currently be built as a module.
Is it possible to exercise the driver remove paths, today?
Yes, through the sysfs unbind interface.
It doesn't make a lot of sense for MPAM as the moment you unbind the driver from one MSC it has to work out if it needs to teardown resctrl...
Quoted 75 lines
Quoted 75 lines
drivers/resctrl/mpam_devices.c new file +75
-0+struct mpam_msc *msc =platform_get_drvdata (pdev); + +if (!msc) +return ; + +mutex_lock (&mpam_list_lock); + mpam_num_msc--; +platform_set_drvdata (pdev,NULL ); +list_del_rcu (&msc->glbl_list); +synchronize_srcu (&mpam_srcu); +devm_kfree (&pdev->dev, msc); +mutex_unlock (&mpam_list_lock); +} + +static int mpam_msc_drv_probe (struct platform_device *pdev) +{ +int err; +struct mpam_msc *msc; +struct resource *msc_res; +void *plat_data = pdev->dev.platform_data; + +mutex_lock (&mpam_list_lock); + do { + msc = devm_kzalloc(&pdev->dev, sizeof(*msc), GFP_KERNEL); + if (!msc) { + err = -ENOMEM; + break; + } + + mutex_init(&msc->probe_lock); + mutex_init(&msc->part_sel_lock); + mutex_init(&msc->outer_mon_sel_lock); + raw_spin_lock_init(&msc->inner_mon_sel_lock); + msc->id = mpam_num_msc++; + msc->pdev = pdev; + INIT_LIST_HEAD_RCU(&msc->glbl_list); + INIT_LIST_HEAD_RCU(&msc->ris); + + err = update_msc_accessibility(msc); + if (err) + break; + if (cpumask_empty(&msc->accessibility)) { + pr_err_once("msc:%u is not accessible from any CPU!", + msc->id); + err = -EINVAL; + break; + } + + if (device_property_read_u32(&pdev->dev, "pcc-channel", + &msc->pcc_subspace_id)) + msc->iface = MPAM_IFACE_MMIO; + else + msc->iface = MPAM_IFACE_PCC; + + if (msc->iface == MPAM_IFACE_MMIO) { + void __iomem *io; + + io = devm_platform_get_and_ioremap_resource(pdev, 0, + &msc_res); + if (IS_ERR(io)) { + pr_err("Failed to map MSC base address\n"); + err = PTR_ERR(io); + break; + } + msc->mapped_hwpage_sz = msc_res->end - msc_res->start; + msc->mapped_hwpage = io; + } else if (msc->iface == MPAM_IFACE_PCC) { + msc->pcc_cl.dev = &pdev->dev; + msc->pcc_cl.rx_callback = mpam_pcc_rx_callback; + msc->pcc_cl.tx_block = false; + msc->pcc_cl.tx_tout = 1000; /* 1s */ + msc->pcc_cl.knows_txdone = false; + + msc->pcc_chan = pcc_mbox_request_channel(&msc->pcc_cl, + msc->pcc_subspace_id);
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 258
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +6
-0+if (IS_ERR (msc->pcc_chan)) { +pr_err ("Failed to request MSC PCC channel\n" ); + err =PTR_ERR (msc->pcc_chan); +break ; + } + }Should the lock be held across initialisation of the msc fields?
The msc isn't visible until its added to the list, so provided all that inialisation is done 'before' its added to the list, it doesn't matter.
Quoted 5 lines
list_add_rcu()might imply sufficient barriers to ensure that the initialisations are visible to other threads that obtain the msc pointer by iterating overmpam_all_msc.It's probably cleaner to hold the lock explicitly, though.
The list lock? We do.
But the readers don't need to take the list lock, its only there to prevent concurrent writers.
Quoted 1 line
What other ways of obtaining the msc pointer exist?
The class/component/device structures in a subsequent patch, protected in the same way. Once MPAM is enabled all that can be sprayed through resctrl - at which point no modifications are allowed, and teardown for fatal errors depends on the static-key.
Quoted 9 lines
Quoted 9 lines
drivers/resctrl/mpam_devices.c new file +9
-0+ +list_add_rcu (&msc->glbl_list, &mpam_all_msc); +platform_set_drvdata (pdev, msc); + }while (0 ); +mutex_unlock (&mpam_list_lock); + +if (!err) { +/* Create RIS entries described by firmware */ +if (!acpi_disabled)
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 273
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +6
-0+ err =acpi_mpam_parse_resources (msc, plat_data); + else + err =mpam_dt_parse_resources (msc, plat_data); + } + +if (!err && fw_num_msc == mpam_num_msc)Unlocked read of
mpam_num_msc?
Fixed as an atomic_t flavoured thing.
Quoted 6 lines
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +6
-0+mpam_discovery_complete (); + +if (err && msc) +mpam_msc_drv_remove (pdev); + +return err;
Quoted 8 lines
drivers/resctrl/mpam_devices.c line 301
Quoted 6 lines
drivers/resctrl/mpam_devices.c new file +6
-0+} +/* +* MSC that are hidden under caches are not created as platform devices +* as there is no cache driver. Caches are also special-cased in +* update_msc_accessibility(). +*/ Can you elaborate? I don't understand quite what this is doing.
/ {
my_thing {
compatible = "my_thing";
msc {
compatible = "arm,mpam-msc";
};
};
other_thing {
compatible = "other_thing";
};
msc {
compatible = "arm,mpam-msc";
arm,mpam-device = <&other_thing>;
};
l2-cache {
compatible = "cache";
msc {
compatible = "arm,mpam-msc";
};
};
};my_thing and other_thing's MSC will have devices created - but the cache will not, because it's a cache not a device, and anything below it is ignored.
Quoted 31 lines
Quoted 11 lines
drivers/resctrl/mpam_devices.c new file +11
-0+static void mpam_dt_create_foundling_msc (void ) +{ +int err; +struct device_node *cache; + +for_each_compatible_node (cache,NULL ,"cache" ) { + err =of_platform_populate (cache, mpam_of_match,NULL ,NULL ); +if (err) +pr_err ("Failed to create MSC devices under caches\n" ); + } +}[...]
Quoted 17 lines
drivers/resctrl/mpam_internal.h new file +11
-0@@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +// Copyright (C) 2024 Arm Ltd. + +#ifndef MPAM_INTERNAL_H +#define MPAM_INTERNAL_H + +#include <linux/arm_mpam.h> +#include <linux/cpumask.h> +#include <linux/io.h> +#include <linux/mailbox_client.h> +#include <linux/mutex.h>
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 17
Quoted 6 lines
drivers/resctrl/mpam_internal.h new file +6
-0+#include <linux/resctrl.h> +#include <linux/sizes.h> + +struct mpam_msc { +/* member of mpam_all_msc */ +struct list_head glbl_list;It is worth making these names less mismatched?
all_msc_list ?
It's because its global. The pattern otherwise is parent has a list foo, and all the children have a member 'foo_list'.
Quoted 13 lines
Quoted 13 lines
drivers/resctrl/mpam_internal.h new file +13
-0+ +int id; +struct platform_device *pdev; + +/* Not modified after mpam_is_enabled() becomes true */ +enum mpam_msc_iface iface; +u32 pcc_subspace_id; +struct mbox_client pcc_cl; +struct pcc_mbox_chan *pcc_chan; +u32 nrdy_usec; +cpumask_t accessibility; + +/*
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 34
Quoted 4 lines
drivers/resctrl/mpam_internal.h new file +4
-0+* probe_lock is only take during discovery. After discovery these +* properties become read-only and the lists are protected by SRCU. +*/ +struct mutex probe_lock;Can we have more clarify about the locking strategy, including details of which things each lock is supposed to apply to and when, and how (if at all) the locks are intended to nest?
The comment above is supposed to describe that. A rule of thumb is its all the stuff 'below' it in the struct, there is also a comment for each order. This one is a bit of a catch all for the values in this struct that can be written.
The purpose of the part_sel_lock and mon_sel_lock arrangement becomes much more obvious in subsequent patches when the driver starts accessing the registers of those names.
Quoted 8 lines
(Similarly for the global locks.)
drivers/resctrl/mpam_internal.h line 36
Quoted 2 lines
drivers/resctrl/mpam_internal.h new file +2
-0+unsigned long ris_idxs[128 /BITS_PER_LONG ]; +u32 ris_max;
nrdy_usec,ris_idxsandris_maxappear unused in this patch (though I suppose they get initialised by virtue ofkzalloc()). Is this intentional?
To reduce the amount of churn in the series the bulk of the structure is added here, then the stuff that build more complicated structures, accesses hardware, deals with interrupts etc.
Quoted 4 lines
Quoted 4 lines
drivers/resctrl/mpam_internal.h new file +4
-0+ +/* mpam_msc_ris of this component */ +struct list_head ris; +
Quoted 8 lines
drivers/resctrl/mpam_internal.h line 46
Quoted 6 lines
drivers/resctrl/mpam_internal.h new file +6
-0+/* +* part_sel_lock protects access to the MSC hardware registers that are +* affected by MPAMCFG_PART_SEL. (including the ID registers that vary +* by RIS). +* If needed, take msc->lock first. +*/ What's
msc->lock?
The old name for probe_lock, before it was necessary to lock the hardware registers separately. I've fixed the name.
Thanks,
James