x86/resctrl: Add support for Sub-NUMA cluster (SNC) systems
Subsystems (4)
Reinette Chatre
maintainerRe: [PATCH v5 6/8] x86/resctrl: Sub NUMA Cluster detection and enable
In reply to Luck, TonyHi Tony,
Quoted 21 lines · Tony
Quoted 3 lines
Quoted 1 line
+#include <
linux/mod_devicetable.h>I didn't see the need for this include.
struct
x86_cpu_idis defined in this #include file.Quoted 5 lines
Quoted 1 line
+static void
snc_remap_rmids(int cpu)While adding the new functions, i see that new function names start with
resctrl_prefix. However, we are all not very consistent. Can ypu rename this function toresctrl_snc_remap_rmids?I try to put a subsystem prefix on any global symbols to avoid random conflicts in other parts of the kernel. But I'm less sure of the value for static functions and variables that are only visible inside a single ".c" file.
If it must have a prefix, should it be "
intel_" rather than "resctrl_" to indicate that it is an Intel specific function?
Quoted 9 lines
Quoted 3 lines
Quoted 1 line
+static
__initintget_snc_config(void)Same comment as above.
Same answer.
Reinette: Any opinions on these?
I agree with you that a consistent prefix is expected from the global symbols and a prefix of resctrl_ is indeed the goal as can be seen in the growing include/linux/resctrl.h. resctrl has no established pattern for static functions (look at the other static functions in this file being changed) or even those non-static functions and data structures shared between the resctrl files (see arch/x86/kernel/cpu/resctrl/internal.h).
I'm ok with the static functions named snc_remap_rmids() and get_snc_config().
We can surely start a discussion if there is concern about resctrl static function prefixes but that discussion will have larger scope than this series.
If we do want to focus on function naming I do think a change that may benefit this work is to be consistent with verb placement in the function names. i.e either always have verb first or have a consistent snc_ prefix followed by a verb. I do not recall if there are x86 requirements in this regard.
Reinette
Tony Luck
maintainerPATCH v5 7/8 x86/resctrl: Update documentation with Sub-NUMA cluster changes
In reply to Tony LuckWith Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
--- Documentation/arch/x86/resctrl.rst | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)
Documentation/arch/x86/resctrl.rst +22 -3
@@ -345,9 +345,15 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. E.g. on a system with + SNC mode disabled with two L3 domains there will be subdirectories + "mon_L3_00" and "mon_L3_01". The numerical suffix refers to the + L3 cache id. With SNC enabled the directory names are the same, + but the numerical suffix refers to the node id. + Mappings from node ids to CPUs are available in the + /sys/devices/system/node/node*/cpulist files. Each of these directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for
@@ -452,6 +458,19 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled the "llc_occupancy", "mbm_total_bytes", and +"mbm_local_bytes" will only give accurate results for well behaved NUMA +applications. I.e. those that perform the majority of memory accesses +to memory on the local NUMA node to the CPU where the task is executing. + +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v5 7/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
Hi Tony,
Quoted 22 lines · Tony Luck
With Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
Signed-off-by Tony Luck <[email protected]>--- Documentation/arch/x86/resctrl.rst | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)Documentation/arch/x86/resctrl.rst +0
-2@@ -345,9 +345,15 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will
Documentation/arch/x86/resctrl.rst When control is enabled all CTRL_MON groups will also contain: · line 356
Quoted 10 lines
Documentation/arch/x86/resctrl.rst +9
-1- be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. E.g. on a system with + SNC mode disabled with two L3 domains there will be subdirectories + "mon_L3_00" and "mon_L3_01". The numerical suffix refers to the + L3 cache id. With SNC enabled the directory names are the same, + but the numerical suffix refers to the node id. + Mappings from node ids to CPUs are available in the + /sys/devices/system/node/node*/cpulist files. Each of these
Please be consistent with tabs vs spaces.
Quoted 4 lines
Documentation/arch/x86/resctrl.rst +0
-0directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for
Documentation/arch/x86/resctrl.rst and 0xA are not. On a system with a 20-bit mask each bit represents 5% · line 467
Quoted 10 lines
Documentation/arch/x86/resctrl.rst +7
-0@@ -452,6 +458,19 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled the "llc_occupancy", "mbm_total_bytes", and +"mbm_local_bytes" will only give accurate results for well behaved NUMA +applications. I.e. those that perform the majority of memory accesses +to memory on the local NUMA node to the CPU where the task is executing. +
Following our discussion of the cover letter I believe this will change to be specific about what user space can expect and what is actually "accurate".
Documentation/arch/x86/resctrl.rst and 0xA are not. On a system with a 20-bit mask each bit represents 5% · line 476
Quoted 9 lines
Documentation/arch/x86/resctrl.rst +6
-0+The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================
Reinette
Tony Luck
maintainerRe: [PATCH v5 7/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
Quoted 22 lines · Reinette Chatre
Hi Tony,
Quoted 19 lines · Tony Luck
With Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
Signed-off-by Tony Luck <[email protected]>--- Documentation/arch/x86/resctrl.rst | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)Documentation/arch/x86/resctrl.rst +0
-0@@ -345,9 +345,15 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain:
Quoted 15 lines
Documentation/arch/x86/resctrl.rst When control is enabled all CTRL_MON groups will also contain: · line 356
Quoted 13 lines
Documentation/arch/x86/resctrl.rst +9
-3"mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. E.g. on a system with + SNC mode disabled with two L3 domains there will be subdirectories + "mon_L3_00" and "mon_L3_01". The numerical suffix refers to the + L3 cache id. With SNC enabled the directory names are the same, + but the numerical suffix refers to the node id. + Mappings from node ids to CPUs are available in the + /sys/devices/system/node/node*/cpulist files. Each of thesePlease be consistent with tabs vs spaces.
I blame "vim" for this one. Typing <TAB> while working on a .rst file throws in 8 spaces. Maybe there's a vim setting to stop this annoying behaviour (set syntax=off didn't help). Had to use CTRL(V)<TAB> to make it work.
Quoted 8 lines
Quoted 8 lines
Documentation/arch/x86/resctrl.rst +0
-0directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for@@ -452,6 +458,19 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000.
Quoted 10 lines
Documentation/arch/x86/resctrl.rst and 0xA are not. On a system with a 20-bit mask each bit represents 5% · line 467
Quoted 7 lines
Documentation/arch/x86/resctrl.rst +7
-0+Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled the "llc_occupancy", "mbm_total_bytes", and +"mbm_local_bytes" will only give accurate results for well behaved NUMA +applications. I.e. those that perform the majority of memory accesses +to memory on the local NUMA node to the CPU where the task is executing. +Following our discussion of the cover letter I believe this will change to be specific about what user space can expect and what is actually "accurate".
I added more text about the importance of binding tasks to SNC nodes.
Quoted 12 lines
Documentation/arch/x86/resctrl.rst and 0xA are not. On a system with a 20-bit mask each bit represents 5% · line 476
Quoted 10 lines
Documentation/arch/x86/resctrl.rst +6
-0+The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================Reinette
Moger, Babu
reviewerRe: [PATCH v5 7/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
In reply to Tony LuckHi Tony,
How does user know Sub-NUMA Cluster mode is being enabled on the system?
Do you have any information in /sys/fs/resctrl/info?
Below documentation does not have any info about it. Would it be better to add that in "info" directory?
Thanks
Babu
Quoted 55 lines · Tony Luck
With Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
Signed-off-by Tony Luck <[email protected]>--- Documentation/arch/x86/resctrl.rst | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)Documentation/arch/x86/resctrl.rst +22
-3@@ -345,9 +345,15 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. E.g. on a system with + SNC mode disabled with two L3 domains there will be subdirectories + "mon_L3_00" and "mon_L3_01". The numerical suffix refers to the + L3 cache id. With SNC enabled the directory names are the same, + but the numerical suffix refers to the node id. + Mappings from node ids to CPUs are available in the + /sys/devices/system/node/node*/cpulist files. Each of these directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for@@ -452,6 +458,19 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled the "llc_occupancy", "mbm_total_bytes", and +"mbm_local_bytes" will only give accurate results for well behaved NUMA +applications. I.e. those that perform the majority of memory accesses +to memory on the local NUMA node to the CPU where the task is executing. + +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================
Signature · 2 lines
Thanks Babu Moger
Luck, Tony
maintainerRE: [PATCH v5 7/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
Quoted 6 lines
How does user know Sub-NUMA Cluster mode is being enabled on the system?
Do you have any information in
/sys/fs/resctrl/info?Below documentation does not have any info about it. Would it be better to add that in "info" directory?
Babu,
My original patch series added an "snc_ways" file to the info/ directory to make this visible. But I was talked out of it because of a lack of clear user mode use case that needs it.
https://lore.kernel.org/all/[email protected]/
Users can figure out whether SNC is enabled by looking at topology files for nodes and L3 caches in /sys. The open question is whether we should provide a simple short cut to that information, or make them walk two miles uphill in the snow to figure it out.
-Tony
Moger, Babu
reviewerRe: [PATCH v5 7/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
Quoted 8 lines · Tony
Quoted 6 lines
How does user know Sub-NUMA Cluster mode is being enabled on the system?
Do you have any information in
/sys/fs/resctrl/info?Below documentation does not have any info about it. Would it be better to add that in "info" directory?
Quoted 5 lines
Babu,
My original patch series added an "
snc_ways" file to the info/ directory to make this visible. But I was talked out of it because of a lack of clear user mode use case that needs it.
ok. Lets go with it.
Thanks
Babu Moger
Luck, Tony
maintainerRE: [PATCH v5 7/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
Quoted 5 lines
Quoted 3 lines
My original patch series added an "
snc_ways" file to the info/ directory to make this visible. But I was talked out of it because of a lack of clear user mode use case that needs it.ok. Lets go with it.
Babu,
I'm not sure whether you are voting to bring the snc_ways patch back into the series. Or to stick with the decision to drop that patch.
-Tony
Moger, Babu
reviewerRe: [PATCH v5 7/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
Quoted 11 lines · Tony
Quoted 5 lines
Quoted 3 lines
My original patch series added an "
snc_ways" file to the info/ directory to make this visible. But I was talked out of it because of a lack of clear user mode use case that needs it.ok. Lets go with it.
Babu,
I'm not sure whether you are voting to bring the
snc_wayspatch back into the series. Or to stick with the decision to drop that patch.
Sorry.. for not being clear. Lets stick with the James comment to drop it.
Signature · 2 lines
Thanks Babu Moger
Tony Luck
maintainerPATCH v5 8/8 selftests/resctrl: Adjust effective L3 cache size when SNC enabled
In reply to Tony LuckSub-NUMA Cluster divides CPUs sharing an L3 cache into separate NUMA nodes. Systems may support splitting into either two or four nodes.
When SNC mode is enabled the effective amount of L3 cache available for allocation is divided by the number of nodes per L3.
Detect which SNC mode is active by comparing the number of CPUs that share a cache with CPU0, with the number of CPUs on node0.
This gives some hope of tests passing. But additional test infrastructure changes are needed to bind tests to nodes and guarantee memory allocation from the local node.
--- tools/testing/selftests/resctrl/resctrl.h | 1 + tools/testing/selftests/resctrl/resctrlfs.c | 57 +++++++++++++++++++++ 2 files changed, 58 insertions(+)
tools/testing/selftests/resctrl/resctrl.h +1 -0
@@ -13,6 +13,7 @@#include <signal.h> #include <dirent.h> #include <stdbool.h> +#include <ctype.h> #include <sys/stat.h> #include <sys/ioctl.h> #include <sys/mount.h>
tools/testing/selftests/resctrl/resctrlfs.c +57 -0
69 lines not on this page · raw message
Signature · 1 line
2.41.0
Re: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
Hello,
Quoted 60 lines · Tony Luck
Sub-NUMA Cluster divides CPUs sharing an L3 cache into separate NUMA nodes. Systems may support splitting into either two or four nodes.
When SNC mode is enabled the effective amount of L3 cache available for allocation is divided by the number of nodes per L3.
Detect which SNC mode is active by comparing the number of CPUs that share a cache with CPU0, with the number of CPUs on node0.
This gives some hope of tests passing. But additional test infrastructure changes are needed to bind tests to nodes and guarantee memory allocation from the local node.
Reported-by "Shaopeng Tan (Fujitsu)" <[email protected]>Signed-off-by Tony Luck <[email protected]>---
tools/testing/selftests/resctrl/resctrl.h| 1 +tools/testing/selftests/resctrl/resctrlfs.c| 57 +++++++++++++++++++++2 files changed, 58 insertions(+)tools/testing/selftests/resctrl/resctrl.h +1
-0@@ -13,6 +13,7 @@#include <signal.h> #include <dirent.h> #include <stdbool.h> +#include <ctype.h> #include <sys/stat.h> #include <sys/ioctl.h> #include <sys/mount.h> tools/testing/selftests/resctrl/resctrlfs.c +20
-0@@ -130,6 +130,61 @@ int get_resource_id(int cpu_no, int *resource_id)return 0 ; } +/* +* Count number of CPUs in a /sys bit map +*/ +static int count_sys_bitmap_bits (char *name) +{ +FILE *fp =fopen (name,"r" ); +int count =0 , c; + +if (!fp) +return 0 ; + +while ((c =fgetc (fp)) !=EOF ) { +if (!isxdigit (c)) +continue ; +switch (c) { +case 'f' : + count++; +case '7' :case 'b' :case 'd' :case 'e' : + count++; +case '3' :case '5' :case '6' :case '9' :case 'a' :case 'c' :
tools/testing/selftests/resctrl/resctrlfs.c int get_resource_id(int cpu_no, int *resource_id) · line 162
Quoted 10 lines
tools/testing/selftests/resctrl/resctrlfs.c +10
-0+ count++; +case '1' :case '2' :case '4' :case '8' : + count++; + } + } +fclose (fp); + +return count; +} +
The resctrl selftest has a function for counting bits, could it be used here instead of the switch statement like this for example?
count = count_bits(c);
Or is there some reason this wouldn't be a good fit here?
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Luck, Tony
maintainerRE: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
Quoted 31 lines
Quoted 17 lines
tools/testing/selftests/resctrl/resctrlfs.c +17
-0+static int count_sys_bitmap_bits (char *name) +{ +FILE *fp =fopen (name,"r" ); +int count =0 , c; + +if (!fp) +return 0 ; + +while ((c =fgetc (fp)) !=EOF ) { +if (!isxdigit (c)) +continue ; +switch (c) { +case 'f' : + count++; +case '7' :case 'b' :case 'd' :case 'e' : + count++; +case '3' :case '5' :case '6' :case '9' :case 'a' :case 'c' :tools/testing/selftests/resctrl/resctrlfs.c int get_resource_id(int cpu_no, int *resource_id) · line 162
Quoted 10 lines
tools/testing/selftests/resctrl/resctrlfs.c +10
-0+ count++; +case '1' :case '2' :case '4' :case '8' : + count++; + } + } +fclose (fp); + +return count; +} +The resctrl selftest has a function for counting bits, could it be used here instead of the switch statement like this for example?
Quoted 3 lines
count =
count_bits(c);Or is there some reason this wouldn't be a good fit here?
Thanks for looking at my patch.
That count_bits() function is doing so with input from an "unsigned long" argument. My function is parsing the string result from a sysfs file which might look like this:
$ cat shared_cpu_map 0000,00000fff,ffffff00,0000000f,ffffffff
To use count_bits() I'd have to use something like strtol() on each of the comma separated fields first to convert from ascii strings to binary values to feed into count_bits().
-Tony
Re: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
Quoted 41 lines · Tony
Quoted 31 lines
Quoted 17 lines
tools/testing/selftests/resctrl/resctrlfs.c +17
-0+static int count_sys_bitmap_bits (char *name) +{ +FILE *fp =fopen (name,"r" ); +int count =0 , c; + +if (!fp) +return 0 ; + +while ((c =fgetc (fp)) !=EOF ) { +if (!isxdigit (c)) +continue ; +switch (c) { +case 'f' : + count++; +case '7' :case 'b' :case 'd' :case 'e' : + count++; +case '3' :case '5' :case '6' :case '9' :case 'a' :case 'c' :tools/testing/selftests/resctrl/resctrlfs.c int get_resource_id(int cpu_no, int *resource_id) · line 162
Quoted 10 lines
tools/testing/selftests/resctrl/resctrlfs.c +10
-0+ count++; +case '1' :case '2' :case '4' :case '8' : + count++; + } + } +fclose (fp); + +return count; +} +The resctrl selftest has a function for counting bits, could it be used here instead of the switch statement like this for example?
Quoted 3 lines
count =
count_bits(c);Or is there some reason this wouldn't be a good fit here?
Thanks for looking at my patch.
That
count_bits()function is doing so with input from an "unsigned long" argument. My function is parsing the string result from a sysfs file which might look like this:
Quoted 6 lines
$ cat shared_cpu_map 0000,00000fff,ffffff00,0000000f,ffffffffTo use
count_bits()I'd have to use something likestrtol()on each of the comma separated fields first to convert from ascii strings to binary values to feed intocount_bits().
I missed they are being read as characters and not bytes, sorry.
Out of curiosity, what about using fscanf instead of fgetc? With the format being %x and reading one byte at the time. Then instead of isxdigit just checking if the read number was bigger than 0xF.
I also remembered there is a gcc (and I think clang has it as well) builtin function that returns the number of set bits in a number.
So it would look like this:
while ((fscanf(fp, "%x", c)) != EOF ) {
if (c > 0xF)
continue;
count = __builtin_popcount(c);
}Are there some problems with an approach like that?
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Luck, Tony
maintainerRE: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
Quoted 9 lines
So it would look like this:
while ((fscanf(fp, "%x", c)) != EOF ) { if (c > 0xF) continue; count = __builtin_popcount(c); }Are there some problems with an approach like that?
I think I'd prefer something that does more checking on the input (e.g. the hex numbers are separated by "," and terminated with a '\n').
If Shuah Khan doesn't like my original patch I can re-write to use fscanf() et. al.
-Tony
RE: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
In reply to Tony LuckHi Tony,
Quoted 94 lines
Sub-NUMA Cluster divides CPUs sharing an L3 cache into separate NUMA nodes. Systems may support splitting into either two or four nodes.
When SNC mode is enabled the effective amount of L3 cache available for allocation is divided by the number of nodes per L3.
Detect which SNC mode is active by comparing the number of CPUs that share a cache with CPU0, with the number of CPUs on node0.
This gives some hope of tests passing. But additional test infrastructure changes are needed to bind tests to nodes and guarantee memory allocation from the local node.
Reported-by "Shaopeng Tan (Fujitsu)" <[email protected]>Signed-off-by Tony Luck <[email protected]>--- tools/testing/selftests/resctrl/resctrl.h | 1 + tools/testing/selftests/resctrl/resctrlfs.c | 57 +++++++++++++++++++++ 2 files changed, 58 insertions(+)tools/testing/selftests/resctrl/resctrl.h +1
-0@@ -13,6 +13,7 @@#include <signal.h> #include <dirent.h> #include <stdbool.h> +#include <ctype.h> #include <sys/stat.h> #include <sys/ioctl.h> #include <sys/mount.h> tools/testing/selftests/resctrl/resctrlfs.c +44
-0@@ -130,6 +130,61 @@ int get_resource_id(int cpu_no, int *resource_id)return 0 ; } +/* +* Count number of CPUs in a /sys bit map */ static int +count_sys_bitmap_bits (char *name) { +FILE *fp =fopen (name,"r" ); +int count =0 , c; + +if (!fp) +return 0 ; + +while ((c =fgetc (fp)) !=EOF ) { +if (!isxdigit (c)) +continue ; +switch (c) { +case 'f' : + count++; +case '7' :case 'b' :case 'd' :case 'e' : + count++; +case '3' :case '5' :case '6' :case '9' :case 'a' :case 'c' : + count++; +case '1' :case '2' :case '4' :case '8' : + count++; + } + } +fclose (fp); + +return count; +} + +/* +* Detect SNC by compating #CPUs in node0 with #CPUs sharing LLC with +CPU0 +* Try to get this right, even if a few CPUs are offline so that the +number +* of CPUs in node0 is not exactly half or a quarter of the CPUs +sharing the +* LLC of CPU0. +*/ +static int snc_ways (void ) +{ +int node_cpus, cache_cpus; + + node_cpus =count_sys_bitmap_bits ("/sys/devices/system/node/node0/cpumap" ); + cache_cpus = +count_sys_bitmap_bits("/sys/devices/system/cpu/cpu0/cache/index3/shared
tools/testing/selftests/resctrl/resctrlfs.c +4
-0+_cpu_map"); + + if (!node_cpus || !cache_cpus) { + fprintf(stderr, "Warning could not determine Sub-NUMACluster mode\n");
tools/testing/selftests/resctrl/resctrlfs.c int get_resource_id(int cpu_no, int *resource_id) · line 184
Quoted 7 lines
tools/testing/selftests/resctrl/resctrlfs.c +7
-0+return 1 ; + } + +if (4 * node_cpus >= cache_cpus) +return 4 ; +else if (2 * node_cpus >= cache_cpus) +return 2 ;
If "4 * node_cpus >= cache_cpus " is not true,
"2 * node_cpus >= cache_cpus" will never be true.
Is it the following code?
+ if (2 * node_cpus >= cache_cpus)
+ return 2;
+ else if (4 * node_cpus >= cache_cpus)
+ return 4;
Best regards,
Shaopeng TAN
Luck, Tony
maintainerRE: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
Quoted 10 lines
tools/testing/selftests/resctrl/resctrlfs.c int get_resource_id(int cpu_no, int *resource_id) · line 184
Quoted 4 lines
tools/testing/selftests/resctrl/resctrlfs.c +4
-0+if (4 * node_cpus >= cache_cpus) +return 4 ; +else if (2 * node_cpus >= cache_cpus) +return 2 ;If "4 *
node_cpus>=cache_cpus" is not true,
"2 *node_cpus>=cache_cpus" will never be true.
Is it the following code?
Review
Quoted 4 lines
(quoted diff) +4
-0+ if (2 * node_cpus >= cache_cpus) + return 2; + else if (4 * node_cpus >= cache_cpus) + return 4;
Shaopeng TAN,
Good catch. Your solution is the correct one.
Will fix in next post.
-Tony
Re: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
Quoted 16 lines · Tony
Quoted 10 lines
tools/testing/selftests/resctrl/resctrlfs.c int get_resource_id(int cpu_no, int *resource_id) · line 184
Quoted 4 lines
tools/testing/selftests/resctrl/resctrlfs.c +4
-0+if (4 * node_cpus >= cache_cpus) +return 4 ; +else if (2 * node_cpus >= cache_cpus) +return 2 ;If "4 *
node_cpus>=cache_cpus" is not true,
"2 *node_cpus>=cache_cpus" will never be true.
Is it the following code?Review
Quoted 4 lines
(quoted diff) +4
-0+ if (2 * node_cpus >= cache_cpus) + return 2; + else if (4 * node_cpus >= cache_cpus) + return 4;
Quoted 5 lines
Shaopeng TAN,
Good catch. Your solution is the correct one.
Will fix in next post.
I played around with this code a little and I think the logical expressions are returning wrong values.
On a system that has SNC disabled the function reports both "node_cpus" and "cache_cpus" equal to 56. In this case snc_ways() returns "2". It is the same on a system with SNC enabled that reports the previously mentioned variables to be different by a factor of two (36 and 72).
Is it possible for node_cpus and cache_cpus to not be multiples of each other? (as in for example cache_cpus being 10 and node_cpus being 21?). If not I'd suggest using "==" instead of ">=".
If yes then I guess something like this could work? :
+ if (node_cpus >= cache_cpus)
+ return 1;
+ else if (2 * node_cpus >= cache_cpus)
+ return 2;
+ else if (4 * node_cpus >= cache_cpus)
+ return 4;
PS. I did my tests on two Intel Ice Lakes.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Luck, Tony
maintainerRE: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
Quoted 8 lines
On a system that has SNC disabled the function reports both "
node_cpus" and "cache_cpus" equal to 56. In this casesnc_ways()returns "2". It is the same on a system with SNC enabled that reports the previously mentioned variables to be different by a factor of two (36 and 72).Is it possible for
node_cpusandcache_cpusto not be multiples of each other? (as in for examplecache_cpusbeing 10 andnode_cpusbeing 21?). If not I'd suggest using "==" instead of ">=".
Some CPUs may be offline when the test is run. E.g. with one CPU offline on SNC node 0, you'd see node_cpus = 35 and cache_cpus = 71. But with one CPU offline on node 1, you'd have node_cpus = 36, cache_cpus = 71.
Quoted 1 line
If yes then I guess something like this could work? :
+ if (node_cpus >= cache_cpus)
+ return 1;
+ else if (2 * node_cpus >= cache_cpus)
+ return 2;
+ else if (4 * node_cpus >= cache_cpus)
+ return 4;
This returns "4" for the 36 71 case. But should still be "2".
Quoted 1 line
Quoted 1 line
PS. I did my tests on two Intel Ice Lakes.
Perhaps easier to play with the algorithm in user code?
#include <stdio.h>
#include <stdlib.h>
static int snc(int node_cpus, int cache_cpus)
{
if (node_cpus >= cache_cpus)
return 1;
else if (2 * node_cpus >= cache_cpus)
return 2;
else if (4 * node_cpus >= cache_cpus)
return 4;
return -1;
}int main(int argc, char **argv)
{
printf("%d\n", snc(atoi(argv[1]), atoi(argv[2])));
return 0;
}N.B. it's probably not possible to handle the case where somebody took ALL the CPUs in SNC node 1 offline (or SNC nodes 1,2,3 for the SNC 4 case).
I think it reasonable that the code handle some simple "small number of CPUs offline" cases. But don't worry too much about cases where the user has done something extreme.
-Tony
Re: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
Hi, thanks for the reply
Quoted 5 lines · Tony
Quoted 4 lines
On a system that has SNC disabled the function reports both "
node_cpus" and "cache_cpus" equal to 56. In this casesnc_ways()returns "2". It is the same on a system with SNC enabled that reports the previously mentioned variables to be different by a factor of two (36 and 72).
Quoted 7 lines
Quoted 3 lines
Is it possible for
node_cpusandcache_cpusto not be multiples of each other? (as in for examplecache_cpusbeing 10 andnode_cpusbeing 21?). If not I'd suggest using "==" instead of ">=".Some CPUs may be offline when the test is run. E.g. with one CPU offline on SNC node 0, you'd see
node_cpus= 35 andcache_cpus= 71. But with one CPU offline on node 1, you'd havenode_cpus= 36,cache_cpus= 71.
Okay, thanks, good to know. On systems with disabled SNC that number should be equal even if some CPUs were offline, right? I was mostly concerned that the previous version was returning the same number whether SNC was enabled with 2 nodes or disabled.
Quoted 29 lines
Quoted 1 line
If yes then I guess something like this could work? :
tools/testing/selftests/resctrl/resctrlfs.c +6
-0+if (node_cpus >= cache_cpus) +return 1 ; +else if (2 * node_cpus >= cache_cpus) +return 2 ; +else if (4 * node_cpus >= cache_cpus) +return 4 ;This returns "4" for the 36 71 case. But should still be "2".
Quoted 1 line
Quoted 1 line
PS. I did my tests on two Intel Ice Lakes.
Perhaps easier to play with the algorithm in user code?
#include <
stdio.h>
#include <stdlib.h>static int
snc(int node_cpus, int cache_cpus){ if (node_cpus >= cache_cpus) return 1; else if (2 * node_cpus >= cache_cpus) return 2; else if (4 * node_cpus >= cache_cpus) return 4; return -1; }
Quoted 6 lines
int
main(int argc, char **argv){ printf("%d\n", snc(atoi(argv[1]), atoi(argv[2]))); return 0; }
My previous understanding was that the presence of ">=" comparison implied the number of node_cpus could somehow get larger. So I assumed that keeping it that way would be sufficient but now I can see that wouldn't be the case.
Quoted 8 lines
N.B. it's probably not possible to handle the case where somebody took ALL the CPUs in SNC node 1 offline (or SNC nodes 1,2,3 for the SNC 4 case).
I think it reasonable that the code handle some simple "small number of CPUs offline" cases. But don't worry too much about cases where the user has done something extreme.
-Tony
What about outputing this value to userspace from resctrl? The ratio is already saved inside snc_nodes_per_l3_cache variable. And that would help avoid these difficult cases when some cpus are offline which could cause snc_ways() to return a wrong value. Or are there some pitfalls to that approach?
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Luck, Tony
maintainerRE: [PATCH v5 8/8] selftests/resctrl: Adjust effective L3 cache size when SNC enabled
Quoted 5 lines
What about outputing this value to userspace from resctrl? The ratio is already saved inside
snc_nodes_per_l3_cachevariable. And that would help avoid these difficult cases when some cpus are offline which could causesnc_ways()to return a wrong value. Or are there some pitfalls to that approach?
My original patch series added an "snc_ways" file to the info/ directory to make this visible. But I was talked out of it because of a lack of clear user mode use case that needs it.
https://lore.kernel.org/all/[email protected]/
I don't know if the resctrl self tests constitute a valid use case. Perhaps not as they can figure this out.
But ... maybe the difficulties that user mode has (because of possibly offline CPUs) are an indicator that the kernel should expose this. If only to make the kernel's view clear in case there are situations where it got the calculation wrong. E.g. kernel booted with a maxcpus=N parameter.
-Tony
Reinette Chatre
maintainerRe: [PATCH v5 0/8] Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckHi Tony,
Quoted 9 lines · Tony Luck
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Same typo as in V4.
Quoted 4 lines
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
I have the same question as with V4 that was not answered in that email thread nor in this new version.
https://lore.kernel.org/lkml/[email protected]/
I stop my review of this series here.
Reinette
Tony Luck
maintainerRe: [PATCH v5 0/8] Add support for Sub-NUMA cluster (SNC) systems
Quoted 8 lines · Reinette Chatre
Hi Tony,
Quoted 5 lines · Tony Luck
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Quoted 6 lines
Quoted 4 lines
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Same typo as in V4.
Sorry. Will fix and re-post.
Quoted 8 lines
Quoted 4 lines
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
I have the same question as with V4 that was not answered in that email thread nor in this new version.
https://lore.kernel.org/lkml/[email protected]/
Non-SNC systems already have an issue when reporting memory bandwidth for a task that Linux may migrate the task to a CPU on a different node which means that logging for that task will also move to different files in the mon_data/mon_L3_*/ for the new node.
With SNC enabled, migration between NUMA nodes on the same socket may happen much more frequently because:
1) The CPUs on the other NUMA nodes in the socket are in the same Linux L3 cache domain. So Linux regard the migration as "cheap".
2) The ACPI SLIT table on SNC enabled systems may also report the latency for remote access to another NUMA node on the same socket as significantly lower than the latency for cross-socket access. On my test system the SLIT distance for same socket nodes is 0xC, compared to 0x15 for cross-socket distance. This will also lead to Linux being more likely to migrate a task to a CPU on another SNC NUMA node in the same socket.
To avoid migration issues, users may use sched_setaffinity(2) to bind tasks to the subset of CPUs that share an SNC NUMA node.
I can write this up in a new cover letter.
Quoted 1 line
I stop my review of this series here.
Reinette
Should I repost the whole series as v6 with the new cover letter. The only change to the patches so far is to the selftest reported by Shaopeng Tan[1].
-Tony
Reinette Chatre
maintainerRe: [PATCH v5 0/8] Add support for Sub-NUMA cluster (SNC) systems
Hi Tony,
Quoted 23 lines · Tony Luck
Quoted 8 lines · Reinette Chatre
Hi Tony,
Quoted 5 lines · Tony Luck
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Quoted 6 lines
Quoted 4 lines
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Same typo as in V4.
Sorry. Will fix and re-post.
Quoted 5 lines
Quoted 4 lines
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
Quoted 8 lines
Quoted 3 lines
I have the same question as with V4 that was not answered in that email thread nor in this new version.
https://lore.kernel.org/lkml/[email protected]/Non-SNC systems already have an issue when reporting memory bandwidth for a task that Linux may migrate the task to a CPU on a different node which means that logging for that task will also move to different files in the mon_data/mon_L3_*/ for the new node.
It is not obvious to me that this is an issue. From what I understand the data remains accurate.
How does this map to the earlier "may still not be counted accurately"?
Quoted 18 lines
With SNC enabled, migration between NUMA nodes on the same socket may happen much more frequently because:
1) The CPUs on the other NUMA nodes in the socket are in the same Linux L3 cache domain. So Linux regard the migration as "cheap".
2) The ACPI SLIT table on SNC enabled systems may also report the latency for remote access to another NUMA node on the same socket as significantly lower than the latency for cross-socket access. On my test system the SLIT distance for same socket nodes is0xC, compared to0x15for cross-socket distance. This will also lead to Linux being more likely to migrate a task to a CPU on another SNC NUMA node in the same socket.To avoid migration issues, users may use
sched_setaffinity(2)to bind tasks to the subset of CPUs that share an SNC NUMA node.I can write this up in a new cover letter.
Quoted 8 lines
Quoted 1 line
I stop my review of this series here.
Reinette
Should I repost the whole series as v6 with the new cover letter. The only change to the patches so far is to the selftest reported by Shaopeng Tan[1].
Is this an assurance that the cover letter in no way reflects how feedback was addressed in the rest of this series?
Reinette
Tony Luck
maintainerRe: [PATCH v5 0/8] Add support for Sub-NUMA cluster (SNC) systems
Quoted 35 lines · Reinette Chatre
Hi Tony,
Quoted 23 lines · Tony Luck
Quoted 8 lines · Reinette Chatre
Hi Tony,
Quoted 5 lines · Tony Luck
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Quoted 6 lines
Quoted 4 lines
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Same typo as in V4.
Sorry. Will fix and re-post.
Quoted 5 lines
Quoted 4 lines
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
Quoted 8 lines
Quoted 3 lines
I have the same question as with V4 that was not answered in that email thread nor in this new version.
https://lore.kernel.org/lkml/[email protected]/Non-SNC systems already have an issue when reporting memory bandwidth for a task that Linux may migrate the task to a CPU on a different node which means that logging for that task will also move to different files in the mon_data/mon_L3_*/ for the new node.
Quoted 5 lines
It is not obvious to me that this is an issue. From what I understand the data remains accurate.
How does this map to the earlier "may still not be counted accurately"?
Yes, the data is accurate. But a naive application reading the wrong files from mon_data will not see the accurate data. Without SNC users may only see this issue rarely as Linux tries hard to not migrate processes to other NUMA nodes or L3 cache domains. But with SNC enabled this is no longer the case. the ACPI SLIT distance of 0xC is below the threshold that Linux checks for "is the target CPU for a migration far away" so migration to other SNC nodes may be quite common.
I can move this out of the cover letter and provide guidance/warnings in the patch to Documentation/arch/x86/resctrl.rst
Quoted 23 lines
Quoted 23 lines
With SNC enabled, migration between NUMA nodes on the same socket may happen much more frequently because:
1) The CPUs on the other NUMA nodes in the socket are in the same Linux L3 cache domain. So Linux regard the migration as "cheap".
2) The ACPI SLIT table on SNC enabled systems may also report the latency for remote access to another NUMA node on the same socket as significantly lower than the latency for cross-socket access. On my test system the SLIT distance for same socket nodes is0xC, compared to0x15for cross-socket distance. This will also lead to Linux being more likely to migrate a task to a CPU on another SNC NUMA node in the same socket.To avoid migration issues, users may use
sched_setaffinity(2)to bind tasks to the subset of CPUs that share an SNC NUMA node.I can write this up in a new cover letter.
Quoted 1 line
I stop my review of this series here.
Reinette
Quoted 7 lines
Quoted 4 lines
Should I repost the whole series as v6 with the new cover letter. The only change to the patches so far is to the selftest reported by Shaopeng Tan[1].
Is this an assurance that the cover letter in no way reflects how feedback was addressed in the rest of this series?
My track record here is far from perfect. I believe I addressed all the issues you raised. But it's very possible that I may have missed some, or misintepreted the concerns you raised.
Quoted 2 lines
Reinette
-Tony
Reinette Chatre
maintainerRe: [PATCH v5 0/8] Add support for Sub-NUMA cluster (SNC) systems
Hi Tony,
Quoted 42 lines · Tony Luck
Quoted 35 lines · Reinette Chatre
Hi Tony,
Quoted 23 lines · Tony Luck
Quoted 8 lines · Reinette Chatre
Hi Tony,
Quoted 5 lines · Tony Luck
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Quoted 6 lines
Quoted 4 lines
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Same typo as in V4.
Sorry. Will fix and re-post.
Quoted 5 lines
Quoted 4 lines
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
Quoted 8 lines
Quoted 3 lines
I have the same question as with V4 that was not answered in that email thread nor in this new version.
https://lore.kernel.org/lkml/[email protected]/Non-SNC systems already have an issue when reporting memory bandwidth for a task that Linux may migrate the task to a CPU on a different node which means that logging for that task will also move to different files in the mon_data/mon_L3_*/ for the new node.
Quoted 5 lines
It is not obvious to me that this is an issue. From what I understand the data remains accurate.
How does this map to the earlier "may still not be counted accurately"?
Quoted 7 lines
Yes, the data is accurate. But a naive application reading the wrong files from
mon_datawill not see the accurate data. Without SNC users may only see this issue rarely as Linux tries hard to not migrate processes to other NUMA nodes or L3 cache domains. But with SNC enabled this is no longer the case. the ACPI SLIT distance of0xCis below the threshold that Linux checks for "is the target CPU for a migration far away" so migration to other SNC nodes may be quite common.
I would like to recommend that you take care not to present this work using uncertain terms like "may not be counted accurately" or "there will still be some anomalies". If I understand correctly there is no uncertainty. When that "naive application" reads the wrong files then I think it can be considered a usage error and should not be documented as an issue with the counters. I understand that the user space requirements are not obvious, and there should be guidance.
Quoted 2 lines
I can move this out of the cover letter and provide guidance/warnings in the patch to
Documentation/arch/x86/resctrl.rst
Yes, I do think this will be very helpful.
Quoted 29 lines
Quoted 29 lines
Quoted 28 lines
With SNC enabled, migration between NUMA nodes on the same socket may happen much more frequently because:
1) The CPUs on the other NUMA nodes in the socket are in the same Linux L3 cache domain. So Linux regard the migration as "cheap".
2) The ACPI SLIT table on SNC enabled systems may also report the latency for remote access to another NUMA node on the same socket as significantly lower than the latency for cross-socket access. On my test system the SLIT distance for same socket nodes is0xC, compared to0x15for cross-socket distance. This will also lead to Linux being more likely to migrate a task to a CPU on another SNC NUMA node in the same socket.To avoid migration issues, users may use
sched_setaffinity(2)to bind tasks to the subset of CPUs that share an SNC NUMA node.I can write this up in a new cover letter.
Quoted 1 line
I stop my review of this series here.
Reinette
Should I repost the whole series as v6 with the new cover letter. The only change to the patches so far is to the selftest reported by Shaopeng Tan[1].
Quoted 6 lines
Quoted 2 lines
Is this an assurance that the cover letter in no way reflects how feedback was addressed in the rest of this series?
My track record here is far from perfect. I believe I addressed all the issues you raised. But it's very possible that I may have missed some, or misintepreted the concerns you raised.
This is a familiar response from you that just puts the burden back on me to go dig out previous discussions. This will be the last time.
Reinette
Tony Luck
maintainerv6 cover Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckThe Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
---
Summary of changes since v5 - see each patch commit for more specifics
Rebased to v6.6-rc3
0001 Define "scope" enum with values 2, 3 for caches to simplify some
code (but sanity check before each such usage). Better warning messages when scope lookup fails
0002 New patch so that some code can be shared between looking up
control and monitor domains
0003 Spell "mondomains" as "mon_domains" and be consistent with all
the other "mon" identifiers also having similar "_". Don't leave control stuff with old names, change those too so now have ctrl_scope, ctrl_domains, etc.
0004 Use infrastructure from 0002 to have a common rdt_find_domain()
function for both types of domain structure. 0003 was using same "rdt_domain" structure for both control and monitor domains. Divide it into rdt_ctrl_domain and rdt_mon_domain structures with just the fields they need. Ditto for rdt_hw_domain. Also split and rename many support functions and macros.
Lots of "fir tree local declaration order" changes because lengths of typenames changed.
0005 Better commit description
0006 Better commit and code comments
0007 More explanations in commit and code comments.
Use consistent naming for "snc_*()" functions.
Patch to update selftests dropped from this series. Someone else has taken over that work.
Tony Luck (8):
x86/resctrl: Prepare for new domain scope x86/resctrl: Prepare to split rdt_domain structure x86/resctrl: Prepare for different scope for control/monitor
operations
x86/resctrl: Split the rdt_domain and rdt_hw_domain structures x86/resctrl: Add node-scope to the options for feature scope x86/resctrl: Introduce snc_nodes_per_l3_cache x86/resctrl: Sub NUMA Cluster detection and enable x86/resctrl: Update documentation with Sub-NUMA cluster changes
Documentation/arch/x86/resctrl.rst | 34 +- include/linux/resctrl.h | 78 +++-- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 66 ++-- arch/x86/kernel/cpu/resctrl/core.c | 380 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 52 +-- arch/x86/kernel/cpu/resctrl/monitor.c | 58 ++-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 14 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 131 ++++---- 9 files changed, 567 insertions(+), 247 deletions(-)
base-commit: 6465e260f48790807eef06b583b38ca9789b6072
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v6 1/8 x86/resctrl: Prepare for new domain scope
Resctrl resources operate on subsets of CPUs in the system with the defining attribute of each subset being an instance of a particular level of cache. E.g. all CPUs sharing an L3 cache would be part of the same domain.
In preparation for features that are scoped at the NUMA node level change the code from explicit references to "cache_level" to a more generic scope. At this point the only options for this scope are groups of CPUs that share an L2 cache or L3 cache.
Provide a more detailed warning message if a domain id cannot be found when adding a CPU. Just check and silent return if the domain id can't be found when removing a CPU.
No functional change.
---
Changes since v5:
1) Set enum values of RESCTRL_L2_CACHE and RESCTRL_L3_CACHE to "2" and "3 respectively so they can be passed to get_cpu_cacheinfo_id() (in code paths that check that scope is one of the cache types).
2) Simplified the check on scope in pseudo_lock_region_init() and rdtgroup_cbm_to_size().
3) Added detailed warning if the domain id for a CPU cannot be determined when adding a CPU. Simpler check on removal.
--- include/linux/resctrl.h | 9 +++++-- arch/x86/kernel/cpu/resctrl/core.c | 33 ++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 ++++- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 +++- 4 files changed, 43 insertions(+), 10 deletions(-)
include/linux/resctrl.h +7 -2
@@ -144,13 +144,18 @@ struct resctrl_membw {struct rdt_parse_data ;struct resctrl_schema ; +enum resctrl_scope { +RESCTRL_L2_CACHE =2 , +RESCTRL_L3_CACHE =3 , +}; +/** * struct rdt_resource - attributes of a resctrl resource * @rid: The index of the resource * @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @cache_level: Which cache level defines scope of this resource +* @scope: Scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. * @domains: All domains for this resource
@@ -168,7 +173,7 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -int cache_level; +enum resctrl_scope scope;struct resctrl_cache cache;struct resctrl_membw membw;struct list_head domains;
arch/x86/kernel/cpu/resctrl/core.c +27 -6
@@ -65,7 +65,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L3, .name = "L3", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_L3), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -79,7 +79,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L2, .name = "L2", - .cache_level = 2, + .scope = RESCTRL_L2_CACHE, .domains = domain_init(RDT_RESOURCE_L2), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -93,7 +93,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_MBA, .name = "MB", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_MBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -105,7 +105,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_SMBA, .name = "SMBA", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_SMBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -487,6 +487,19 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)return 0 ; } +static int get_domain_id_from_scope (int cpu,enum resctrl_scope scope) +{ +switch (scope) { +case RESCTRL_L2_CACHE : +case RESCTRL_L3_CACHE : +return get_cpu_cacheinfo_id (cpu, scope); +default : +break ; + } + +return -EINVAL ; +} +/* * domain_add_cpu - Add a cpu to a resource's domain list. *
@@ -502,12 +515,17 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)*/ static void domain_add_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct list_head *add_pos =NULL ;struct rdt_hw_domain *hw_dom;struct rdt_domain *d;int err; +if (id <0 ) { +pr_warn_once ("Can't find domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->scope, r->name); +return ; + } d =rdt_find_domain (r, id, &add_pos);if (IS_ERR (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
@@ -552,10 +570,13 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)static void domain_remove_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct rdt_hw_domain *hw_dom;struct rdt_domain *d; +if (id <0 ) +return ; + d =rdt_find_domain (r, id,NULL );if (IS_ERR_OR_NULL (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +5 -1
@@ -292,10 +292,14 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { +int scope = plr->s->res->scope;struct cpu_cacheinfo *ci;int ret;int i; +if (WARN_ON_ONCE (scope !=RESCTRL_L2_CACHE && scope !=RESCTRL_L3_CACHE )) +return -ENODEV ; +/* Pick the first cpu we find that is associated with the cache. */ plr->cpu =cpumask_first (&plr->d->cpu_mask);
@@ -311,7 +315,7 @@ static int pseudo_lock_region_init(struct pseudo_lock_region *plr) plr->size =rdtgroup_cbm_to_size (plr->s->res, plr->d, plr->cbm);for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == plr->s->res->cache_level) { +if (ci->info_list[i].level == scope) { plr->line_size = ci->info_list[i].coherency_line_size;return 0 ; }
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -1
@@ -1345,10 +1345,13 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,unsigned int size =0 ;int num_b, i; +if (WARN_ON_ONCE (r->scope !=RESCTRL_L2_CACHE && r->scope !=RESCTRL_L3_CACHE )) +return -EINVAL ; + num_b =bitmap_weight (&cbm, r->cache.cbm_len); ci =get_cpu_cacheinfo (cpumask_any (&d->cpu_mask));for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == r->cache_level) { +if (ci->info_list[i].level == r->scope) { size = ci->info_list[i].size / r->cache.cbm_len * num_b;break ; }
Signature · 1 line
2.41.0
Re: [PATCH v6 1/8] x86/resctrl: Prepare for new domain scope
Hi Tony,
Quoted 11 lines · Tony Luck
Resctrl resources operate on subsets of CPUs in the system with the defining attribute of each subset being an instance of a particular level of cache. E.g. all CPUs sharing an L3 cache would be part of the same domain.
In preparation for features that are scoped at the NUMA node level change the code from explicit references to "
cache_level" to a more generic scope. At this point the only options for this scope are groups of CPUs that share an L2 cache or L3 cache.
Quoted 5 lines
Provide a more detailed warning message if a domain id cannot be found when adding a CPU. Just check and silent return if the domain id can't be found when removing a CPU.
No functional change.
I see a number of diagnostic checks added below. Are you sure there's no functional change?
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +0
-0
arch/x86/kernel/cpu/resctrl/pseudo_lock.c static void pseudo_lock_region_clear(struct pseudo_lock_region *plr) · line 301
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3
-0@@ -292,10 +292,14 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { +int scope = plr->s->res->scope;struct cpu_cacheinfo *ci;int ret;int i; +if (WARN_ON_ONCE (scope !=RESCTRL_L2_CACHE && scope !=RESCTRL_L3_CACHE )) +return -ENODEV ;
Functional change?
arch/x86/kernel/cpu/resctrl/rdtgroup.c unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, · line 1349
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-0@@ -1345,10 +1345,13 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,unsigned int size =0 ;int num_b, i; +if (WARN_ON_ONCE (r->scope !=RESCTRL_L2_CACHE && r->scope !=RESCTRL_L3_CACHE )) +return -EINVAL ;
This function returns unsigned int. That's a huge region!
-Peter
Tony Luck
maintainerRe: [PATCH v6 1/8] x86/resctrl: Prepare for new domain scope
Quoted 14 lines · Peter Newman
Hi Tony,
Quoted 11 lines · Tony Luck
Resctrl resources operate on subsets of CPUs in the system with the defining attribute of each subset being an instance of a particular level of cache. E.g. all CPUs sharing an L3 cache would be part of the same domain.
In preparation for features that are scoped at the NUMA node level change the code from explicit references to "
cache_level" to a more generic scope. At this point the only options for this scope are groups of CPUs that share an L2 cache or L3 cache.
Quoted 8 lines
Quoted 5 lines
Provide a more detailed warning message if a domain id cannot be found when adding a CPU. Just check and silent return if the domain id can't be found when removing a CPU.
No functional change.
I see a number of diagnostic checks added below. Are you sure there's no functional change?
Those checks should be for "can't happen" cases where some "scope" that isn't attached to a cache_level somehow made its way down to a routine that is only expecting cache scoped.
But I'll remove the "No functional change" from the commit. comment.
Quoted 29 lines
Quoted 7 lines
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +0
-0arch/x86/kernel/cpu/resctrl/pseudo_lock.c static void pseudo_lock_region_clear(struct pseudo_lock_region *plr) · line 301
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3
-0@@ -292,10 +292,14 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { +int scope = plr->s->res->scope;struct cpu_cacheinfo *ci;int ret;int i; +if (WARN_ON_ONCE (scope !=RESCTRL_L2_CACHE && scope !=RESCTRL_L3_CACHE )) +return -ENODEV ;Functional change?
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0@@ -1345,10 +1345,13 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,unsigned int size =0 ;int num_b, i;
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, · line 1349
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-0+if (WARN_ON_ONCE (r->scope !=RESCTRL_L2_CACHE && r->scope !=RESCTRL_L3_CACHE )) +return -EINVAL ;This function returns unsigned int. That's a huge region!
I wondered for a bit what value to return for this "can't happen" case and initially went with an error code. But the function can already fail (in ways that won't happen) and in that case the return is "0".
I'l update to return 0 here too.
Quoted 1 line
-Peter
Thanks for the review.
-Tony
Tony Luck
maintainerPATCH v6 2/8 x86/resctrl: Prepare to split rdt_domain structure
In reply to Tony LuckThe rdt_domain structure is used for both control and monitor features. It is about to be split into separate structures for these two usages because the scope for control and monitoring features for a resource will be different for future resources.
To allow for common code that scans a list of domains looking for a specific domain id, move the "list" and "id" fields into their own structure within the rdt_domain structure.
---
Changes since v5
This is a new patch. Spawned from Reinette's review comment about my trying to mark some fields in the rdt_ctrl_domain and rdt_mon_domain structures as required to be the same.
This is a better solution that doesn't require that developers read and obey those comments.
--- include/linux/resctrl.h | 14 ++++++-- arch/x86/kernel/cpu/resctrl/core.c | 16 ++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 16 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 44 +++++++++++------------ 6 files changed, 51 insertions(+), 43 deletions(-)
include/linux/resctrl.h +11 -3
@@ -53,9 +53,18 @@ struct resctrl_staged_config { };/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_domain_hdr - common header for different domain types * @list: all instances of this resource * @id: unique id for this instance +*/ +struct rdt_domain_hdr { +struct list_head list; +int id; +}; + +/** +* struct rdt_domain - group of CPUs sharing a resctrl resource +* @hdr: common header for different domain types * @cpu_mask: which CPUs share this resource * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold * @mbm_total: saved state for MBM total bandwidth
@@ -71,8 +80,7 @@ struct resctrl_staged_config {* by closid */ struct rdt_domain { -struct list_head list; -int id; +struct rdt_domain_hdr hdr;struct cpumask cpu_mask;unsigned long *rmid_busy_llc;struct mbm_state *mbm_total;
arch/x86/kernel/cpu/resctrl/core.c +8 -8
@@ -352,7 +352,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) {/* Find the domain that contains this CPU */ if (cpumask_test_cpu (cpu, &d->cpu_mask))return d;
@@ -401,12 +401,12 @@ struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,return ERR_PTR (-ENODEV );list_for_each (l, &r->domains) { - d = list_entry(l, struct rdt_domain, list); + d = list_entry(l, struct rdt_domain, hdr.list);/* When id is found, return its domain. */ -if (id == d->id) +if (id == d->hdr.id)return d;/* Stop searching when finding id's position in sorted list. */ -if (id < d->id) +if (id < d->hdr.id)break ; }
@@ -544,7 +544,7 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; d = &hw_dom->d_resctrl; - d->id = id; + d->hdr.id = id;cpumask_set_cpu (cpu, &d->cpu_mask);rdt_domain_reconfigure_cdp (r);
@@ -559,11 +559,11 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; } -list_add_tail (&d->list, add_pos); +list_add_tail (&d->hdr.list, add_pos); err =resctrl_online_domain (r, d);if (err) { -list_del (&d->list); +list_del (&d->hdr.list);domain_free (hw_dom); } }
@@ -587,7 +587,7 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)cpumask_clear_cpu (cpu, &d->cpu_mask);if (cpumask_empty (&d->cpu_mask)) {resctrl_offline_domain (r, d); -list_del (&d->list); +list_del (&d->hdr.list);/* * rdt_domain "d" is going to be freed below, so clear
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +8 -8
@@ -67,7 +67,7 @@ int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -144,7 +144,7 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -224,8 +224,8 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, list) { -if (d->id == dom_id) { +list_for_each_entry (d, &r->domains, hdr.list) { +if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;if (r->parse_ctrlval (&data, s, d))
@@ -316,7 +316,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -464,7 +464,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, list) { +list_for_each_entry (dom, &r->domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -474,7 +474,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clo ctrl_val =resctrl_arch_get_config (r, dom, closid, schema->conf_type); -seq_printf (s, r->format_str, dom->id, max_data_width, +seq_printf (s, r->format_str, dom->hdr.id, max_data_width, ctrl_val); sep = true; }
@@ -503,7 +503,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }else {seq_printf (s,"%s:%d=%x\n" , rdtgrp->plr->s->res->name, - rdtgrp->plr->d->id, + rdtgrp->plr->d->hdr.id, rdtgrp->plr->cbm); } }else {
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -340,7 +340,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { err = resctrl_arch_rmid_read(r, d, entry->rmid, QOS_L3_OCCUP_EVENT_ID,
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1 -1
@@ -856,7 +856,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, list) { +list_for_each_entry (d_i, &r->domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, &d_i->cpu_mask);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +22 -22
168 lines not on this page · raw message
Signature · 1 line
2.41.0
Re: [PATCH v6 2/8] x86/resctrl: Prepare to split rdt_domain structure
Hi Tony,
Quoted 9 lines · Tony Luck
The
rdt_domainstructure is used for both control and monitor features. It is about to be split into separate structures for these two usages because the scope for control and monitoring features for a resource will be different for future resources.To allow for common code that scans a list of domains looking for a specific domain id, move the "list" and "id" fields into their own structure within the
rdt_domainstructure.
On this one I think you can say "No functional change"
Thanks!
-Peter
Tony Luck
maintainerPATCH v6 3/8 x86/resctrl: Prepare for different scope for control/monitor operations
In reply to Tony LuckResctrl assumes that control and monitor operations on a resource are performed at the same scope.
Prepare for systems that use different scope (specifically L3 scope for cache control and NODE scope for cache occupancy and memory bandwidth monitoring).
Create separate domain lists for control and monitor operations.
Note that errors during initialization of either control or monitor functions on a domain would previously result in that domain being excluded from both control and monitor operations. Now the domains are allocated independently it is no longer required to disable both control and monitor operations if either fail.
---
Changes since v5:
Commit comment: s/Existing resctrl assumes/Resctrl assumes/
Many new names. Put an underscore in "mon_domains" for consistency with "mon_scope". Do same with all the other "mon" changes.
Also rename "scope" to "ctrl_scope", "domains" to "ctrl_domains" and all the assocated functions and macros.
--- include/linux/resctrl.h | 18 +- arch/x86/kernel/cpu/resctrl/internal.h | 4 +- arch/x86/kernel/cpu/resctrl/core.c | 198 ++++++++++++++++------ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 12 +- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 54 +++--- 7 files changed, 200 insertions(+), 92 deletions(-)
include/linux/resctrl.h +12 -6
@@ -163,10 +163,12 @@ enum resctrl_scope {* @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @scope: Scope of this resource +* @ctrl_scope: Scope of this resource for control functions +* @mon_scope: Scope of this resource for monitor functions * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. -* @domains: All domains for this resource +* @ctrl_domains: Control domains for this resource +* @mon_domains: Monitor domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.
@@ -181,10 +183,12 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -enum resctrl_scope scope; +enum resctrl_scope ctrl_scope; +enum resctrl_scope mon_scope;struct resctrl_cache cache;struct resctrl_membw membw; -struct list_head domains; +struct list_head ctrl_domains; +struct list_head mon_domains;char *name;int data_width;u32 default_ctrl;
@@ -230,8 +234,10 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
arch/x86/kernel/cpu/resctrl/internal.h +2 -2
@@ -511,8 +511,8 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn);int rdtgroup_kn_mode_restrict (struct rdtgroup *r,const char *name);int rdtgroup_kn_mode_restore (struct rdtgroup *r,const char *name,umode_t mask); -struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id, -struct list_head **pos); +struct rdt_domain_hdr *rdt_find_domain (struct list_head *h,int id, +struct list_head **pos);ssize_t rdtgroup_schemata_write (struct kernfs_open_file *of,char *buf,size_t nbytes,loff_t off); int rdtgroup_schemata_show(struct kernfs_open_file *of,
arch/x86/kernel/cpu/resctrl/core.c +147 -51
315 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +7 -5
@@ -224,7 +224,7 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) {if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;
@@ -316,7 +316,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -464,7 +464,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, hdr.list) { +list_for_each_entry (dom, &r->ctrl_domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -540,6 +540,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,int rdtgroup_mondata_show (struct seq_file *m,void *arg) {struct kernfs_open_file *of = m->private; +struct rdt_domain_hdr *hdr;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;
@@ -560,11 +561,12 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) evtid = md.u.evtid; r = &rdt_resources_all[resid].r_resctrl; - d =rdt_find_domain (r, domid,NULL ); -if (IS_ERR_OR_NULL (d)) { + hdr =rdt_find_domain (&r->mon_domains, domid,NULL ); +if (IS_ERR_OR_NULL (hdr)) { ret = -ENOENT ; goto out; } + d = container_of(hdr, struct rdt_domain, hdr);mon_event_read (&rr, r, d, rdtgrp, evtid, false);
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -340,7 +340,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->mon_domains, hdr.list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { err = resctrl_arch_rmid_read(r, d, entry->rmid, QOS_L3_OCCUP_EVENT_ID,
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +2 -2
@@ -292,7 +292,7 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { -int scope = plr->s->res->scope; +int scope = plr->s->res->ctrl_scope;struct cpu_cacheinfo *ci;int ret;int i;
@@ -856,7 +856,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, hdr.list) { +list_for_each_entry (d_i, &r->ctrl_domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, &d_i->cpu_mask);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +29 -25
170 lines not on this page · raw message
Signature · 1 line
2.41.0
Re: [PATCH v6 3/8] x86/resctrl: Prepare for different scope for control/monitor operations
Hi Tony,
arch/x86/kernel/cpu/resctrl/core.c struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) · line 358
Quoted 7 lines · Tony Luck
arch/x86/kernel/cpu/resctrl/core.c +1
-1@@ -352,7 +355,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d; -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) {
If someone were to call get_domain_from_cpu() looking for a mon_domain, I don't think they'd be happy with the result.
This problem seems adequately addressed by the next patch where a type mismatch on the return value would result.
In any case, perhaps the name could be updated to set expectations better.
Quoted 15 lines
arch/x86/kernel/cpu/resctrl/core.c +5
-2@@ -549,44 +552,101 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)rdt_domain_reconfigure_cdp (r); -if (r->alloc_capable &&domain_setup_ctrlval (r, d)) { +if (domain_setup_ctrlval (r, d)) {domain_free (hw_dom);return ; } -if (r->mon_capable &&arch_domain_mbm_alloc (r->num_rmid, hw_dom)) { +list_add_tail (&d->hdr.list, add_pos); + + err =resctrl_online_ctrl_domain (r, d); +if (err) {
arch/x86/kernel/cpu/resctrl/core.c static void domain_add_cpu(int cpu, struct rdt_resource *r) · line 573
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +9
-0+list_del (&d->hdr.list);domain_free (hw_dom); + } +} + +static void domain_add_cpu_mon (int cpu,struct rdt_resource *r) +{ +int id =get_domain_id_from_scope (cpu, r->mon_scope); +struct list_head *add_pos =NULL ; +struct rdt_hw_domain *hw_mondom;
It's still hw_dom in domain_add_cpu_ctrl(), so why hw_mondom here?
Quoted 11 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1@@ -3711,16 +3711,16 @@ static void domain_destroy_mon_state(struct rdt_domain *d)kfree (d->mbm_local); } -void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d) +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d) {lockdep_assert_held (&rdtgroup_mutex);if (supports_mba_mbps () && r->rid ==RDT_RESOURCE_MBA )mba_sc_domain_destroy (r, d);
arch/x86/kernel/cpu/resctrl/rdtgroup.c static void domain_destroy_mon_state(struct rdt_domain *d) · line 3726
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +3
-3+} -if (!r->mon_capable) -return ; - +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d) +{/* * If resctrl is mounted, remove all the * per domain monitor data directories.
We did a lockdep_assert_held(&rdtgroup_mutex) for both types before. Should we continue to do so here?
Quoted 3 lines
--
2.41.0
In the resctrl2 prototype I complained that resctrl_resource was awkwardly disjoint in its support for control and monitoring groups[1]. In this patch, you seem to have already done most of the hard work in separating the control and monitoring functionality, so taking the next step and using a different structure to represent control and monitoring resources would further improve the code by statically typing monitoring and control resources, which would be less error-prone than run-time checks on the alloc_capable and mon_capable fields, which seem easy to forget.
I don't think this is necessary to complete SNC support, but it would give me confidence that there isn't a misplaced {alloc,mon}_capable check resulting in the wrong domain list being traversed. I will probably do this myself later if you don't.
Thanks!
-Peter
[1] https://lore.kernel.org/all/CALPaoCj_oa=nATvOO_uysYvu+PdTQtd0pvssvm9_M1+fP-Z8JA@mail.gmail.com/
Tony Luck
maintainerRe: [PATCH v6 3/8] x86/resctrl: Prepare for different scope for control/monitor operations
Quoted 8 lines · Peter Newman
Hi Tony,
Quoted 5 lines · Tony Luck
arch/x86/kernel/cpu/resctrl/core.c +0
-0@@ -352,7 +355,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d;
Quoted 5 lines
Review struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) · line 358
Quoted 2 lines
(quoted diff) +1
-1- list_for_each_entry(d, &r->domains, hdr.list) { + list_for_each_entry(d, &r->ctrl_domains, hdr.list) {If someone were to call
get_domain_from_cpu()looking for amon_domain, I don't think they'd be happy with the result.
Indeed not. Type checking in "C" doesn't seem adequate to address this (when using "container_of()" which blindly trusts the user provided the right tyep/fieldname). I'm using the rdt_domain_hdr.type field to provide necessary checks.
Quoted 26 lines
This problem seems adequately addressed by the next patch where a type mismatch on the return value would result.
In any case, perhaps the name could be updated to set expectations better.
Quoted 19 lines
arch/x86/kernel/cpu/resctrl/core.c +8
-2@@ -549,44 +552,101 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)rdt_domain_reconfigure_cdp (r); -if (r->alloc_capable &&domain_setup_ctrlval (r, d)) { +if (domain_setup_ctrlval (r, d)) {domain_free (hw_dom);return ; } -if (r->mon_capable &&arch_domain_mbm_alloc (r->num_rmid, hw_dom)) { +list_add_tail (&d->hdr.list, add_pos); + + err =resctrl_online_ctrl_domain (r, d); +if (err) { +list_del (&d->hdr.list);domain_free (hw_dom); + } +}
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/core.c static void domain_add_cpu(int cpu, struct rdt_resource *r) · line 573
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/core.c +6
-0+ +static void domain_add_cpu_mon (int cpu,struct rdt_resource *r) +{ +int id =get_domain_id_from_scope (cpu, r->mon_scope); +struct list_head *add_pos =NULL ; +struct rdt_hw_domain *hw_mondom;It's still
hw_domindomain_add_cpu_ctrl(), so whyhw_mondomhere?
No good reason. I'll change it to "hw_dom".
Quoted 15 lines
Quoted 15 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-1@@ -3711,16 +3711,16 @@ static void domain_destroy_mon_state(struct rdt_domain *d)kfree (d->mbm_local); } -void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d) +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d) {lockdep_assert_held (&rdtgroup_mutex);if (supports_mba_mbps () && r->rid ==RDT_RESOURCE_MBA )mba_sc_domain_destroy (r, d); +}
Quoted 11 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c static void domain_destroy_mon_state(struct rdt_domain *d) · line 3726
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-3-if (!r->mon_capable) -return ; - +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d) +{/* * If resctrl is mounted, remove all the * per domain monitor data directories. We did a
lockdep_assert_held(&rdtgroup_mutex)for both types before. Should we continue to do so here?
Yes. Added it.
Quoted 16 lines
Quoted 5 lines
--
2.41.0In the resctrl2 prototype I complained that
resctrl_resourcewas awkwardly disjoint in its support for control and monitoring groups[1]. In this patch, you seem to have already done most of the hard work in separating the control and monitoring functionality, so taking the next step and using a different structure to represent control and monitoring resources would further improve the code by statically typing monitoring and control resources, which would be less error-prone than run-time checks on thealloc_capableandmon_capablefields, which seem easy to forget.
Quoted 4 lines
I don't think this is necessary to complete SNC support, but it would give me confidence that there isn't a misplaced {alloc,mon}_capable check resulting in the wrong domain list being traversed. I will probably do this myself later if you don't.
Simple change. It's split between previous patch to add the field and current patch to initialize and check.
Quoted 5 lines
Thanks!
-Peter[1] https://lore.kernel.org/all/CALPaoCj_oa=nATvOO_uysYvu+PdTQtd0pvssvm9_M1+fP-Z8JA@mail.gmail.com/
Thanks
-Tony
Tony Luck
maintainerPATCH v6 4/8 x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
In reply to Tony LuckThe same rdt_domain structure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.
Split into separate rdt_ctrl_domain and rdt_mon_domain structures with just the fields required for control and monitoring respectively.
Similar split of the rdt_hw_domain structure into rdt_hw_ctrl_domain and rdt_hw_mon_domain.
---
Changes since v5:
Make rdt_find_domain() work on either control or monitor domains using infrastructure setup in previous patch to have a common header stucture in each.
Don't use a field paramter in the domain_init() macro, just provide separate "ctrl_domain_init()" and "mon_domain_init()" versions.
Improve error messages if domain_add_cpu_{ctrl,mon}() fail to locate a domain when adding a CPU.
Re-order local variable declarations to maintain reverse fir tree pattern in functions when the name changes in this patch broke the pattern.
Moved the comment describing how domain lists are ordered that used to be in front of domain_add_cpu() to rdt_find_domain() which is doing the majority of this work.
Dropped two blank lines that don't belong.
--- include/linux/resctrl.h | 50 +++++++------ arch/x86/kernel/cpu/resctrl/internal.h | 60 ++++++++++------ arch/x86/kernel/cpu/resctrl/core.c | 87 ++++++++++++----------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 32 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 40 +++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++-------- 7 files changed, 184 insertions(+), 153 deletions(-)
include/linux/resctrl.h +30 -20
@@ -63,7 +63,25 @@ struct rdt_domain_hdr { };/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_ctrl_domain - group of CPUs sharing a resctrl control resource +* @hdr: common header for different domain types +* @cpu_mask: which CPUs share this resource +* @plr: pseudo-locked region (if any) associated with domain +* @staged_config: parsed configuration to be applied +* @mbps_val: When mba_sc is enabled, this holds the array of user +* specified control values for mba_sc in MBps, indexed +* by closid +*/ +struct rdt_ctrl_domain { +struct rdt_domain_hdr hdr; +struct cpumask cpu_mask; +struct pseudo_lock_region *plr; +struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; +u32 *mbps_val; +}; + +/** +* struct rdt_mon_domain - group of CPUs sharing a resctrl control resource * @hdr: common header for different domain types * @cpu_mask: which CPUs share this resource * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold
@@ -73,13 +91,8 @@ struct rdt_domain_hdr {* @cqm_limbo: worker to periodically read CQM h/w counters * @mbm_work_cpu: worker CPU for MBM h/w counters * @cqm_work_cpu: worker CPU for CQM h/w counters -* @plr: pseudo-locked region (if any) associated with domain -* @staged_config: parsed configuration to be applied -* @mbps_val: When mba_sc is enabled, this holds the array of user -* specified control values for mba_sc in MBps, indexed -* by closid */ -struct rdt_domain { +struct rdt_mon_domain {struct rdt_domain_hdr hdr;struct cpumask cpu_mask;unsigned long *rmid_busy_llc;
@@ -89,9 +102,6 @@ struct rdt_domain {struct delayed_work cqm_limbo;int mbm_work_cpu;int cqm_work_cpu; -struct pseudo_lock_region *plr; -struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; -u32 *mbps_val; };/**
@@ -195,7 +205,7 @@ struct rdt_resource {const char *format_str;int (*parse_ctrlval)(struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d); +struct rdt_ctrl_domain *d);struct list_head evt_list;unsigned long fflags;bool cdp_capable;
@@ -229,15 +239,15 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);* Update the ctrl_val and apply this config right now. * Must be called on one of the domain's CPUs. */ -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val); -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
@@ -253,7 +263,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_domain *d);* Return: * 0 on success, or -EIO, -EINVAL etc on error. */ -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val);/**
@@ -266,7 +276,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid);/**
@@ -278,7 +288,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d);extern unsigned int resctrl_rmid_realloc_threshold;extern unsigned int resctrl_rmid_realloc_limit;
arch/x86/kernel/cpu/resctrl/internal.h +38 -22
131 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/core.c +46 -41
246 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +16 -16
@@ -58,7 +58,7 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r) }int parse_bw (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct resctrl_staged_config *cfg;u32 closid = data->rdtgrp->closid;
@@ -135,7 +135,7 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)* resource type. */ int parse_cbm (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct rdtgroup *rdtgrp = data->rdtgrp;struct resctrl_staged_config *cfg;
@@ -205,7 +205,7 @@ static int parse_line(char *line, struct resctrl_schema *s,struct rdt_resource *r = s->res;struct rdt_parse_data data;char *dom =NULL , *id; -struct rdt_domain *d; +struct rdt_ctrl_domain *d;unsigned long dom_id; if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
@@ -265,11 +265,11 @@ static u32 get_config_index(u32 closid, enum resctrl_conf_type type) } } -static bool apply_config (struct rdt_hw_domain *hw_dom, +static bool apply_config (struct rdt_hw_ctrl_domain *hw_dom,struct resctrl_staged_config *cfg,u32 idx,cpumask_var_t cpu_mask) { -struct rdt_domain *dom = &hw_dom->d_resctrl; +struct rdt_ctrl_domain *dom = &hw_dom->d_resctrl;if (cfg->new_ctrl != hw_dom->ctrl_val[idx]) {cpumask_set_cpu (cpumask_any (&dom->cpu_mask), cpu_mask);
@@ -281,11 +281,11 @@ static bool apply_config(struct rdt_hw_domain *hw_dom,return false; } -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val) { +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);u32 idx =get_config_index (closid, t);struct msr_param msr_param;
@@ -305,11 +305,11 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,int resctrl_arch_update_domains (struct rdt_resource *r,u32 closid) {struct resctrl_staged_config *cfg; -struct rdt_hw_domain *hw_dom; +struct rdt_hw_ctrl_domain *hw_dom;struct msr_param msr_param;enum resctrl_conf_type t; +struct rdt_ctrl_domain *d;cpumask_var_t cpu_mask; -struct rdt_domain *d;u32 idx;if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))
@@ -317,7 +317,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid) msr_param.res =NULL ;list_for_each_entry (d, &r->ctrl_domains, hdr.list) { - hw_dom =resctrl_to_arch_dom (d); + hw_dom =resctrl_to_arch_ctrl_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];if (!cfg->have_new_ctrl)
@@ -447,10 +447,10 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,return ret ?: nbytes; } -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);u32 idx =get_config_index (closid, type);return hw_dom->ctrl_val[idx];
@@ -459,7 +459,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,static void show_doms (struct seq_file *s,struct resctrl_schema *schema,int closid) {struct rdt_resource *r = schema->res; -struct rdt_domain *dom; +struct rdt_ctrl_domain *dom;bool sep = false;u32 ctrl_val;
@@ -521,7 +521,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }void mon_event_read (struct rmid_read *rr,struct rdt_resource *r, -struct rdt_domain *d,struct rdtgroup *rdtgrp, +struct rdt_mon_domain *d,struct rdtgroup *rdtgrp,int evtid,int first) {/*
@@ -541,11 +541,11 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) {struct kernfs_open_file *of = m->private;struct rdt_domain_hdr *hdr; +struct rdt_mon_domain *d;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;union mon_data_bits md; -struct rdt_domain *d;struct rmid_read rr;int ret =0 ;
@@ -566,7 +566,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) ret = -ENOENT ; goto out; } - d = container_of(hdr, struct rdt_domain, hdr); + d = container_of(hdr, struct rdt_mon_domain, hdr);mon_event_read (&rr, r, d, rdtgrp, evtid, false);
arch/x86/kernel/cpu/resctrl/monitor.c +20 -20
@@ -170,7 +170,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)return 0 ; } -static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_domain *hw_dom, +static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_mon_domain *hw_dom,u32 rmid,enum resctrl_event_id eventid) {
@@ -189,10 +189,10 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom,return NULL ; } -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct arch_mbm_state *am; am =get_arch_mbm_state (hw_dom, rmid, eventid);
@@ -208,9 +208,9 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* Assumes that hardware counters are also reset and thus that there is * no need to record initial non-zero counts. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d) +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);if (is_mbm_total_enabled ()) memset(hw_dom->arch_mbm_total, 0,
@@ -229,11 +229,11 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)return chunks >> shift; } -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);struct arch_mbm_state *am;u64 msr_val, chunks;int ret;
@@ -266,7 +266,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* decrement the count. If the busy count gets to zero on an RMID, we * free the RMID */ -void __check_limbo (struct rdt_domain *d,bool force_free) +void __check_limbo (struct rdt_mon_domain *d,bool force_free) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rmid_entry *entry;
@@ -305,7 +305,7 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } } -bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d) +bool has_busy_rmid (struct rdt_resource *r,struct rdt_mon_domain *d) {return find_first_bit (d->rmid_busy_llc, r->num_rmid) != r->num_rmid; }
@@ -334,7 +334,7 @@ int alloc_rmid(void)static void add_rmid_to_limbo (struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; -struct rdt_domain *d; +struct rdt_mon_domain *d;int cpu, err;u64 val =0 ;
@@ -383,7 +383,7 @@ void free_rmid(u32 rmid)list_add_tail (&entry->list, &rmid_free_lru); } -static struct mbm_state *get_mbm_state (struct rdt_domain *d,u32 rmid, +static struct mbm_state *get_mbm_state (struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id evtid) {switch (evtid) {
@@ -516,13 +516,13 @@ void mon_event_count(void *info)* throttle MSRs already have low percentage values. To avoid * unnecessarily restricting such rdtgroups, we also increase the bandwidth. */ -static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_domain *dom_mbm) +static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_mon_domain *dom_mbm) {u32 closid, rmid, cur_msr_val, new_msr_val;struct mbm_state *pmbm_data, *cmbm_data; +struct rdt_ctrl_domain *dom_mba;u32 cur_bw, delta_bw, user_bw;struct rdt_resource *r_mba; -struct rdt_domain *dom_mba;struct list_head *head;struct rdtgroup *entry;
@@ -600,7 +600,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) } } -static void mbm_update (struct rdt_resource *r,struct rdt_domain *d,int rmid) +static void mbm_update (struct rdt_resource *r,struct rdt_mon_domain *d,int rmid) {struct rmid_read rr;
@@ -640,13 +640,13 @@ void cqm_handle_limbo(struct work_struct *work) {unsigned long delay =msecs_to_jiffies (CQM_LIMBOCHECK_INTERVAL );int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex); r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, cqm_limbo.work); + d = container_of(work, struct rdt_mon_domain, cqm_limbo.work);__check_limbo (d, false);
@@ -656,7 +656,7 @@ void cqm_handle_limbo(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms) +void cqm_setup_limbo_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
@@ -672,9 +672,9 @@ void mbm_handle_overflow(struct work_struct *work)unsigned long delay =msecs_to_jiffies (MBM_OVERFLOW_INTERVAL );struct rdtgroup *prgrp, *crgrp;int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct list_head *head;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex);
@@ -682,7 +682,7 @@ void mbm_handle_overflow(struct work_struct *work) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, mbm_over.work); + d = container_of(work, struct rdt_mon_domain, mbm_over.work);list_for_each_entry (prgrp, &rdt_all_groups, rdtgroup_list) {mbm_update (r, d, prgrp->mon.rmid);
@@ -701,7 +701,7 @@ void mbm_handle_overflow(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms) +void mbm_setup_overflow_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -3
@@ -814,7 +814,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)* Return: true if @cbm overlaps with pseudo-locked region on @d, false * otherwise. */ -bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,unsigned long cbm) +bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_ctrl_domain *d,unsigned long cbm) {unsigned int cbm_len;unsigned long cbm_b;
@@ -841,11 +841,11 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm* if it is not possible to test due to memory allocation issue, * false otherwise. */ -bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d) +bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_ctrl_domain *d) { +struct rdt_ctrl_domain *d_i;cpumask_var_t cpu_with_psl;struct rdt_resource *r; -struct rdt_domain *d_i;bool ret = false;if (!zalloc_cpumask_var (&cpu_with_psl,GFP_KERNEL ))
arch/x86/kernel/cpu/resctrl/rdtgroup.c +31 -31
252 lines not on this page · raw message
Signature · 1 line
2.41.0
Re: [PATCH v6 4/8] x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
Hi Tony,
Quoted 5 lines · Tony Luck
/**
- * structrdt_hw_domain- Arch private attributes of a set of CPUs that share
- * a resource
+ * structrdt_hw_ctrl_domain- Arch private attributes of a set of CPUs that share
+ * a resource for a control function
wrapped line not quite aligned anymore
arch/x86/kernel/cpu/resctrl/internal.h struct arch_mbm_state { · line 336
Quoted 13 lines
* @d_resctrl: Properties exposed to the resctrl file system
* @ctrl_val: array of cache or mem ctrl values (indexed by CLOSID)
+ *
+ * Members of this structure are accessed via helpers that provide abstraction.
+ */arch/x86/kernel/cpu/resctrl/internal.h +8
-0+struct rdt_hw_ctrl_domain { +struct rdt_ctrl_domain d_resctrl; +u32 *ctrl_val; +}; + +/** +* struct rdt_hw_mon_domain - Arch private attributes of a set of CPUs that share +* a resource for a monitor function
wrapped line not quite aligned anymore
Quoted 3 lines
--
2.41.0
Tony Luck
maintainerRe: [PATCH v6 4/8] x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
Quoted 10 lines · Peter Newman
Hi Tony,
Quoted 5 lines · Tony Luck
/**
- * structrdt_hw_domain- Arch private attributes of a set of CPUs that share
- * a resource
+ * structrdt_hw_ctrl_domain- Arch private attributes of a set of CPUs that share
+ * a resource for a control functionwrapped line not quite aligned anymore
Fixed
Quoted 16 lines
Quoted 6 lines
* @d_resctrl: Properties exposed to the resctrl file system
* @ctrl_val: array of cache or mem ctrl values (indexed by CLOSID)
+ *
+ * Members of this structure are accessed via helpers that provide abstraction.
+ */arch/x86/kernel/cpu/resctrl/internal.h struct arch_mbm_state { · line 336
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/internal.h +8
-0+struct rdt_hw_ctrl_domain { +struct rdt_ctrl_domain d_resctrl; +u32 *ctrl_val; +}; + +/** +* struct rdt_hw_mon_domain - Arch private attributes of a set of CPUs that share +* a resource for a monitor function wrapped line not quite aligned anymore
Fixed
Quoted 7 lines
Quoted 5 lines
--
2.41.0Reviewed-by Peter Newman <[email protected]>
Thanks
-Tony
Tony Luck
maintainerPATCH v6 5/8 x86/resctrl: Add node-scope to the options for feature scope
In reply to Tony LuckCurrently supported resctrl features are all domain scoped the same as the scope of the L2 or L3 caches.
Add RESCTRL_NODE as a new option for features that are scoped at the same granularity as NUMA nodes. This is needed for Intel's Sub-NUMA Cluster (SNC) feature where monitoring features are node scoped.
---
Changes since v5:
Updates to commit message.
include/linux/resctrl.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 2 ++ 2 files changed, 3 insertions(+)
include/linux/resctrl.h +1 -0
@@ -165,6 +165,7 @@ struct resctrl_schema;enum resctrl_scope {RESCTRL_L2_CACHE =2 ,RESCTRL_L3_CACHE =3 , +RESCTRL_NODE , };/**
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -511,6 +511,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)case RESCTRL_L2_CACHE :case RESCTRL_L3_CACHE :return get_cpu_cacheinfo_id (cpu, scope); +case RESCTRL_NODE : +return cpu_to_node (cpu);default :break ; }
Signature · 1 line
2.41.0
Re: [PATCH v6 5/8] x86/resctrl: Add node-scope to the options for feature scope
Hi Tony,
Quoted 45 lines · Tony Luck
Currently supported resctrl features are all domain scoped the same as the scope of the L2 or L3 caches.
Add
RESCTRL_NODEas a new option for features that are scoped at the same granularity as NUMA nodes. This is needed for Intel's Sub-NUMA Cluster (SNC) feature where monitoring features are node scoped.Signed-off-by Tony Luck <[email protected]>---
Changes since v5:
Updates to commit message.
include/linux/resctrl.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 2 ++ 2 files changed, 3 insertions(+)include/linux/resctrl.h +1
-0@@ -165,6 +165,7 @@ struct resctrl_schema;enum resctrl_scope {RESCTRL_L2_CACHE =2 ,RESCTRL_L3_CACHE =3 , +RESCTRL_NODE , };/** arch/x86/kernel/cpu/resctrl/core.c +2
-1@@ -511,6 +511,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)case RESCTRL_L2_CACHE :case RESCTRL_L3_CACHE :return get_cpu_cacheinfo_id (cpu, scope); +case RESCTRL_NODE : +return cpu_to_node (cpu);default :break ; } --
Quoted 2 lines
2.41.0
Looks fine.
Tony Luck
maintainerPATCH v6 6/8 x86/resctrl: Introduce snc_nodes_per_l3_cache
In reply to Tony LuckIntel Sub-NUMA Cluster (SNC) is a feature that subdivides the CPU cores and memory controllers on a socket into two or more groups. These are presented to the operating system as NUMA nodes.
This may enable some workloads to have slightly lower latency to memory as the memory controller(s) in an SNC node are electrically closer to the CPU cores on that SNC node. This cost may be offset by lower bandwidth since the memory accesses for each core can only be interleaved between the memory controllers on the same SNC node.
Resctrl monitoring on Intel system depends upon attaching RMIDs to tasks to track L3 cache occupancy and memory bandwidth. There is an MSR that controls how the RMIDs are shared between SNC nodes.
The default mode divides them numerically. E.g. when there are two SNC nodes on a socket the lower number half of the RMIDs are given to the first node, the remainder to the second node. This would be difficult to use with the Linux resctrl interface as specific RMID values assigned to resctrl groups are not visible to users.
The other mode divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
Even with this redumbering SNC mode requires several changes in resctrl behavior for correct operation.
Add a global integer "snc_nodes_per_l3_cache" that will show how many SNC nodes share each L3 cache. When this is "1", SNC mode is either not implemented, or not enabled.
A later patch will detect SNC mode and set snc_nodes_per_l3_cache to the appropriate value. For now it remains at the default "1" to indicate SNC mode is not active.
Code that needs to take action when SNC is enabled is:
1) The number of logical RMIDs per L3 cache available for use is the number of physical RMIDs divided by the number of SNC nodes.
2) Likewise the "mon_scale" value must be adjusted for the number of SNC nodes.
3) The RMID renumbering operates when using the value from the IA32_PQR_ASSOC MSR to count accesses by a task. When reading an RMID counter, code must adjust from the logical RMID used to the physical RMID value for the SNC node that it wishes to read and load the adjusted value into the IA32_QM_EVTSEL MSR.
4) The L3 cache is divided between the SNC nodes. So the value reported in the resctrl "size" file is adjusted.
5) The "-o mba_MBps" mount option must be disabled in SNC mode because the monitoring is being done per SNC node, while the bandwidth allocation is still done at the L3 cache scope. Trying to use this feedback loop might result in contradictory changes to the throttling level coming from each of the SNC node bandwidth measurements.
---
Changes since v5:
Major overhaul to the commit message. Starts with high level overview of what SNC is, before going into details on changes needed.
Begin with definiton of the SNC acronym.
Clarify in point "1" that available RMIDs are per L3 cache.
Add extra detail in "5" why mba_MBps is incompatible with SNC mode.
Code changes:
Reformat a comment to use longer lines.
Added a period at end of sentence for a comment.
--- arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/core.c | 6 ++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 16 +++++++++++++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 4 ++-- 4 files changed, 23 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -446,6 +446,8 @@ DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);extern struct dentry *debugfs_resctrl; +extern int snc_nodes_per_l3_cache; +enum resctrl_res_level {RDT_RESOURCE_L3 ,RDT_RESOURCE_L2 ,
arch/x86/kernel/cpu/resctrl/core.c +6 -0
@@ -48,6 +48,12 @@ int max_name_width, max_data_width;*/ bool rdt_alloc_capable; +/* +* Number of SNC nodes that share each L3 cache. Default is 1 for +* systems that do not support SNC, or have SNC disabled. +*/ +int snc_nodes_per_l3_cache =1 ; +static void mba_wrmsr_intel (struct rdt_ctrl_domain *d,struct msr_param *m,struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/monitor.c +13 -3
@@ -148,8 +148,18 @@ static inline struct rmid_entry *__rmid_entry(u32 rmid)static int __rmid_read (u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +int cpu =smp_processor_id (); +int rmid_offset =0 ;u64 msr_val; +/* +* When SNC mode is on, need to compute the offset to read the +* physical RMID counter for the node to which this CPU belongs. +*/ +if (snc_nodes_per_l3_cache >1 ) + rmid_offset = (cpu_to_node (cpu) % snc_nodes_per_l3_cache) * r->num_rmid; +/* * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured * with a valid event code for supported resource type and the bits
@@ -158,7 +168,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)* IA32_QM_CTR.Error (bit 63) and IA32_QM_CTR.Unavailable (bit 62) * are error bits. */ -wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid); +wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid + rmid_offset);rdmsrl (MSR_IA32_QM_CTR , msr_val);if (msr_val &RMID_VAL_ERROR )
@@ -783,8 +793,8 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)int ret; resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size *1024 ; - hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale; - r->num_rmid = boot_cpu_data.x86_cache_max_rmid +1 ; + hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache; + r->num_rmid = (boot_cpu_data.x86_cache_max_rmid +1 ) / snc_nodes_per_l3_cache; hw_res->mbm_width =MBM_CNTR_WIDTH_BASE ;if (mbm_offset >0 && mbm_offset <=MBM_CNTR_WIDTH_OFFSET_MAX )
arch/x86/kernel/cpu/resctrl/rdtgroup.c +2 -2
@@ -1357,7 +1357,7 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, } } -return size; +return size / snc_nodes_per_l3_cache; }/**
@@ -2590,7 +2590,7 @@ static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param) ctx->enable_cdpl2 = true;return 0 ;case Opt_mba_mbps: -if (!supports_mba_mbps ()) +if (!supports_mba_mbps () || snc_nodes_per_l3_cache >1 )return -EINVAL ; ctx->enable_mba_mbps = true;return 0 ;
Signature · 1 line
2.41.0
Re: [PATCH v6 6/8] x86/resctrl: Introduce snc_nodes_per_l3_cache
Hi Tony,
Quoted 21 lines · Tony Luck
Intel Sub-NUMA Cluster (SNC) is a feature that subdivides the CPU cores and memory controllers on a socket into two or more groups. These are presented to the operating system as NUMA nodes.
This may enable some workloads to have slightly lower latency to memory as the memory controller(s) in an SNC node are electrically closer to the CPU cores on that SNC node. This cost may be offset by lower bandwidth since the memory accesses for each core can only be interleaved between the memory controllers on the same SNC node.
Resctrl monitoring on Intel system depends upon attaching RMIDs to tasks to track L3 cache occupancy and memory bandwidth. There is an MSR that controls how the RMIDs are shared between SNC nodes.
The default mode divides them numerically. E.g. when there are two SNC nodes on a socket the lower number half of the RMIDs are given to the first node, the remainder to the second node. This would be difficult to use with the Linux resctrl interface as specific RMID values assigned to resctrl groups are not visible to users.
Quoted 5 lines
The other mode divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
Even with this redumbering SNC mode requires several changes in resctrl behavior for correct operation.
redumbering? Harsh.
arch/x86/kernel/cpu/resctrl/rdtgroup.c unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, · line 1360
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1@@ -1357,7 +1357,7 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, } } -return size; +return size / snc_nodes_per_l3_cache;
To confirm, the size represented by a bit goes down rather than the CBM mask shrinking in each sub-NUMA domain?
I would maybe have expected the CBM mask to already be allocating at the smallest granularity the hardware supports.
arch/x86/kernel/cpu/resctrl/rdtgroup.c static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param) · line 2593
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1}/** @@ -2590,7 +2590,7 @@ static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param) ctx->enable_cdpl2 = true;return 0 ;case Opt_mba_mbps: -if (!supports_mba_mbps ()) +if (!supports_mba_mbps () || snc_nodes_per_l3_cache >1 )
Factor into supports_mba_mbps()?
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-1return -EINVAL ; ctx->enable_mba_mbps = true;return 0 ; --2.41.0
Tony Luck
maintainerRe: [PATCH v6 6/8] x86/resctrl: Introduce snc_nodes_per_l3_cache
Quoted 24 lines · Peter Newman
Hi Tony,
Quoted 21 lines · Tony Luck
Intel Sub-NUMA Cluster (SNC) is a feature that subdivides the CPU cores and memory controllers on a socket into two or more groups. These are presented to the operating system as NUMA nodes.
This may enable some workloads to have slightly lower latency to memory as the memory controller(s) in an SNC node are electrically closer to the CPU cores on that SNC node. This cost may be offset by lower bandwidth since the memory accesses for each core can only be interleaved between the memory controllers on the same SNC node.
Resctrl monitoring on Intel system depends upon attaching RMIDs to tasks to track L3 cache occupancy and memory bandwidth. There is an MSR that controls how the RMIDs are shared between SNC nodes.
The default mode divides them numerically. E.g. when there are two SNC nodes on a socket the lower number half of the RMIDs are given to the first node, the remainder to the second node. This would be difficult to use with the Linux resctrl interface as specific RMID values assigned to resctrl groups are not visible to users.
Quoted 7 lines
Quoted 5 lines
The other mode divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
Even with this redumbering SNC mode requires several changes in resctrl behavior for correct operation.
redumbering? Harsh.
Spell check didn't catch this. Implies that redumbering is a real word! Now I need to find an opportunity to use it :-)
Changed to renumbering.
Quoted 10 lines
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0@@ -1357,7 +1357,7 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, } }
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, · line 1360
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1-return size; +return size / snc_nodes_per_l3_cache;To confirm, the size represented by a bit goes down rather than the CBM mask shrinking in each sub-NUMA domain?
Correct. The CBM mask keeps the same number of bits. Those bits are all available to control allocation in each SNC node. But the amount of cache you get per-bit is reduced in the ratio of the number of SNC ways.
Quoted 6 lines
I would maybe have expected the CBM mask to already be allocating at the smallest granularity the hardware supports.
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0}
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param) · line 2593
Quoted 7 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1/** @@ -2590,7 +2590,7 @@ static int rdt_parse_param(struct fs_context *fc, struct fs_parameter *param) ctx->enable_cdpl2 = true;return 0 ;case Opt_mba_mbps: -if (!supports_mba_mbps ()) +if (!supports_mba_mbps () || snc_nodes_per_l3_cache >1 )Factor into
supports_mba_mbps()?
Agreed. That's a better place. Moved this test into supports_mba_mbps().
Quoted 9 lines
Quoted 7 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-1return -EINVAL ; ctx->enable_mba_mbps = true;return 0 ; --2.41.0
Reviewed-by Peter Newman <[email protected]>
Thanks
-Tony
Tony Luck
maintainerPATCH v6 7/8 x86/resctrl: Sub NUMA Cluster detection and enable
In reply to Tony LuckThere isn't a simple h/w bit that indicates whether a CPU is running in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the ratio of NUMA nodes to L3 cache instances.
When SNC mode is detected, reconfigure the RMID counters by updating the MSR_RMID_SNC_CONFIG MSR on each socket as CPUs are seen.
Clearing bit zero of the MSR divides the RMIDs and renumbers the ones on the second SNC node to start from zero. An earlier commit includes all the required changes in Linux to operate in this reconfigured mode.
---
Changes since v5:
Short explanation of RMID reconfiguration added in the commit message (longer one included in previous patch that has all the code that takes action when "snc_nodes_per_l3_cache" is greater than "1").
Added to the comment before snc_remap_rmids() describing what "remapping" is occuring.
Added a comment before snc_get_config() [renamed from get_snc_config() to be consistent with "snc_" as a prefix] describing how it works, and that it can fail if the system is booted with "maxcpus=N" parameter.
Now using bitmap_zalloc() to allocate bitmap.
Add code to defend against divide by zero if no caches are found.
--- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 90 ++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+)
arch/x86/include/asm/msr-index.h +1 -0
@@ -1100,6 +1100,7 @@#define MSR_IA32_QM_CTR 0xc8e#define MSR_IA32_PQR_ASSOC 0xc8f#define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0#define MSR_IA32_L2_CBM_BASE 0xd10#define MSR_IA32_MBA_THRTL_BASE 0xd50
arch/x86/kernel/cpu/resctrl/core.c +90 -0
123 lines not on this page · raw message
Signature · 1 line
2.41.0
Re: [PATCH v6 7/8] x86/resctrl: Sub NUMA Cluster detection and enable
Hi Tony,
Quoted 20 lines · Tony Luck
arch/x86/kernel/cpu/resctrl/core.c +20
-0+static __init int snc_get_config (void ) +{ +unsigned long *node_caches; +int mem_only_nodes =0 ; +int cpu, node, ret; +int num_l3_caches; + +if (!x86_match_cpu (snc_cpu_ids)) +return 1 ; + + node_caches =bitmap_zalloc (nr_node_ids,GFP_KERNEL ); +if (!node_caches) +return 1 ; + +cpus_read_lock (); +for_each_node (node) { + cpu =cpumask_first (cpumask_of_node (node)); +if (cpu < nr_cpu_ids) +set_bit (get_cpu_cacheinfo_id (cpu,3 ), node_caches); +else
arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_resources(void) · line 1075
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +10
-0+ mem_only_nodes++; + } +cpus_read_unlock (); + + num_l3_caches =bitmap_weight (node_caches, nr_node_ids); +if (!num_l3_caches) +return 1 ; + + ret = (nr_node_ids - mem_only_nodes) / num_l3_caches; +kfree (node_caches);
It seems a little peculiar to free node_caches a couple of lines after you're actually done with it.
Quoted 18 lines
arch/x86/kernel/cpu/resctrl/core.c +9
-1+ +if (ret >1 ) + rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl.mon_scope =RESCTRL_NODE ; + +return ret; +} +static __init void rdt_init_res_defs_intel (void ) {struct rdt_hw_resource *hw_res;struct rdt_resource *r; + snc_nodes_per_l3_cache =snc_get_config (); +for_each_rdt_resource (r) { hw_res =resctrl_to_arch_res (r); --
Quoted 2 lines
2.41.0
Tony Luck
maintainerPATCH v6 8/8 x86/resctrl: Update documentation with Sub-NUMA cluster changes
In reply to Tony LuckWith Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
---
Changes since v5:
Added addtional details about challenges tracking tasks when SNC mode is enabled.
--- Documentation/arch/x86/resctrl.rst | 34 +++++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-)
Documentation/arch/x86/resctrl.rst +31 -3
@@ -345,9 +345,15 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. E.g. on a system with + SNC mode disabled with two L3 domains there will be subdirectories + "mon_L3_00" and "mon_L3_01". The numerical suffix refers to the + L3 cache id. With SNC enabled the directory names are the same, + but the numerical suffix refers to the node id. + Mappings from node ids to CPUs are available in the + /sys/devices/system/node/node*/cpulist files. Each of these directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for
@@ -452,6 +458,28 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled the "llc_occupancy", "mbm_total_bytes", and +"mbm_local_bytes" will only give meaningful results for well behaved NUMA +applications. I.e. those that perform the majority of memory accesses +to memory on the local NUMA node to the CPU where the task is executing. +Note that Linux may load balance tasks between Sub-NUMA nodes much +more readily than between regular NUMA nodes since the CPUs on SNC +share the same L3 cache and the system may report the NUMA distance +between SNC nodes with a lower value than used for regular NUMA nodes. +Tasks that migrate between nodes will have their traffic recorded by the +counters in different SNC nodes so a user will need to read mon_data +files from each node on which the task executed to get the full +view of traffic for which the task was the source. + + +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================
Signature · 1 line
2.41.0
Re: [PATCH v6 8/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
Hi Tony,
Quoted 10 lines · Tony Luck
Documentation/arch/x86/resctrl.rst +0
-2@@ -345,9 +345,15 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will
Documentation/arch/x86/resctrl.rst When control is enabled all CTRL_MON groups will also contain: · line 356
Quoted 10 lines
Documentation/arch/x86/resctrl.rst +9
-1- be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. E.g. on a system with + SNC mode disabled with two L3 domains there will be subdirectories + "mon_L3_00" and "mon_L3_01". The numerical suffix refers to the + L3 cache id. With SNC enabled the directory names are the same, + but the numerical suffix refers to the node id. + Mappings from node ids to CPUs are available in the + /sys/devices/system/node/node*/cpulist files. Each of these
The explanation of mon_data seems overwhelmingly SNC-centric now. Maybe the SNC section should be responsible for explaining its impact on the mon_data directory. Mainly by reminding the reader that domain ids in the mon_data directory are node ids in SNC mode.
Documentation/arch/x86/resctrl.rst and 0xA are not. On a system with a 20-bit mask each bit represents 5% · line 466
Quoted 13 lines
Documentation/arch/x86/resctrl.rst +6
-0directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for@@ -452,6 +458,28 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled the "llc_occupancy", "mbm_total_bytes", and +"mbm_local_bytes" will only give meaningful results for well behaved NUMA +applications. I.e. those that perform the majority of memory accesses +to memory on the local NUMA node to the CPU where the task is executing.
Not being specific about why the results aren't meaningful, this sounds vague and alarming.
Quoted 20 lines
Documentation/arch/x86/resctrl.rst +16
-1+Note that Linux may load balance tasks between Sub-NUMA nodes much +more readily than between regular NUMA nodes since the CPUs on SNC +share the same L3 cache and the system may report the NUMA distance +between SNC nodes with a lower value than used for regular NUMA nodes. +Tasks that migrate between nodes will have their traffic recorded by the +counters in different SNC nodes so a user will need to read mon_data +files from each node on which the task executed to get the full +view of traffic for which the task was the source. + + +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ========================================== --
Quoted 2 lines
2.41.0
Tony Luck
maintainerRe: [PATCH v6 8/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
Quoted 20 lines · Peter Newman
Hi Tony,
Quoted 17 lines · Tony Luck
Documentation/arch/x86/resctrl.rst +6
-3@@ -345,9 +345,15 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. E.g. on a system with + SNC mode disabled with two L3 domains there will be subdirectories + "mon_L3_00" and "mon_L3_01". The numerical suffix refers to the + L3 cache id. With SNC enabled the directory names are the same,
Quoted 8 lines
Documentation/arch/x86/resctrl.rst When control is enabled all CTRL_MON groups will also contain: · line 356
Quoted 3 lines
Documentation/arch/x86/resctrl.rst +3
-0+ but the numerical suffix refers to the node id. + Mappings from node ids to CPUs are available in the + /sys/devices/system/node/node*/cpulist files. Each of theseThe explanation of
mon_dataseems overwhelmingly SNC-centric now. Maybe the SNC section should be responsible for explaining its impact on themon_datadirectory. Mainly by reminding the reader that domain ids in themon_datadirectory are node ids in SNC mode.
I cut out all the examples and just note that the numerical suffices are nodes instead of cache instances.
This bit of the git diff now reads:
- This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. Each of these
Quoted 9 lines
Quoted 9 lines
Documentation/arch/x86/resctrl.rst +0
-0directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for@@ -452,6 +458,28 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000.
Quoted 9 lines
Documentation/arch/x86/resctrl.rst and 0xA are not. On a system with a 20-bit mask each bit represents 5% · line 466
Quoted 6 lines
Documentation/arch/x86/resctrl.rst +6
-0+Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled the "llc_occupancy", "mbm_total_bytes", and +"mbm_local_bytes" will only give meaningful results for well behaved NUMA +applications. I.e. those that perform the majority of memory accesses +to memory on the local NUMA node to the CPU where the task is executing.Not being specific about why the results aren't meaningful, this sounds vague and alarming.
Removed the trigger word "meaningful" and re-worded to just explain the increased liklihood that tasks will migrate between nodes, so users must collect data from all nodes. Technically this has always been true on multi-socket systems. But since there is a much higher barrier to task migration between sockets, users may find that simple measurements that used to work now behave differently.
New version:
+Notes on Sub-NUMA Cluster mode
+==============================
+When SNC mode is enabled Linux may load balance tasks between Sub-NUMA +nodes much more readily than between regular NUMA nodes since the CPUs +on Sub-NUMA nodes share the same L3 cache and the system may report +the NUMA distance between Sub-NUMA nodes with a lower value than used +for regular NUMA nodes. Users who do not bind tasks to the CPUs of a +specific Sub-NUMA node must read the "llc_occupancy", "mbm_total_bytes", +and "mbm_local_bytes" for all Sub-NUMA nodes where the tasks may execute +to get the full view of traffic for which the tasks were the source. +
+The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly.
Quoted 25 lines
Quoted 21 lines
Documentation/arch/x86/resctrl.rst +16
-1+Note that Linux may load balance tasks between Sub-NUMA nodes much +more readily than between regular NUMA nodes since the CPUs on SNC +share the same L3 cache and the system may report the NUMA distance +between SNC nodes with a lower value than used for regular NUMA nodes. +Tasks that migrate between nodes will have their traffic recorded by the +counters in different SNC nodes so a user will need to read mon_data +files from each node on which the task executed to get the full +view of traffic for which the task was the source. + + +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ========================================== --Quoted 2 lines
2.41.0
Reviewed-by Peter Newman <[email protected]>
Re: [PATCH v6 0/8] Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckHi Tony,
Quoted 10 lines · Tony Luck
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Is an "SNC NUMA node" a "sub-NUMA node", or a NUMA node on which SNC has been enabled?
Thanks!
-Peter
Tony Luck
maintainerRe: [PATCH v6 0/8] Add support for Sub-NUMA cluster (SNC) systems
Quoted 9 lines · Peter Newman
Hi Tony,
Quoted 6 lines · Tony Luck
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Quoted 7 lines
Quoted 4 lines
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Is an "SNC NUMA node" a "sub-NUMA node", or a NUMA node on which SNC has been enabled?
It would be architecturally possible to enable SNC mode on a subset of CPU sockets. But there isn't a BIOS setup option to do that. You either have SNC everywhere, or nowhere.
I prefer "SNC NUMA node" == "sub-NUMA node".
This version "NUMA node on which SNC has been enabled" makes it sound like there is a control on a NUMA node that can be switched. The control is on the CPU socket. That's often equivalent to a NUMA node, but Intel has had CPUs in the past where this isn't the case (e.g. Cascade Lake -AP and Cooper Lake).
Quoted 3 lines
Thanks!
-Peter
Thanks for the review of the series. I've applied changes to my local tree. Will post v7 of the series early next week if no other reviews come in.
-Tony
Tony Luck
maintainerv7 cover Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckThe Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
Tony Luck (8):
x86/resctrl: Prepare for new domain scope x86/resctrl: Prepare to split rdt_domain structure x86/resctrl: Prepare for different scope for control/monitor
operations
x86/resctrl: Split the rdt_domain and rdt_hw_domain structures x86/resctrl: Add node-scope to the options for feature scope x86/resctrl: Introduce snc_nodes_per_l3_cache x86/resctrl: Sub NUMA Cluster detection and enable x86/resctrl: Update documentation with Sub-NUMA cluster changes
Documentation/arch/x86/resctrl.rst | 23 +- include/linux/resctrl.h | 85 +++-- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 66 ++-- arch/x86/kernel/cpu/resctrl/core.c | 400 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 58 ++-- arch/x86/kernel/cpu/resctrl/monitor.c | 58 ++-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 14 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 132 +++---- 9 files changed, 591 insertions(+), 246 deletions(-)
base-commit: 6465e260f48790807eef06b583b38ca9789b6072
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v7 1/8 x86/resctrl: Prepare for new domain scope
Resctrl resources operate on subsets of CPUs in the system with the defining attribute of each subset being an instance of a particular level of cache. E.g. all CPUs sharing an L3 cache would be part of the same domain.
In preparation for features that are scoped at the NUMA node level change the code from explicit references to "cache_level" to a more generic scope. At this point the only options for this scope are groups of CPUs that share an L2 cache or L3 cache.
Provide a more detailed warning message if a domain id cannot be found when adding a CPU. Just check and silent return if the domain id can't be found when removing a CPU.
---
Changes since last version:
s/-EINVAL/0/ for return value of rdtgroup_cbm_to_size()
include/linux/resctrl.h | 9 +++++-- arch/x86/kernel/cpu/resctrl/core.c | 33 ++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 ++++- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 +++- 4 files changed, 43 insertions(+), 10 deletions(-)
include/linux/resctrl.h +7 -2
@@ -144,13 +144,18 @@ struct resctrl_membw {struct rdt_parse_data ;struct resctrl_schema ; +enum resctrl_scope { +RESCTRL_L2_CACHE =2 , +RESCTRL_L3_CACHE =3 , +}; +/** * struct rdt_resource - attributes of a resctrl resource * @rid: The index of the resource * @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @cache_level: Which cache level defines scope of this resource +* @scope: Scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. * @domains: All domains for this resource
@@ -168,7 +173,7 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -int cache_level; +enum resctrl_scope scope;struct resctrl_cache cache;struct resctrl_membw membw;struct list_head domains;
arch/x86/kernel/cpu/resctrl/core.c +27 -6
@@ -65,7 +65,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L3, .name = "L3", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_L3), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -79,7 +79,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L2, .name = "L2", - .cache_level = 2, + .scope = RESCTRL_L2_CACHE, .domains = domain_init(RDT_RESOURCE_L2), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -93,7 +93,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_MBA, .name = "MB", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_MBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -105,7 +105,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_SMBA, .name = "SMBA", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_SMBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -487,6 +487,19 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)return 0 ; } +static int get_domain_id_from_scope (int cpu,enum resctrl_scope scope) +{ +switch (scope) { +case RESCTRL_L2_CACHE : +case RESCTRL_L3_CACHE : +return get_cpu_cacheinfo_id (cpu, scope); +default : +break ; + } + +return -EINVAL ; +} +/* * domain_add_cpu - Add a cpu to a resource's domain list. *
@@ -502,12 +515,17 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)*/ static void domain_add_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct list_head *add_pos =NULL ;struct rdt_hw_domain *hw_dom;struct rdt_domain *d;int err; +if (id <0 ) { +pr_warn_once ("Can't find domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->scope, r->name); +return ; + } d =rdt_find_domain (r, id, &add_pos);if (IS_ERR (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
@@ -552,10 +570,13 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)static void domain_remove_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct rdt_hw_domain *hw_dom;struct rdt_domain *d; +if (id <0 ) +return ; + d =rdt_find_domain (r, id,NULL );if (IS_ERR_OR_NULL (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +5 -1
@@ -292,10 +292,14 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { +int scope = plr->s->res->scope;struct cpu_cacheinfo *ci;int ret;int i; +if (WARN_ON_ONCE (scope !=RESCTRL_L2_CACHE && scope !=RESCTRL_L3_CACHE )) +return -ENODEV ; +/* Pick the first cpu we find that is associated with the cache. */ plr->cpu =cpumask_first (&plr->d->cpu_mask);
@@ -311,7 +315,7 @@ static int pseudo_lock_region_init(struct pseudo_lock_region *plr) plr->size =rdtgroup_cbm_to_size (plr->s->res, plr->d, plr->cbm);for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == plr->s->res->cache_level) { +if (ci->info_list[i].level == scope) { plr->line_size = ci->info_list[i].coherency_line_size;return 0 ; }
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -1
@@ -1345,10 +1345,13 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,unsigned int size =0 ;int num_b, i; +if (WARN_ON_ONCE (r->scope !=RESCTRL_L2_CACHE && r->scope !=RESCTRL_L3_CACHE )) +return size; + num_b =bitmap_weight (&cbm, r->cache.cbm_len); ci =get_cpu_cacheinfo (cpumask_any (&d->cpu_mask));for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == r->cache_level) { +if (ci->info_list[i].level == r->scope) { size = ci->info_list[i].size / r->cache.cbm_len * num_b;break ; }
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v7 2/8 x86/resctrl: Prepare to split rdt_domain structure
In reply to Tony LuckThe rdt_domain structure is used for both control and monitor features. It is about to be split into separate structures for these two usages because the scope for control and monitoring features for a resource will be different for future resources.
To allow for common code that scans a list of domains looking for a specific domain id, move the "list" and "id" fields into their own structure within the rdt_domain structure.
Add a "type" field to the header to be used later so that callers looking up a domain can be assured that they found one of the expected type.
---
Changes since last version:
Add the "type" field to rdt_domain_hdr.
include/linux/resctrl.h | 21 +++++++++-- arch/x86/kernel/cpu/resctrl/core.c | 16 ++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 16 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 44 +++++++++++------------ 6 files changed, 58 insertions(+), 43 deletions(-)
include/linux/resctrl.h +18 -3
@@ -52,10 +52,26 @@ struct resctrl_staged_config {bool have_new_ctrl; }; +enum resctrl_domain_type { +RESCTRL_CTRL_DOMAIN , +RESCTRL_MON_DOMAIN , +}; +/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_domain_hdr - common header for different domain types * @list: all instances of this resource * @id: unique id for this instance +* @type: type of this instance +*/ +struct rdt_domain_hdr { +struct list_head list; +int id; +enum resctrl_domain_type type; +}; + +/** +* struct rdt_domain - group of CPUs sharing a resctrl resource +* @hdr: common header for different domain types * @cpu_mask: which CPUs share this resource * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold * @mbm_total: saved state for MBM total bandwidth
@@ -71,8 +87,7 @@ struct resctrl_staged_config {* by closid */ struct rdt_domain { -struct list_head list; -int id; +struct rdt_domain_hdr hdr;struct cpumask cpu_mask;unsigned long *rmid_busy_llc;struct mbm_state *mbm_total;
arch/x86/kernel/cpu/resctrl/core.c +8 -8
@@ -352,7 +352,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) {/* Find the domain that contains this CPU */ if (cpumask_test_cpu (cpu, &d->cpu_mask))return d;
@@ -401,12 +401,12 @@ struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,return ERR_PTR (-ENODEV );list_for_each (l, &r->domains) { - d = list_entry(l, struct rdt_domain, list); + d = list_entry(l, struct rdt_domain, hdr.list);/* When id is found, return its domain. */ -if (id == d->id) +if (id == d->hdr.id)return d;/* Stop searching when finding id's position in sorted list. */ -if (id < d->id) +if (id < d->hdr.id)break ; }
@@ -544,7 +544,7 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; d = &hw_dom->d_resctrl; - d->id = id; + d->hdr.id = id;cpumask_set_cpu (cpu, &d->cpu_mask);rdt_domain_reconfigure_cdp (r);
@@ -559,11 +559,11 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; } -list_add_tail (&d->list, add_pos); +list_add_tail (&d->hdr.list, add_pos); err =resctrl_online_domain (r, d);if (err) { -list_del (&d->list); +list_del (&d->hdr.list);domain_free (hw_dom); } }
@@ -587,7 +587,7 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)cpumask_clear_cpu (cpu, &d->cpu_mask);if (cpumask_empty (&d->cpu_mask)) {resctrl_offline_domain (r, d); -list_del (&d->list); +list_del (&d->hdr.list);/* * rdt_domain "d" is going to be freed below, so clear
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +8 -8
@@ -67,7 +67,7 @@ int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -144,7 +144,7 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -224,8 +224,8 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, list) { -if (d->id == dom_id) { +list_for_each_entry (d, &r->domains, hdr.list) { +if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;if (r->parse_ctrlval (&data, s, d))
@@ -316,7 +316,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -464,7 +464,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, list) { +list_for_each_entry (dom, &r->domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -474,7 +474,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clo ctrl_val =resctrl_arch_get_config (r, dom, closid, schema->conf_type); -seq_printf (s, r->format_str, dom->id, max_data_width, +seq_printf (s, r->format_str, dom->hdr.id, max_data_width, ctrl_val); sep = true; }
@@ -503,7 +503,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }else {seq_printf (s,"%s:%d=%x\n" , rdtgrp->plr->s->res->name, - rdtgrp->plr->d->id, + rdtgrp->plr->d->hdr.id, rdtgrp->plr->cbm); } }else {
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -340,7 +340,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { err = resctrl_arch_rmid_read(r, d, entry->rmid, QOS_L3_OCCUP_EVENT_ID,
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1 -1
@@ -856,7 +856,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, list) { +list_for_each_entry (d_i, &r->domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, &d_i->cpu_mask);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +22 -22
168 lines not on this page · raw message
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v7 3/8 x86/resctrl: Prepare for different scope for control/monitor operations
In reply to Tony LuckResctrl assumes that control and monitor operations on a resource are performed at the same scope.
Prepare for systems that use different scope (specifically L3 scope for cache control and NODE scope for cache occupancy and memory bandwidth monitoring).
Create separate domain lists for control and monitor operations.
Note that errors during initialization of either control or monitor functions on a domain would previously result in that domain being excluded from both control and monitor operations. Now the domains are allocated independently it is no longer required to disable both control and monitor operations if either fail.
---
Changes since last version:
Initialize the "type" in rdt_domain_hdr when creating domains. Check type has expected value before using container_of() to get to the surrounding structure.
Rename "hw_mondom" to "hw_dom" in domain_add_cpu_mon() and in domain_remove_cpu_mon().
Add lockdep_assert_held(&rdtgroup_mutex) to resctrl_offline_mon_domain()
include/linux/resctrl.h | 18 +- arch/x86/kernel/cpu/resctrl/internal.h | 4 +- arch/x86/kernel/cpu/resctrl/core.c | 214 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 18 +- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 54 +++--- 7 files changed, 224 insertions(+), 90 deletions(-)
include/linux/resctrl.h +12 -6
@@ -170,10 +170,12 @@ enum resctrl_scope {* @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @scope: Scope of this resource +* @ctrl_scope: Scope of this resource for control functions +* @mon_scope: Scope of this resource for monitor functions * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. -* @domains: All domains for this resource +* @ctrl_domains: Control domains for this resource +* @mon_domains: Monitor domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.
@@ -188,10 +190,12 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -enum resctrl_scope scope; +enum resctrl_scope ctrl_scope; +enum resctrl_scope mon_scope;struct resctrl_cache cache;struct resctrl_membw membw; -struct list_head domains; +struct list_head ctrl_domains; +struct list_head mon_domains;char *name;int data_width;u32 default_ctrl;
@@ -237,8 +241,10 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
arch/x86/kernel/cpu/resctrl/internal.h +2 -2
@@ -511,8 +511,8 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn);int rdtgroup_kn_mode_restrict (struct rdtgroup *r,const char *name);int rdtgroup_kn_mode_restore (struct rdtgroup *r,const char *name,umode_t mask); -struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id, -struct list_head **pos); +struct rdt_domain_hdr *rdt_find_domain (struct list_head *h,int id, +struct list_head **pos);ssize_t rdtgroup_schemata_write (struct kernfs_open_file *of,char *buf,size_t nbytes,loff_t off); int rdtgroup_schemata_show(struct kernfs_open_file *of,
arch/x86/kernel/cpu/resctrl/core.c +164 -50
337 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +13 -5
@@ -224,7 +224,7 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) {if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;
@@ -316,7 +316,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -464,7 +464,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, hdr.list) { +list_for_each_entry (dom, &r->ctrl_domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -540,6 +540,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,int rdtgroup_mondata_show (struct seq_file *m,void *arg) {struct kernfs_open_file *of = m->private; +struct rdt_domain_hdr *hdr;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;
@@ -560,12 +561,19 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) evtid = md.u.evtid; r = &rdt_resources_all[resid].r_resctrl; - d =rdt_find_domain (r, domid,NULL ); -if (IS_ERR_OR_NULL (d)) { + hdr =rdt_find_domain (&r->mon_domains, domid,NULL ); +if (IS_ERR_OR_NULL (hdr)) { ret = -ENOENT ; goto out; } +if (WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) { + ret = -EINVAL ; + goto out; + } + + d = container_of(hdr, struct rdt_domain, hdr); +mon_event_read (&rr, r, d, rdtgrp, evtid, false);if (rr.err == -EIO )
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -340,7 +340,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->mon_domains, hdr.list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { err = resctrl_arch_rmid_read(r, d, entry->rmid, QOS_L3_OCCUP_EVENT_ID,
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +2 -2
@@ -292,7 +292,7 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { -int scope = plr->s->res->scope; +int scope = plr->s->res->ctrl_scope;struct cpu_cacheinfo *ci;int ret;int i;
@@ -856,7 +856,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, hdr.list) { +list_for_each_entry (d_i, &r->ctrl_domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, &d_i->cpu_mask);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +30 -24
170 lines not on this page · raw message
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v7 4/8 x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
In reply to Tony LuckThe same rdt_domain structure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.
Split into separate rdt_ctrl_domain and rdt_mon_domain structures with just the fields required for control and monitoring respectively.
Similar split of the rdt_hw_domain structure into rdt_hw_ctrl_domain and rdt_hw_mon_domain.
---
Changes since last version:
"wrapped line not quite aligned anymore" * 2. Both fixed.
include/linux/resctrl.h | 50 +++++++------ arch/x86/kernel/cpu/resctrl/internal.h | 60 ++++++++++------ arch/x86/kernel/cpu/resctrl/core.c | 87 ++++++++++++----------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 32 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 40 +++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++-------- 7 files changed, 184 insertions(+), 153 deletions(-)
include/linux/resctrl.h +30 -20
@@ -63,7 +63,25 @@ struct rdt_domain_hdr { };/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_ctrl_domain - group of CPUs sharing a resctrl control resource +* @hdr: common header for different domain types +* @cpu_mask: which CPUs share this resource +* @plr: pseudo-locked region (if any) associated with domain +* @staged_config: parsed configuration to be applied +* @mbps_val: When mba_sc is enabled, this holds the array of user +* specified control values for mba_sc in MBps, indexed +* by closid +*/ +struct rdt_ctrl_domain { +struct rdt_domain_hdr hdr; +struct cpumask cpu_mask; +struct pseudo_lock_region *plr; +struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; +u32 *mbps_val; +}; + +/** +* struct rdt_mon_domain - group of CPUs sharing a resctrl control resource * @hdr: common header for different domain types * @cpu_mask: which CPUs share this resource * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold
@@ -73,13 +91,8 @@ struct rdt_domain_hdr {* @cqm_limbo: worker to periodically read CQM h/w counters * @mbm_work_cpu: worker CPU for MBM h/w counters * @cqm_work_cpu: worker CPU for CQM h/w counters -* @plr: pseudo-locked region (if any) associated with domain -* @staged_config: parsed configuration to be applied -* @mbps_val: When mba_sc is enabled, this holds the array of user -* specified control values for mba_sc in MBps, indexed -* by closid */ -struct rdt_domain { +struct rdt_mon_domain {struct rdt_domain_hdr hdr;struct cpumask cpu_mask;unsigned long *rmid_busy_llc;
@@ -89,9 +102,6 @@ struct rdt_domain {struct delayed_work cqm_limbo;int mbm_work_cpu;int cqm_work_cpu; -struct pseudo_lock_region *plr; -struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; -u32 *mbps_val; };/**
@@ -195,7 +205,7 @@ struct rdt_resource {const char *format_str;int (*parse_ctrlval)(struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d); +struct rdt_ctrl_domain *d);struct list_head evt_list;unsigned long fflags;bool cdp_capable;
@@ -229,15 +239,15 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);* Update the ctrl_val and apply this config right now. * Must be called on one of the domain's CPUs. */ -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val); -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
@@ -253,7 +263,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_domain *d);* Return: * 0 on success, or -EIO, -EINVAL etc on error. */ -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val);/**
@@ -266,7 +276,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid);/**
@@ -278,7 +288,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d);extern unsigned int resctrl_rmid_realloc_threshold;extern unsigned int resctrl_rmid_realloc_limit;
arch/x86/kernel/cpu/resctrl/internal.h +38 -22
131 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/core.c +46 -41
246 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +16 -16
@@ -58,7 +58,7 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r) }int parse_bw (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct resctrl_staged_config *cfg;u32 closid = data->rdtgrp->closid;
@@ -135,7 +135,7 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)* resource type. */ int parse_cbm (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct rdtgroup *rdtgrp = data->rdtgrp;struct resctrl_staged_config *cfg;
@@ -205,7 +205,7 @@ static int parse_line(char *line, struct resctrl_schema *s,struct rdt_resource *r = s->res;struct rdt_parse_data data;char *dom =NULL , *id; -struct rdt_domain *d; +struct rdt_ctrl_domain *d;unsigned long dom_id; if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
@@ -265,11 +265,11 @@ static u32 get_config_index(u32 closid, enum resctrl_conf_type type) } } -static bool apply_config (struct rdt_hw_domain *hw_dom, +static bool apply_config (struct rdt_hw_ctrl_domain *hw_dom,struct resctrl_staged_config *cfg,u32 idx,cpumask_var_t cpu_mask) { -struct rdt_domain *dom = &hw_dom->d_resctrl; +struct rdt_ctrl_domain *dom = &hw_dom->d_resctrl;if (cfg->new_ctrl != hw_dom->ctrl_val[idx]) {cpumask_set_cpu (cpumask_any (&dom->cpu_mask), cpu_mask);
@@ -281,11 +281,11 @@ static bool apply_config(struct rdt_hw_domain *hw_dom,return false; } -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val) { +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);u32 idx =get_config_index (closid, t);struct msr_param msr_param;
@@ -305,11 +305,11 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,int resctrl_arch_update_domains (struct rdt_resource *r,u32 closid) {struct resctrl_staged_config *cfg; -struct rdt_hw_domain *hw_dom; +struct rdt_hw_ctrl_domain *hw_dom;struct msr_param msr_param;enum resctrl_conf_type t; +struct rdt_ctrl_domain *d;cpumask_var_t cpu_mask; -struct rdt_domain *d;u32 idx;if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))
@@ -317,7 +317,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid) msr_param.res =NULL ;list_for_each_entry (d, &r->ctrl_domains, hdr.list) { - hw_dom =resctrl_to_arch_dom (d); + hw_dom =resctrl_to_arch_ctrl_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];if (!cfg->have_new_ctrl)
@@ -447,10 +447,10 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,return ret ?: nbytes; } -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);u32 idx =get_config_index (closid, type);return hw_dom->ctrl_val[idx];
@@ -459,7 +459,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,static void show_doms (struct seq_file *s,struct resctrl_schema *schema,int closid) {struct rdt_resource *r = schema->res; -struct rdt_domain *dom; +struct rdt_ctrl_domain *dom;bool sep = false;u32 ctrl_val;
@@ -521,7 +521,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }void mon_event_read (struct rmid_read *rr,struct rdt_resource *r, -struct rdt_domain *d,struct rdtgroup *rdtgrp, +struct rdt_mon_domain *d,struct rdtgroup *rdtgrp,int evtid,int first) {/*
@@ -541,11 +541,11 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) {struct kernfs_open_file *of = m->private;struct rdt_domain_hdr *hdr; +struct rdt_mon_domain *d;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;union mon_data_bits md; -struct rdt_domain *d;struct rmid_read rr;int ret =0 ;
@@ -566,7 +566,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) ret = -ENOENT ; goto out; } - d = container_of(hdr, struct rdt_domain, hdr); + d = container_of(hdr, struct rdt_mon_domain, hdr);mon_event_read (&rr, r, d, rdtgrp, evtid, false);
arch/x86/kernel/cpu/resctrl/monitor.c +20 -20
@@ -170,7 +170,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)return 0 ; } -static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_domain *hw_dom, +static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_mon_domain *hw_dom,u32 rmid,enum resctrl_event_id eventid) {
@@ -189,10 +189,10 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom,return NULL ; } -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct arch_mbm_state *am; am =get_arch_mbm_state (hw_dom, rmid, eventid);
@@ -208,9 +208,9 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* Assumes that hardware counters are also reset and thus that there is * no need to record initial non-zero counts. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d) +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);if (is_mbm_total_enabled ()) memset(hw_dom->arch_mbm_total, 0,
@@ -229,11 +229,11 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)return chunks >> shift; } -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);struct arch_mbm_state *am;u64 msr_val, chunks;int ret;
@@ -266,7 +266,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* decrement the count. If the busy count gets to zero on an RMID, we * free the RMID */ -void __check_limbo (struct rdt_domain *d,bool force_free) +void __check_limbo (struct rdt_mon_domain *d,bool force_free) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rmid_entry *entry;
@@ -305,7 +305,7 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } } -bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d) +bool has_busy_rmid (struct rdt_resource *r,struct rdt_mon_domain *d) {return find_first_bit (d->rmid_busy_llc, r->num_rmid) != r->num_rmid; }
@@ -334,7 +334,7 @@ int alloc_rmid(void)static void add_rmid_to_limbo (struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; -struct rdt_domain *d; +struct rdt_mon_domain *d;int cpu, err;u64 val =0 ;
@@ -383,7 +383,7 @@ void free_rmid(u32 rmid)list_add_tail (&entry->list, &rmid_free_lru); } -static struct mbm_state *get_mbm_state (struct rdt_domain *d,u32 rmid, +static struct mbm_state *get_mbm_state (struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id evtid) {switch (evtid) {
@@ -516,13 +516,13 @@ void mon_event_count(void *info)* throttle MSRs already have low percentage values. To avoid * unnecessarily restricting such rdtgroups, we also increase the bandwidth. */ -static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_domain *dom_mbm) +static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_mon_domain *dom_mbm) {u32 closid, rmid, cur_msr_val, new_msr_val;struct mbm_state *pmbm_data, *cmbm_data; +struct rdt_ctrl_domain *dom_mba;u32 cur_bw, delta_bw, user_bw;struct rdt_resource *r_mba; -struct rdt_domain *dom_mba;struct list_head *head;struct rdtgroup *entry;
@@ -600,7 +600,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) } } -static void mbm_update (struct rdt_resource *r,struct rdt_domain *d,int rmid) +static void mbm_update (struct rdt_resource *r,struct rdt_mon_domain *d,int rmid) {struct rmid_read rr;
@@ -640,13 +640,13 @@ void cqm_handle_limbo(struct work_struct *work) {unsigned long delay =msecs_to_jiffies (CQM_LIMBOCHECK_INTERVAL );int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex); r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, cqm_limbo.work); + d = container_of(work, struct rdt_mon_domain, cqm_limbo.work);__check_limbo (d, false);
@@ -656,7 +656,7 @@ void cqm_handle_limbo(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms) +void cqm_setup_limbo_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
@@ -672,9 +672,9 @@ void mbm_handle_overflow(struct work_struct *work)unsigned long delay =msecs_to_jiffies (MBM_OVERFLOW_INTERVAL );struct rdtgroup *prgrp, *crgrp;int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct list_head *head;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex);
@@ -682,7 +682,7 @@ void mbm_handle_overflow(struct work_struct *work) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, mbm_over.work); + d = container_of(work, struct rdt_mon_domain, mbm_over.work);list_for_each_entry (prgrp, &rdt_all_groups, rdtgroup_list) {mbm_update (r, d, prgrp->mon.rmid);
@@ -701,7 +701,7 @@ void mbm_handle_overflow(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms) +void mbm_setup_overflow_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -3
@@ -814,7 +814,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)* Return: true if @cbm overlaps with pseudo-locked region on @d, false * otherwise. */ -bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,unsigned long cbm) +bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_ctrl_domain *d,unsigned long cbm) {unsigned int cbm_len;unsigned long cbm_b;
@@ -841,11 +841,11 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm* if it is not possible to test due to memory allocation issue, * false otherwise. */ -bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d) +bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_ctrl_domain *d) { +struct rdt_ctrl_domain *d_i;cpumask_var_t cpu_with_psl;struct rdt_resource *r; -struct rdt_domain *d_i;bool ret = false;if (!zalloc_cpumask_var (&cpu_with_psl,GFP_KERNEL ))
arch/x86/kernel/cpu/resctrl/rdtgroup.c +31 -31
252 lines not on this page · raw message
--
2.41.0
--- include/linux/resctrl.h | 50 +++++++------ arch/x86/kernel/cpu/resctrl/internal.h | 60 ++++++++++------ arch/x86/kernel/cpu/resctrl/core.c | 87 ++++++++++++----------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 32 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 40 +++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++-------- 7 files changed, 184 insertions(+), 153 deletions(-)
include/linux/resctrl.h +30 -20
102 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/internal.h +38 -22
131 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/core.c +46 -41
246 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +16 -16
@@ -58,7 +58,7 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r) }int parse_bw (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct resctrl_staged_config *cfg;u32 closid = data->rdtgrp->closid;
@@ -135,7 +135,7 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)* resource type. */ int parse_cbm (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct rdtgroup *rdtgrp = data->rdtgrp;struct resctrl_staged_config *cfg;
@@ -205,7 +205,7 @@ static int parse_line(char *line, struct resctrl_schema *s,struct rdt_resource *r = s->res;struct rdt_parse_data data;char *dom =NULL , *id; -struct rdt_domain *d; +struct rdt_ctrl_domain *d;unsigned long dom_id; if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
@@ -265,11 +265,11 @@ static u32 get_config_index(u32 closid, enum resctrl_conf_type type) } } -static bool apply_config (struct rdt_hw_domain *hw_dom, +static bool apply_config (struct rdt_hw_ctrl_domain *hw_dom,struct resctrl_staged_config *cfg,u32 idx,cpumask_var_t cpu_mask) { -struct rdt_domain *dom = &hw_dom->d_resctrl; +struct rdt_ctrl_domain *dom = &hw_dom->d_resctrl;if (cfg->new_ctrl != hw_dom->ctrl_val[idx]) {cpumask_set_cpu (cpumask_any (&dom->cpu_mask), cpu_mask);
@@ -281,11 +281,11 @@ static bool apply_config(struct rdt_hw_domain *hw_dom,return false; } -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val) { +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);u32 idx =get_config_index (closid, t);struct msr_param msr_param;
@@ -305,11 +305,11 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,int resctrl_arch_update_domains (struct rdt_resource *r,u32 closid) {struct resctrl_staged_config *cfg; -struct rdt_hw_domain *hw_dom; +struct rdt_hw_ctrl_domain *hw_dom;struct msr_param msr_param;enum resctrl_conf_type t; +struct rdt_ctrl_domain *d;cpumask_var_t cpu_mask; -struct rdt_domain *d;u32 idx;if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))
@@ -317,7 +317,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid) msr_param.res =NULL ;list_for_each_entry (d, &r->ctrl_domains, hdr.list) { - hw_dom =resctrl_to_arch_dom (d); + hw_dom =resctrl_to_arch_ctrl_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];if (!cfg->have_new_ctrl)
@@ -447,10 +447,10 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,return ret ?: nbytes; } -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);u32 idx =get_config_index (closid, type);return hw_dom->ctrl_val[idx];
@@ -459,7 +459,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,static void show_doms (struct seq_file *s,struct resctrl_schema *schema,int closid) {struct rdt_resource *r = schema->res; -struct rdt_domain *dom; +struct rdt_ctrl_domain *dom;bool sep = false;u32 ctrl_val;
@@ -521,7 +521,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }void mon_event_read (struct rmid_read *rr,struct rdt_resource *r, -struct rdt_domain *d,struct rdtgroup *rdtgrp, +struct rdt_mon_domain *d,struct rdtgroup *rdtgrp,int evtid,int first) {/*
@@ -541,11 +541,11 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) {struct kernfs_open_file *of = m->private;struct rdt_domain_hdr *hdr; +struct rdt_mon_domain *d;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;union mon_data_bits md; -struct rdt_domain *d;struct rmid_read rr;int ret =0 ;
@@ -572,7 +572,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) goto out; } - d = container_of(hdr, struct rdt_domain, hdr); + d = container_of(hdr, struct rdt_mon_domain, hdr);mon_event_read (&rr, r, d, rdtgrp, evtid, false);
arch/x86/kernel/cpu/resctrl/monitor.c +20 -20
@@ -170,7 +170,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)return 0 ; } -static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_domain *hw_dom, +static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_mon_domain *hw_dom,u32 rmid,enum resctrl_event_id eventid) {
@@ -189,10 +189,10 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom,return NULL ; } -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct arch_mbm_state *am; am =get_arch_mbm_state (hw_dom, rmid, eventid);
@@ -208,9 +208,9 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* Assumes that hardware counters are also reset and thus that there is * no need to record initial non-zero counts. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d) +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);if (is_mbm_total_enabled ()) memset(hw_dom->arch_mbm_total, 0,
@@ -229,11 +229,11 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)return chunks >> shift; } -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);struct arch_mbm_state *am;u64 msr_val, chunks;int ret;
@@ -266,7 +266,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* decrement the count. If the busy count gets to zero on an RMID, we * free the RMID */ -void __check_limbo (struct rdt_domain *d,bool force_free) +void __check_limbo (struct rdt_mon_domain *d,bool force_free) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rmid_entry *entry;
@@ -305,7 +305,7 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } } -bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d) +bool has_busy_rmid (struct rdt_resource *r,struct rdt_mon_domain *d) {return find_first_bit (d->rmid_busy_llc, r->num_rmid) != r->num_rmid; }
@@ -334,7 +334,7 @@ int alloc_rmid(void)static void add_rmid_to_limbo (struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; -struct rdt_domain *d; +struct rdt_mon_domain *d;int cpu, err;u64 val =0 ;
@@ -383,7 +383,7 @@ void free_rmid(u32 rmid)list_add_tail (&entry->list, &rmid_free_lru); } -static struct mbm_state *get_mbm_state (struct rdt_domain *d,u32 rmid, +static struct mbm_state *get_mbm_state (struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id evtid) {switch (evtid) {
@@ -516,13 +516,13 @@ void mon_event_count(void *info)* throttle MSRs already have low percentage values. To avoid * unnecessarily restricting such rdtgroups, we also increase the bandwidth. */ -static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_domain *dom_mbm) +static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_mon_domain *dom_mbm) {u32 closid, rmid, cur_msr_val, new_msr_val;struct mbm_state *pmbm_data, *cmbm_data; +struct rdt_ctrl_domain *dom_mba;u32 cur_bw, delta_bw, user_bw;struct rdt_resource *r_mba; -struct rdt_domain *dom_mba;struct list_head *head;struct rdtgroup *entry;
@@ -600,7 +600,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) } } -static void mbm_update (struct rdt_resource *r,struct rdt_domain *d,int rmid) +static void mbm_update (struct rdt_resource *r,struct rdt_mon_domain *d,int rmid) {struct rmid_read rr;
@@ -640,13 +640,13 @@ void cqm_handle_limbo(struct work_struct *work) {unsigned long delay =msecs_to_jiffies (CQM_LIMBOCHECK_INTERVAL );int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex); r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, cqm_limbo.work); + d = container_of(work, struct rdt_mon_domain, cqm_limbo.work);__check_limbo (d, false);
@@ -656,7 +656,7 @@ void cqm_handle_limbo(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms) +void cqm_setup_limbo_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
@@ -672,9 +672,9 @@ void mbm_handle_overflow(struct work_struct *work)unsigned long delay =msecs_to_jiffies (MBM_OVERFLOW_INTERVAL );struct rdtgroup *prgrp, *crgrp;int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct list_head *head;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex);
@@ -682,7 +682,7 @@ void mbm_handle_overflow(struct work_struct *work) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, mbm_over.work); + d = container_of(work, struct rdt_mon_domain, mbm_over.work);list_for_each_entry (prgrp, &rdt_all_groups, rdtgroup_list) {mbm_update (r, d, prgrp->mon.rmid);
@@ -701,7 +701,7 @@ void mbm_handle_overflow(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms) +void mbm_setup_overflow_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -3
@@ -814,7 +814,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)* Return: true if @cbm overlaps with pseudo-locked region on @d, false * otherwise. */ -bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,unsigned long cbm) +bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_ctrl_domain *d,unsigned long cbm) {unsigned int cbm_len;unsigned long cbm_b;
@@ -841,11 +841,11 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm* if it is not possible to test due to memory allocation issue, * false otherwise. */ -bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d) +bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_ctrl_domain *d) { +struct rdt_ctrl_domain *d_i;cpumask_var_t cpu_with_psl;struct rdt_resource *r; -struct rdt_domain *d_i;bool ret = false;if (!zalloc_cpumask_var (&cpu_with_psl,GFP_KERNEL ))
arch/x86/kernel/cpu/resctrl/rdtgroup.c +31 -31
252 lines not on this page · raw message
Signature · 1 line
2.41.0
Luck, Tony
maintainerRE: [PATCH v7 4/8] x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
Quoted 5 lines
The same
rdt_domainstructure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.
Quoted 8 lines
Split into separate
rdt_ctrl_domainandrdt_mon_domainstructures with just the fields required for control and monitoring respectively.Similar split of the
rdt_hw_domainstructure intordt_hw_ctrl_domainandrdt_hw_mon_domain.Signed-off-by Tony Luck <[email protected]>Reviewed-by Peter Newman <[email protected]>
Bother^2 ... lkp complained it can't apply this series to any of the usual upstream targets (linus, tip, linx-next). Digging into that it seems that "git format-patch" may have generated something that can't be applied using "git am".
Not sure if I've got something bad in my config, or my git tree.
Will keep digging.
-Tony
Tony Luck
maintainerPATCH v7 5/8 x86/resctrl: Add node-scope to the options for feature scope
In reply to Tony LuckCurrently supported resctrl features are all domain scoped the same as the scope of the L2 or L3 caches.
Add RESCTRL_NODE as a new option for features that are scoped at the same granularity as NUMA nodes. This is needed for Intel's Sub-NUMA Cluster (SNC) feature where monitoring features are node scoped.
---
No changes since last version
include/linux/resctrl.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 2 ++ 2 files changed, 3 insertions(+)
include/linux/resctrl.h +1 -0
@@ -172,6 +172,7 @@ struct resctrl_schema;enum resctrl_scope {RESCTRL_L2_CACHE =2 ,RESCTRL_L3_CACHE =3 , +RESCTRL_NODE , };/**
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -511,6 +511,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)case RESCTRL_L2_CACHE :case RESCTRL_L3_CACHE :return get_cpu_cacheinfo_id (cpu, scope); +case RESCTRL_NODE : +return cpu_to_node (cpu);default :break ; }
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v7 6/8 x86/resctrl: Introduce snc_nodes_per_l3_cache
In reply to Tony LuckIntel Sub-NUMA Cluster (SNC) is a feature that subdivides the CPU cores and memory controllers on a socket into two or more groups. These are presented to the operating system as NUMA nodes.
This may enable some workloads to have slightly lower latency to memory as the memory controller(s) in an SNC node are electrically closer to the CPU cores on that SNC node. This cost may be offset by lower bandwidth since the memory accesses for each core can only be interleaved between the memory controllers on the same SNC node.
Resctrl monitoring on Intel system depends upon attaching RMIDs to tasks to track L3 cache occupancy and memory bandwidth. There is an MSR that controls how the RMIDs are shared between SNC nodes.
The default mode divides them numerically. E.g. when there are two SNC nodes on a socket the lower number half of the RMIDs are given to the first node, the remainder to the second node. This would be difficult to use with the Linux resctrl interface as specific RMID values assigned to resctrl groups are not visible to users.
The other mode divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
Even with this renumbering SNC mode requires several changes in resctrl behavior for correct operation.
Add a global integer "snc_nodes_per_l3_cache" that will show how many SNC nodes share each L3 cache. When this is "1", SNC mode is either not implemented, or not enabled.
A later patch will detect SNC mode and set snc_nodes_per_l3_cache to the appropriate value. For now it remains at the default "1" to indicate SNC mode is not active.
Code that needs to take action when SNC is enabled is:
1) The number of logical RMIDs per L3 cache available for use is the number of physical RMIDs divided by the number of SNC nodes.
2) Likewise the "mon_scale" value must be adjusted for the number of SNC nodes.
3) The RMID renumbering operates when using the value from the IA32_PQR_ASSOC MSR to count accesses by a task. When reading an RMID counter, code must adjust from the logical RMID used to the physical RMID value for the SNC node that it wishes to read and load the adjusted value into the IA32_QM_EVTSEL MSR.
4) The L3 cache is divided between the SNC nodes. So the value reported in the resctrl "size" file is adjusted.
5) The "-o mba_MBps" mount option must be disabled in SNC mode because the monitoring is being done per SNC node, while the bandwidth allocation is still done at the L3 cache scope. Trying to use this feedback loop might result in contradictory changes to the throttling level coming from each of the SNC node bandwidth measurements.
---
Changes since last version:
In commit comment s/redumbering/renumbering/
Move check that SNC is not enabled into supports_mba_mbps().
arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/core.c | 6 ++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 16 +++++++++++++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 +++-- 4 files changed, 24 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -446,6 +446,8 @@ DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);extern struct dentry *debugfs_resctrl; +extern int snc_nodes_per_l3_cache; +enum resctrl_res_level {RDT_RESOURCE_L3 ,RDT_RESOURCE_L2 ,
arch/x86/kernel/cpu/resctrl/core.c +6 -0
@@ -48,6 +48,12 @@ int max_name_width, max_data_width;*/ bool rdt_alloc_capable; +/* +* Number of SNC nodes that share each L3 cache. Default is 1 for +* systems that do not support SNC, or have SNC disabled. +*/ +int snc_nodes_per_l3_cache =1 ; +static void mba_wrmsr_intel (struct rdt_ctrl_domain *d,struct msr_param *m,struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/monitor.c +13 -3
@@ -148,8 +148,18 @@ static inline struct rmid_entry *__rmid_entry(u32 rmid)static int __rmid_read (u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +int cpu =smp_processor_id (); +int rmid_offset =0 ;u64 msr_val; +/* +* When SNC mode is on, need to compute the offset to read the +* physical RMID counter for the node to which this CPU belongs. +*/ +if (snc_nodes_per_l3_cache >1 ) + rmid_offset = (cpu_to_node (cpu) % snc_nodes_per_l3_cache) * r->num_rmid; +/* * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured * with a valid event code for supported resource type and the bits
@@ -158,7 +168,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)* IA32_QM_CTR.Error (bit 63) and IA32_QM_CTR.Unavailable (bit 62) * are error bits. */ -wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid); +wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid + rmid_offset);rdmsrl (MSR_IA32_QM_CTR , msr_val);if (msr_val &RMID_VAL_ERROR )
@@ -783,8 +793,8 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)int ret; resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size *1024 ; - hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale; - r->num_rmid = boot_cpu_data.x86_cache_max_rmid +1 ; + hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache; + r->num_rmid = (boot_cpu_data.x86_cache_max_rmid +1 ) / snc_nodes_per_l3_cache; hw_res->mbm_width =MBM_CNTR_WIDTH_BASE ;if (mbm_offset >0 && mbm_offset <=MBM_CNTR_WIDTH_OFFSET_MAX )
arch/x86/kernel/cpu/resctrl/rdtgroup.c +3 -2
@@ -1357,7 +1357,7 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, } } -return size; +return size / snc_nodes_per_l3_cache; }/**
@@ -2207,7 +2207,8 @@ static bool supports_mba_mbps(void)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA ].r_resctrl;return (is_mbm_local_enabled () && - r->alloc_capable &&is_mba_linear ()); + r->alloc_capable &&is_mba_linear () && + snc_nodes_per_l3_cache ==1 ); }/*
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v7 7/8 x86/resctrl: Sub NUMA Cluster detection and enable
In reply to Tony LuckThere isn't a simple h/w bit that indicates whether a CPU is running in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the ratio of NUMA nodes to L3 cache instances.
When SNC mode is detected, reconfigure the RMID counters by updating the MSR_RMID_SNC_CONFIG MSR on each socket as CPUs are seen.
Clearing bit zero of the MSR divides the RMIDs and renumbers the ones on the second SNC node to start from zero. An earlier commit includes all the required changes in Linux to operate in this reconfigured mode.
---
Changes since last version:
Moved kfree(node_caches); earlier, to the earliest point where it is no longer needed.
Added Granite Rapids to list of CPU models that support SNC mode.
arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 92 ++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+)
arch/x86/include/asm/msr-index.h +1 -0
@@ -1100,6 +1100,7 @@#define MSR_IA32_QM_CTR 0xc8e#define MSR_IA32_PQR_ASSOC 0xc8f#define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0#define MSR_IA32_L2_CBM_BASE 0xd10#define MSR_IA32_MBA_THRTL_BASE 0xd50
arch/x86/kernel/cpu/resctrl/core.c +92 -0
125 lines not on this page · raw message
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v7 8/8 x86/resctrl: Update documentation with Sub-NUMA cluster changes
In reply to Tony LuckWith Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
---
Changes since v5:
Added addtional details about challenges tracking tasks when SNC mode is enabled.
Documentation/arch/x86/resctrl.rst | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
Documentation/arch/x86/resctrl.rst +20 -3
@@ -345,9 +345,9 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. Each of these directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for
@@ -452,6 +452,23 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled Linux may load balance tasks between Sub-NUMA +nodes much more readily than between regular NUMA nodes since the CPUs +on Sub-NUMA nodes share the same L3 cache and the system may report +the NUMA distance between Sub-NUMA nodes with a lower value than used +for regular NUMA nodes. Users who do not bind tasks to the CPUs of a +specific Sub-NUMA node must read the "llc_occupancy", "mbm_total_bytes", +and "mbm_local_bytes" for all Sub-NUMA nodes where the tasks may execute +to get the full view of traffic for which the tasks were the source. + +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================
Signature · 1 line
2.41.0
Luck, Tony
maintainerRE: [PATCH v7 0/8] Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckQuoted 5 lines
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Quoted 8 lines
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
Bother .. forgot to add the changes since last version summary to the cover letter. I fixed all the issues called out by Peter Newman in his review of v6 series. Specific details are included in each patch (except for patch 0005 which is unchanged).
I added Peter's "Reviewed-by" to patches where he offered it AND where I didn't make substantive changes (parts 4, 5, 6, 7)
-Tony
Reinette Chatre
maintainerRe: [PATCH v7 0/8] Add support for Sub-NUMA cluster (SNC) systems
Quoted 9 lines · Tony
Quoted 9 lines
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accuratlely.
The typo that I pointed out in V4 as well as V5 remains. Not fixing something this fundamental reflects poorly on the rest of this work.
Reinette
Tony Luck
maintainerv8 cover Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckThe Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accurately.
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
---
Please ignore v7 posting. There was some glitch in how I created the patches with "git format-patch" that meant part 0004 would not apply.
Changes since v6:
* Fixed spelling of "accurately" in cover letter.
* Applied changes from Peter Newman's review
(and follow-on posts against other patches in the v6 series).
See comments in indivdual patches for specific details.
Added Peter's "Reviewed-by" to parts 4-7.q
Tony Luck (8):
x86/resctrl: Prepare for new domain scope x86/resctrl: Prepare to split rdt_domain structure x86/resctrl: Prepare for different scope for control/monitor
operations
x86/resctrl: Split the rdt_domain and rdt_hw_domain structures x86/resctrl: Add node-scope to the options for feature scope x86/resctrl: Introduce snc_nodes_per_l3_cache x86/resctrl: Sub NUMA Cluster detection and enable x86/resctrl: Update documentation with Sub-NUMA cluster changes
Documentation/arch/x86/resctrl.rst | 23 +- include/linux/resctrl.h | 85 +++-- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 66 ++-- arch/x86/kernel/cpu/resctrl/core.c | 400 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 58 ++-- arch/x86/kernel/cpu/resctrl/monitor.c | 58 ++-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 14 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 132 +++---- 9 files changed, 591 insertions(+), 246 deletions(-)
base-commit: 6465e260f48790807eef06b583b38ca9789b6072
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v8 1/8 x86/resctrl: Prepare for new domain scope
Resctrl resources operate on subsets of CPUs in the system with the defining attribute of each subset being an instance of a particular level of cache. E.g. all CPUs sharing an L3 cache would be part of the same domain.
In preparation for features that are scoped at the NUMA node level change the code from explicit references to "cache_level" to a more generic scope. At this point the only options for this scope are groups of CPUs that share an L2 cache or L3 cache.
Provide a more detailed warning message if a domain id cannot be found when adding a CPU. Just check and silent return if the domain id can't be found when removing a CPU.
---
Changes since last version:
s/-EINVAL/0/ for return value of rdtgroup_cbm_to_size()
---
include/linux/resctrl.h +7 -2
@@ -144,13 +144,18 @@ struct resctrl_membw {struct rdt_parse_data ;struct resctrl_schema ; +enum resctrl_scope { +RESCTRL_L2_CACHE =2 , +RESCTRL_L3_CACHE =3 , +}; +/** * struct rdt_resource - attributes of a resctrl resource * @rid: The index of the resource * @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @cache_level: Which cache level defines scope of this resource +* @scope: Scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. * @domains: All domains for this resource
@@ -168,7 +173,7 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -int cache_level; +enum resctrl_scope scope;struct resctrl_cache cache;struct resctrl_membw membw;struct list_head domains;
arch/x86/kernel/cpu/resctrl/core.c +27 -6
@@ -65,7 +65,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L3, .name = "L3", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_L3), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -79,7 +79,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L2, .name = "L2", - .cache_level = 2, + .scope = RESCTRL_L2_CACHE, .domains = domain_init(RDT_RESOURCE_L2), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -93,7 +93,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_MBA, .name = "MB", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_MBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -105,7 +105,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_SMBA, .name = "SMBA", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_SMBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -487,6 +487,19 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)return 0 ; } +static int get_domain_id_from_scope (int cpu,enum resctrl_scope scope) +{ +switch (scope) { +case RESCTRL_L2_CACHE : +case RESCTRL_L3_CACHE : +return get_cpu_cacheinfo_id (cpu, scope); +default : +break ; + } + +return -EINVAL ; +} +/* * domain_add_cpu - Add a cpu to a resource's domain list. *
@@ -502,12 +515,17 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)*/ static void domain_add_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct list_head *add_pos =NULL ;struct rdt_hw_domain *hw_dom;struct rdt_domain *d;int err; +if (id <0 ) { +pr_warn_once ("Can't find domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->scope, r->name); +return ; + } d =rdt_find_domain (r, id, &add_pos);if (IS_ERR (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
@@ -552,10 +570,13 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)static void domain_remove_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct rdt_hw_domain *hw_dom;struct rdt_domain *d; +if (id <0 ) +return ; + d =rdt_find_domain (r, id,NULL );if (IS_ERR_OR_NULL (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +5 -1
@@ -292,10 +292,14 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { +int scope = plr->s->res->scope;struct cpu_cacheinfo *ci;int ret;int i; +if (WARN_ON_ONCE (scope !=RESCTRL_L2_CACHE && scope !=RESCTRL_L3_CACHE )) +return -ENODEV ; +/* Pick the first cpu we find that is associated with the cache. */ plr->cpu =cpumask_first (&plr->d->cpu_mask);
@@ -311,7 +315,7 @@ static int pseudo_lock_region_init(struct pseudo_lock_region *plr) plr->size =rdtgroup_cbm_to_size (plr->s->res, plr->d, plr->cbm);for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == plr->s->res->cache_level) { +if (ci->info_list[i].level == scope) { plr->line_size = ci->info_list[i].coherency_line_size;return 0 ; }
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -1
@@ -1345,10 +1345,13 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,unsigned int size =0 ;int num_b, i; +if (WARN_ON_ONCE (r->scope !=RESCTRL_L2_CACHE && r->scope !=RESCTRL_L3_CACHE )) +return size; + num_b =bitmap_weight (&cbm, r->cache.cbm_len); ci =get_cpu_cacheinfo (cpumask_any (&d->cpu_mask));for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == r->cache_level) { +if (ci->info_list[i].level == r->scope) { size = ci->info_list[i].size / r->cache.cbm_len * num_b;break ; }
Re: [PATCH v8 1/8] x86/resctrl: Prepare for new domain scope
arch/x86/kernel/cpu/resctrl/rdtgroup.c unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, · line 1349
Quoted 10 lines · Tony Luck
arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-0@@ -1345,10 +1345,13 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,unsigned int size =0 ;int num_b, i; +if (WARN_ON_ONCE (r->scope !=RESCTRL_L2_CACHE && r->scope !=RESCTRL_L3_CACHE )) +return size;
Thanks!
Tony Luck
maintainerPATCH v8 2/8 x86/resctrl: Prepare to split rdt_domain structure
In reply to Tony LuckThe rdt_domain structure is used for both control and monitor features. It is about to be split into separate structures for these two usages because the scope for control and monitoring features for a resource will be different for future resources.
To allow for common code that scans a list of domains looking for a specific domain id, move the "list" and "id" fields into their own structure within the rdt_domain structure.
Add a "type" field to the header to be used later so that callers looking up a domain can be assured that they found one of the expected type.
---
Changes since last version:
Add the "type" field to rdt_domain_hdr.
---
include/linux/resctrl.h +18 -3
@@ -52,10 +52,26 @@ struct resctrl_staged_config {bool have_new_ctrl; }; +enum resctrl_domain_type { +RESCTRL_CTRL_DOMAIN , +RESCTRL_MON_DOMAIN , +}; +/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_domain_hdr - common header for different domain types * @list: all instances of this resource * @id: unique id for this instance +* @type: type of this instance +*/ +struct rdt_domain_hdr { +struct list_head list; +int id; +enum resctrl_domain_type type; +}; + +/** +* struct rdt_domain - group of CPUs sharing a resctrl resource +* @hdr: common header for different domain types * @cpu_mask: which CPUs share this resource * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold * @mbm_total: saved state for MBM total bandwidth
@@ -71,8 +87,7 @@ struct resctrl_staged_config {* by closid */ struct rdt_domain { -struct list_head list; -int id; +struct rdt_domain_hdr hdr;struct cpumask cpu_mask;unsigned long *rmid_busy_llc;struct mbm_state *mbm_total;
arch/x86/kernel/cpu/resctrl/core.c +8 -8
@@ -352,7 +352,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) {/* Find the domain that contains this CPU */ if (cpumask_test_cpu (cpu, &d->cpu_mask))return d;
@@ -401,12 +401,12 @@ struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,return ERR_PTR (-ENODEV );list_for_each (l, &r->domains) { - d = list_entry(l, struct rdt_domain, list); + d = list_entry(l, struct rdt_domain, hdr.list);/* When id is found, return its domain. */ -if (id == d->id) +if (id == d->hdr.id)return d;/* Stop searching when finding id's position in sorted list. */ -if (id < d->id) +if (id < d->hdr.id)break ; }
@@ -544,7 +544,7 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; d = &hw_dom->d_resctrl; - d->id = id; + d->hdr.id = id;cpumask_set_cpu (cpu, &d->cpu_mask);rdt_domain_reconfigure_cdp (r);
@@ -559,11 +559,11 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; } -list_add_tail (&d->list, add_pos); +list_add_tail (&d->hdr.list, add_pos); err =resctrl_online_domain (r, d);if (err) { -list_del (&d->list); +list_del (&d->hdr.list);domain_free (hw_dom); } }
@@ -587,7 +587,7 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)cpumask_clear_cpu (cpu, &d->cpu_mask);if (cpumask_empty (&d->cpu_mask)) {resctrl_offline_domain (r, d); -list_del (&d->list); +list_del (&d->hdr.list);/* * rdt_domain "d" is going to be freed below, so clear
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +8 -8
@@ -67,7 +67,7 @@ int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -144,7 +144,7 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -224,8 +224,8 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, list) { -if (d->id == dom_id) { +list_for_each_entry (d, &r->domains, hdr.list) { +if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;if (r->parse_ctrlval (&data, s, d))
@@ -316,7 +316,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -464,7 +464,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, list) { +list_for_each_entry (dom, &r->domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -474,7 +474,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clo ctrl_val =resctrl_arch_get_config (r, dom, closid, schema->conf_type); -seq_printf (s, r->format_str, dom->id, max_data_width, +seq_printf (s, r->format_str, dom->hdr.id, max_data_width, ctrl_val); sep = true; }
@@ -503,7 +503,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }else {seq_printf (s,"%s:%d=%x\n" , rdtgrp->plr->s->res->name, - rdtgrp->plr->d->id, + rdtgrp->plr->d->hdr.id, rdtgrp->plr->cbm); } }else {
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -340,7 +340,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { err = resctrl_arch_rmid_read(r, d, entry->rmid, QOS_L3_OCCUP_EVENT_ID,
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1 -1
@@ -856,7 +856,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, list) { +list_for_each_entry (d_i, &r->domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, &d_i->cpu_mask);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +22 -22
168 lines not on this page · raw message
Tony Luck
maintainerPATCH v8 3/8 x86/resctrl: Prepare for different scope for control/monitor operations
In reply to Tony LuckResctrl assumes that control and monitor operations on a resource are performed at the same scope.
Prepare for systems that use different scope (specifically L3 scope for cache control and NODE scope for cache occupancy and memory bandwidth monitoring).
Create separate domain lists for control and monitor operations.
Note that errors during initialization of either control or monitor functions on a domain would previously result in that domain being excluded from both control and monitor operations. Now the domains are allocated independently it is no longer required to disable both control and monitor operations if either fail.
---
Changes since last version:
Initialize the "type" in rdt_domain_hdr when creating domains. Check type has expected value before using container_of() to get to the surrounding structure.
Rename "hw_mondom" to "hw_dom" in domain_add_cpu_mon() and in domain_remove_cpu_mon().
Add lockdep_assert_held(&rdtgroup_mutex) to resctrl_offline_mon_domain()
---
include/linux/resctrl.h +12 -6
@@ -170,10 +170,12 @@ enum resctrl_scope {* @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @scope: Scope of this resource +* @ctrl_scope: Scope of this resource for control functions +* @mon_scope: Scope of this resource for monitor functions * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. -* @domains: All domains for this resource +* @ctrl_domains: Control domains for this resource +* @mon_domains: Monitor domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.
@@ -188,10 +190,12 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -enum resctrl_scope scope; +enum resctrl_scope ctrl_scope; +enum resctrl_scope mon_scope;struct resctrl_cache cache;struct resctrl_membw membw; -struct list_head domains; +struct list_head ctrl_domains; +struct list_head mon_domains;char *name;int data_width;u32 default_ctrl;
@@ -237,8 +241,10 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
arch/x86/kernel/cpu/resctrl/internal.h +2 -2
@@ -511,8 +511,8 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn);int rdtgroup_kn_mode_restrict (struct rdtgroup *r,const char *name);int rdtgroup_kn_mode_restore (struct rdtgroup *r,const char *name,umode_t mask); -struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id, -struct list_head **pos); +struct rdt_domain_hdr *rdt_find_domain (struct list_head *h,int id, +struct list_head **pos);ssize_t rdtgroup_schemata_write (struct kernfs_open_file *of,char *buf,size_t nbytes,loff_t off); int rdtgroup_schemata_show(struct kernfs_open_file *of,
arch/x86/kernel/cpu/resctrl/core.c +164 -50
337 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +13 -5
@@ -224,7 +224,7 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) {if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;
@@ -316,7 +316,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -464,7 +464,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, hdr.list) { +list_for_each_entry (dom, &r->ctrl_domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -540,6 +540,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,int rdtgroup_mondata_show (struct seq_file *m,void *arg) {struct kernfs_open_file *of = m->private; +struct rdt_domain_hdr *hdr;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;
@@ -560,12 +561,19 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) evtid = md.u.evtid; r = &rdt_resources_all[resid].r_resctrl; - d =rdt_find_domain (r, domid,NULL ); -if (IS_ERR_OR_NULL (d)) { + hdr =rdt_find_domain (&r->mon_domains, domid,NULL ); +if (IS_ERR_OR_NULL (hdr)) { ret = -ENOENT ; goto out; } +if (WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) { + ret = -EINVAL ; + goto out; + } + + d = container_of(hdr, struct rdt_domain, hdr); +mon_event_read (&rr, r, d, rdtgrp, evtid, false);if (rr.err == -EIO )
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -340,7 +340,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->mon_domains, hdr.list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { err = resctrl_arch_rmid_read(r, d, entry->rmid, QOS_L3_OCCUP_EVENT_ID,
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +2 -2
@@ -292,7 +292,7 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { -int scope = plr->s->res->scope; +int scope = plr->s->res->ctrl_scope;struct cpu_cacheinfo *ci;int ret;int i;
@@ -856,7 +856,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, hdr.list) { +list_for_each_entry (d_i, &r->ctrl_domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, &d_i->cpu_mask);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +30 -24
170 lines not on this page · raw message
Re: [PATCH v8 3/8] x86/resctrl: Prepare for different scope for control/monitor operations
Quoted 22 lines · Tony Luck
Resctrl assumes that control and monitor operations on a resource are performed at the same scope.
Prepare for systems that use different scope (specifically L3 scope for cache control and NODE scope for cache occupancy and memory bandwidth monitoring).
Create separate domain lists for control and monitor operations.
Note that errors during initialization of either control or monitor functions on a domain would previously result in that domain being excluded from both control and monitor operations. Now the domains are allocated independently it is no longer required to disable both control and monitor operations if either fail.
Signed-off-by Tony Luck <[email protected]>---
Changes since last version:
Quoted 8 lines
Initialize the "type" in
rdt_domain_hdrwhen creating domains. Check type has expected value before usingcontainer_of()to get to the surrounding structure.Rename "
hw_mondom" to "hw_dom" indomain_add_cpu_mon()and indomain_remove_cpu_mon().Add
lockdep_assert_held(&rdtgroup_mutex)toresctrl_offline_mon_domain()
Thanks!
Tony Luck
maintainerPATCH v8 4/8 x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
In reply to Tony LuckThe same rdt_domain structure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.
Split into separate rdt_ctrl_domain and rdt_mon_domain structures with just the fields required for control and monitoring respectively.
Similar split of the rdt_hw_domain structure into rdt_hw_ctrl_domain and rdt_hw_mon_domain.
---
Changes since last version:
Fixed two places where line wrapping inside comments were not properly aligned.
---
include/linux/resctrl.h +30 -20
@@ -70,7 +70,25 @@ struct rdt_domain_hdr { };/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_ctrl_domain - group of CPUs sharing a resctrl control resource +* @hdr: common header for different domain types +* @cpu_mask: which CPUs share this resource +* @plr: pseudo-locked region (if any) associated with domain +* @staged_config: parsed configuration to be applied +* @mbps_val: When mba_sc is enabled, this holds the array of user +* specified control values for mba_sc in MBps, indexed +* by closid +*/ +struct rdt_ctrl_domain { +struct rdt_domain_hdr hdr; +struct cpumask cpu_mask; +struct pseudo_lock_region *plr; +struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; +u32 *mbps_val; +}; + +/** +* struct rdt_mon_domain - group of CPUs sharing a resctrl control resource * @hdr: common header for different domain types * @cpu_mask: which CPUs share this resource * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold
@@ -80,13 +98,8 @@ struct rdt_domain_hdr {* @cqm_limbo: worker to periodically read CQM h/w counters * @mbm_work_cpu: worker CPU for MBM h/w counters * @cqm_work_cpu: worker CPU for CQM h/w counters -* @plr: pseudo-locked region (if any) associated with domain -* @staged_config: parsed configuration to be applied -* @mbps_val: When mba_sc is enabled, this holds the array of user -* specified control values for mba_sc in MBps, indexed -* by closid */ -struct rdt_domain { +struct rdt_mon_domain {struct rdt_domain_hdr hdr;struct cpumask cpu_mask;unsigned long *rmid_busy_llc;
@@ -96,9 +109,6 @@ struct rdt_domain {struct delayed_work cqm_limbo;int mbm_work_cpu;int cqm_work_cpu; -struct pseudo_lock_region *plr; -struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; -u32 *mbps_val; };/**
@@ -202,7 +212,7 @@ struct rdt_resource {const char *format_str;int (*parse_ctrlval)(struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d); +struct rdt_ctrl_domain *d);struct list_head evt_list;unsigned long fflags;bool cdp_capable;
@@ -236,15 +246,15 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);* Update the ctrl_val and apply this config right now. * Must be called on one of the domain's CPUs. */ -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val); -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
@@ -260,7 +270,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_domain *d);* Return: * 0 on success, or -EIO, -EINVAL etc on error. */ -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val);/**
@@ -273,7 +283,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid);/**
@@ -285,7 +295,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d);extern unsigned int resctrl_rmid_realloc_threshold;extern unsigned int resctrl_rmid_realloc_limit;
arch/x86/kernel/cpu/resctrl/internal.h +38 -22
131 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/core.c +46 -41
246 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +16 -16
@@ -58,7 +58,7 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r) }int parse_bw (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct resctrl_staged_config *cfg;u32 closid = data->rdtgrp->closid;
@@ -135,7 +135,7 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)* resource type. */ int parse_cbm (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct rdtgroup *rdtgrp = data->rdtgrp;struct resctrl_staged_config *cfg;
@@ -205,7 +205,7 @@ static int parse_line(char *line, struct resctrl_schema *s,struct rdt_resource *r = s->res;struct rdt_parse_data data;char *dom =NULL , *id; -struct rdt_domain *d; +struct rdt_ctrl_domain *d;unsigned long dom_id; if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
@@ -265,11 +265,11 @@ static u32 get_config_index(u32 closid, enum resctrl_conf_type type) } } -static bool apply_config (struct rdt_hw_domain *hw_dom, +static bool apply_config (struct rdt_hw_ctrl_domain *hw_dom,struct resctrl_staged_config *cfg,u32 idx,cpumask_var_t cpu_mask) { -struct rdt_domain *dom = &hw_dom->d_resctrl; +struct rdt_ctrl_domain *dom = &hw_dom->d_resctrl;if (cfg->new_ctrl != hw_dom->ctrl_val[idx]) {cpumask_set_cpu (cpumask_any (&dom->cpu_mask), cpu_mask);
@@ -281,11 +281,11 @@ static bool apply_config(struct rdt_hw_domain *hw_dom,return false; } -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val) { +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);u32 idx =get_config_index (closid, t);struct msr_param msr_param;
@@ -305,11 +305,11 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,int resctrl_arch_update_domains (struct rdt_resource *r,u32 closid) {struct resctrl_staged_config *cfg; -struct rdt_hw_domain *hw_dom; +struct rdt_hw_ctrl_domain *hw_dom;struct msr_param msr_param;enum resctrl_conf_type t; +struct rdt_ctrl_domain *d;cpumask_var_t cpu_mask; -struct rdt_domain *d;u32 idx;if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))
@@ -317,7 +317,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid) msr_param.res =NULL ;list_for_each_entry (d, &r->ctrl_domains, hdr.list) { - hw_dom =resctrl_to_arch_dom (d); + hw_dom =resctrl_to_arch_ctrl_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];if (!cfg->have_new_ctrl)
@@ -447,10 +447,10 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,return ret ?: nbytes; } -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);u32 idx =get_config_index (closid, type);return hw_dom->ctrl_val[idx];
@@ -459,7 +459,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,static void show_doms (struct seq_file *s,struct resctrl_schema *schema,int closid) {struct rdt_resource *r = schema->res; -struct rdt_domain *dom; +struct rdt_ctrl_domain *dom;bool sep = false;u32 ctrl_val;
@@ -521,7 +521,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }void mon_event_read (struct rmid_read *rr,struct rdt_resource *r, -struct rdt_domain *d,struct rdtgroup *rdtgrp, +struct rdt_mon_domain *d,struct rdtgroup *rdtgrp,int evtid,int first) {/*
@@ -541,11 +541,11 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) {struct kernfs_open_file *of = m->private;struct rdt_domain_hdr *hdr; +struct rdt_mon_domain *d;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;union mon_data_bits md; -struct rdt_domain *d;struct rmid_read rr;int ret =0 ;
@@ -572,7 +572,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) goto out; } - d = container_of(hdr, struct rdt_domain, hdr); + d = container_of(hdr, struct rdt_mon_domain, hdr);mon_event_read (&rr, r, d, rdtgrp, evtid, false);
arch/x86/kernel/cpu/resctrl/monitor.c +20 -20
@@ -170,7 +170,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)return 0 ; } -static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_domain *hw_dom, +static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_mon_domain *hw_dom,u32 rmid,enum resctrl_event_id eventid) {
@@ -189,10 +189,10 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom,return NULL ; } -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct arch_mbm_state *am; am =get_arch_mbm_state (hw_dom, rmid, eventid);
@@ -208,9 +208,9 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* Assumes that hardware counters are also reset and thus that there is * no need to record initial non-zero counts. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d) +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);if (is_mbm_total_enabled ()) memset(hw_dom->arch_mbm_total, 0,
@@ -229,11 +229,11 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)return chunks >> shift; } -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);struct arch_mbm_state *am;u64 msr_val, chunks;int ret;
@@ -266,7 +266,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* decrement the count. If the busy count gets to zero on an RMID, we * free the RMID */ -void __check_limbo (struct rdt_domain *d,bool force_free) +void __check_limbo (struct rdt_mon_domain *d,bool force_free) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rmid_entry *entry;
@@ -305,7 +305,7 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } } -bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d) +bool has_busy_rmid (struct rdt_resource *r,struct rdt_mon_domain *d) {return find_first_bit (d->rmid_busy_llc, r->num_rmid) != r->num_rmid; }
@@ -334,7 +334,7 @@ int alloc_rmid(void)static void add_rmid_to_limbo (struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; -struct rdt_domain *d; +struct rdt_mon_domain *d;int cpu, err;u64 val =0 ;
@@ -383,7 +383,7 @@ void free_rmid(u32 rmid)list_add_tail (&entry->list, &rmid_free_lru); } -static struct mbm_state *get_mbm_state (struct rdt_domain *d,u32 rmid, +static struct mbm_state *get_mbm_state (struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id evtid) {switch (evtid) {
@@ -516,13 +516,13 @@ void mon_event_count(void *info)* throttle MSRs already have low percentage values. To avoid * unnecessarily restricting such rdtgroups, we also increase the bandwidth. */ -static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_domain *dom_mbm) +static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_mon_domain *dom_mbm) {u32 closid, rmid, cur_msr_val, new_msr_val;struct mbm_state *pmbm_data, *cmbm_data; +struct rdt_ctrl_domain *dom_mba;u32 cur_bw, delta_bw, user_bw;struct rdt_resource *r_mba; -struct rdt_domain *dom_mba;struct list_head *head;struct rdtgroup *entry;
@@ -600,7 +600,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) } } -static void mbm_update (struct rdt_resource *r,struct rdt_domain *d,int rmid) +static void mbm_update (struct rdt_resource *r,struct rdt_mon_domain *d,int rmid) {struct rmid_read rr;
@@ -640,13 +640,13 @@ void cqm_handle_limbo(struct work_struct *work) {unsigned long delay =msecs_to_jiffies (CQM_LIMBOCHECK_INTERVAL );int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex); r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, cqm_limbo.work); + d = container_of(work, struct rdt_mon_domain, cqm_limbo.work);__check_limbo (d, false);
@@ -656,7 +656,7 @@ void cqm_handle_limbo(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms) +void cqm_setup_limbo_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
@@ -672,9 +672,9 @@ void mbm_handle_overflow(struct work_struct *work)unsigned long delay =msecs_to_jiffies (MBM_OVERFLOW_INTERVAL );struct rdtgroup *prgrp, *crgrp;int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct list_head *head;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex);
@@ -682,7 +682,7 @@ void mbm_handle_overflow(struct work_struct *work) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, mbm_over.work); + d = container_of(work, struct rdt_mon_domain, mbm_over.work);list_for_each_entry (prgrp, &rdt_all_groups, rdtgroup_list) {mbm_update (r, d, prgrp->mon.rmid);
@@ -701,7 +701,7 @@ void mbm_handle_overflow(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms) +void mbm_setup_overflow_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -3
@@ -814,7 +814,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)* Return: true if @cbm overlaps with pseudo-locked region on @d, false * otherwise. */ -bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,unsigned long cbm) +bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_ctrl_domain *d,unsigned long cbm) {unsigned int cbm_len;unsigned long cbm_b;
@@ -841,11 +841,11 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm* if it is not possible to test due to memory allocation issue, * false otherwise. */ -bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d) +bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_ctrl_domain *d) { +struct rdt_ctrl_domain *d_i;cpumask_var_t cpu_with_psl;struct rdt_resource *r; -struct rdt_domain *d_i;bool ret = false;if (!zalloc_cpumask_var (&cpu_with_psl,GFP_KERNEL ))
arch/x86/kernel/cpu/resctrl/rdtgroup.c +31 -31
252 lines not on this page · raw message
Tony Luck
maintainerPATCH v8 5/8 x86/resctrl: Add node-scope to the options for feature scope
In reply to Tony LuckCurrently supported resctrl features are all domain scoped the same as the scope of the L2 or L3 caches.
Add RESCTRL_NODE as a new option for features that are scoped at the same granularity as NUMA nodes. This is needed for Intel's Sub-NUMA Cluster (SNC) feature where monitoring features are node scoped.
---
No changes since last version
---
include/linux/resctrl.h +1 -0
@@ -172,6 +172,7 @@ struct resctrl_schema;enum resctrl_scope {RESCTRL_L2_CACHE =2 ,RESCTRL_L3_CACHE =3 , +RESCTRL_NODE , };/**
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -511,6 +511,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)case RESCTRL_L2_CACHE :case RESCTRL_L3_CACHE :return get_cpu_cacheinfo_id (cpu, scope); +case RESCTRL_NODE : +return cpu_to_node (cpu);default :break ; }
Tony Luck
maintainerPATCH v8 6/8 x86/resctrl: Introduce snc_nodes_per_l3_cache
In reply to Tony LuckIntel Sub-NUMA Cluster (SNC) is a feature that subdivides the CPU cores and memory controllers on a socket into two or more groups. These are presented to the operating system as NUMA nodes.
This may enable some workloads to have slightly lower latency to memory as the memory controller(s) in an SNC node are electrically closer to the CPU cores on that SNC node. This cost may be offset by lower bandwidth since the memory accesses for each core can only be interleaved between the memory controllers on the same SNC node.
Resctrl monitoring on Intel system depends upon attaching RMIDs to tasks to track L3 cache occupancy and memory bandwidth. There is an MSR that controls how the RMIDs are shared between SNC nodes.
The default mode divides them numerically. E.g. when there are two SNC nodes on a socket the lower number half of the RMIDs are given to the first node, the remainder to the second node. This would be difficult to use with the Linux resctrl interface as specific RMID values assigned to resctrl groups are not visible to users.
The other mode divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
Even with this renumbering SNC mode requires several changes in resctrl behavior for correct operation.
Add a global integer "snc_nodes_per_l3_cache" that will show how many SNC nodes share each L3 cache. When this is "1", SNC mode is either not implemented, or not enabled.
A later patch will detect SNC mode and set snc_nodes_per_l3_cache to the appropriate value. For now it remains at the default "1" to indicate SNC mode is not active.
Code that needs to take action when SNC is enabled is:
1) The number of logical RMIDs per L3 cache available for use is the number of physical RMIDs divided by the number of SNC nodes.
2) Likewise the "mon_scale" value must be adjusted for the number of SNC nodes.
3) The RMID renumbering operates when using the value from the IA32_PQR_ASSOC MSR to count accesses by a task. When reading an RMID counter, code must adjust from the logical RMID used to the physical RMID value for the SNC node that it wishes to read and load the adjusted value into the IA32_QM_EVTSEL MSR.
4) The L3 cache is divided between the SNC nodes. So the value reported in the resctrl "size" file is adjusted.
5) The "-o mba_MBps" mount option must be disabled in SNC mode because the monitoring is being done per SNC node, while the bandwidth allocation is still done at the L3 cache scope. Trying to use this feedback loop might result in contradictory changes to the throttling level coming from each of the SNC node bandwidth measurements.
---
Changes since last version:
In commit comment s/redumbering/renumbering/
Move check that SNC is not enabled into supports_mba_mbps().
---
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -446,6 +446,8 @@ DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);extern struct dentry *debugfs_resctrl; +extern int snc_nodes_per_l3_cache; +enum resctrl_res_level {RDT_RESOURCE_L3 ,RDT_RESOURCE_L2 ,
arch/x86/kernel/cpu/resctrl/core.c +6 -0
@@ -48,6 +48,12 @@ int max_name_width, max_data_width;*/ bool rdt_alloc_capable; +/* +* Number of SNC nodes that share each L3 cache. Default is 1 for +* systems that do not support SNC, or have SNC disabled. +*/ +int snc_nodes_per_l3_cache =1 ; +static void mba_wrmsr_intel (struct rdt_ctrl_domain *d,struct msr_param *m,struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/monitor.c +13 -3
@@ -148,8 +148,18 @@ static inline struct rmid_entry *__rmid_entry(u32 rmid)static int __rmid_read (u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +int cpu =smp_processor_id (); +int rmid_offset =0 ;u64 msr_val; +/* +* When SNC mode is on, need to compute the offset to read the +* physical RMID counter for the node to which this CPU belongs. +*/ +if (snc_nodes_per_l3_cache >1 ) + rmid_offset = (cpu_to_node (cpu) % snc_nodes_per_l3_cache) * r->num_rmid; +/* * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured * with a valid event code for supported resource type and the bits
@@ -158,7 +168,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)* IA32_QM_CTR.Error (bit 63) and IA32_QM_CTR.Unavailable (bit 62) * are error bits. */ -wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid); +wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid + rmid_offset);rdmsrl (MSR_IA32_QM_CTR , msr_val);if (msr_val &RMID_VAL_ERROR )
@@ -783,8 +793,8 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)int ret; resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size *1024 ; - hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale; - r->num_rmid = boot_cpu_data.x86_cache_max_rmid +1 ; + hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache; + r->num_rmid = (boot_cpu_data.x86_cache_max_rmid +1 ) / snc_nodes_per_l3_cache; hw_res->mbm_width =MBM_CNTR_WIDTH_BASE ;if (mbm_offset >0 && mbm_offset <=MBM_CNTR_WIDTH_OFFSET_MAX )
arch/x86/kernel/cpu/resctrl/rdtgroup.c +3 -2
@@ -1357,7 +1357,7 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, } } -return size; +return size / snc_nodes_per_l3_cache; }/**
@@ -2207,7 +2207,8 @@ static bool supports_mba_mbps(void)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA ].r_resctrl;return (is_mbm_local_enabled () && - r->alloc_capable &&is_mba_linear ()); + r->alloc_capable &&is_mba_linear () && + snc_nodes_per_l3_cache ==1 ); }/*
Tony Luck
maintainerPATCH v8 7/8 x86/resctrl: Sub NUMA Cluster detection and enable
In reply to Tony LuckThere isn't a simple h/w bit that indicates whether a CPU is running in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the ratio of NUMA nodes to L3 cache instances.
When SNC mode is detected, reconfigure the RMID counters by updating the MSR_RMID_SNC_CONFIG MSR on each socket as CPUs are seen.
Clearing bit zero of the MSR divides the RMIDs and renumbers the ones on the second SNC node to start from zero. An earlier commit includes all the required changes in Linux to operate in this reconfigured mode.
---
Changes since last version:
Moved kfree(node_caches); earlier, to the earliest point where it is no longer needed.
Added Granite Rapids to list of CPU models that support SNC mode.
---
arch/x86/include/asm/msr-index.h +1 -0
@@ -1100,6 +1100,7 @@#define MSR_IA32_QM_CTR 0xc8e#define MSR_IA32_PQR_ASSOC 0xc8f#define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0#define MSR_IA32_L2_CBM_BASE 0xd10#define MSR_IA32_MBA_THRTL_BASE 0xd50
arch/x86/kernel/cpu/resctrl/core.c +92 -0
125 lines not on this page · raw message
Tony Luck
maintainerPATCH v8 8/8 x86/resctrl: Update documentation with Sub-NUMA cluster changes
In reply to Tony LuckWith Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
---
Changes since v5:
Added addtional details about challenges tracking tasks when SNC mode is enabled.
---
Documentation/arch/x86/resctrl.rst +20 -3
@@ -345,9 +345,9 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. Each of these directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for
@@ -452,6 +452,23 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled Linux may load balance tasks between Sub-NUMA +nodes much more readily than between regular NUMA nodes since the CPUs +on Sub-NUMA nodes share the same L3 cache and the system may report +the NUMA distance between Sub-NUMA nodes with a lower value than used +for regular NUMA nodes. Users who do not bind tasks to the CPUs of a +specific Sub-NUMA node must read the "llc_occupancy", "mbm_total_bytes", +and "mbm_local_bytes" for all Sub-NUMA nodes where the tasks may execute +to get the full view of traffic for which the tasks were the source. + +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================
Re: [PATCH v8 8/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
Quoted 49 lines · Tony Luck
With Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
Signed-off-by Tony Luck <[email protected]>---
Changes since v5:
Added addtional details about challenges tracking tasks when SNC mode is enabled.
---
Documentation/arch/x86/resctrl.rst +13
-3@@ -345,9 +345,9 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. Each of these directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for@@ -452,6 +452,23 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled Linux may load balance tasks between Sub-NUMA +nodes much more readily than between regular NUMA nodes since the CPUs +on Sub-NUMA nodes share the same L3 cache and the system may report +the NUMA distance between Sub-NUMA nodes with a lower value than used +for regular NUMA nodes. Users who do not bind tasks to the CPUs of a +specific Sub-NUMA node must read the "llc_occupancy", "mbm_total_bytes", +and "mbm_local_bytes" for all Sub-NUMA nodes where the tasks may execute +to get the full view of traffic for which the tasks were the source.
Documentation/arch/x86/resctrl.rst and 0xA are not. On a system with a 20-bit mask each bit represents 5% · line 474
Quoted 10 lines
Documentation/arch/x86/resctrl.rst +7
-0+ +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================
Sounds better, thanks.
Reinette Chatre
maintainerRe: [PATCH v8 0/8] Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckTony,
Quoted 9 lines · Tony Luck
The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows for monitoring features to be used (with the caveat that memory accesses between different SNC NUMA nodes may still not be counted accurately.
Almost. For reference:
https://lore.kernel.org/lkml/[email protected]/
No need to send a new series just for this, but this series does find itself at the back of my queue.
Quoted 5 lines
Note that this patch series improves resctrl reporting considerably on systems with SNC enabled, but there will still be some anomalies for processes accessing memory from other sub-NUMA nodes.
Signed-off-by Tony Luck <[email protected]>
Reinette
RE: [PATCH v8 0/8] Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckHi Tony,
I applied this patch series to kernel v6.5 and v6.6-rc4, but the kernel cannot be booted. Could you tell me what kernel version this patch series is based on?
Best regards,
Shaopeng TAN
Luck, Tony
maintainerRE: [PATCH v8 0/8] Add support for Sub-NUMA cluster (SNC) systems
Quoted 2 lines
I applied this patch series to kernel v6.5 and v6.6-rc4, but the kernel cannot be booted. Could you tell me what kernel version this patch series is based on?
Hi Shaopeng,
Patches are based on v6.6-rc3 (see the "base-commit" line at the end of the cover letter).
Which CPU family/model/stepping & microcode are you testing on?
Are there error or panic messages when it does not boot?
-Tony
Reinette Chatre
maintainerRe: [PATCH v8 0/8] Add support for Sub-NUMA cluster (SNC) systems
Hi Tony,
Quoted 10 lines · Tony
Quoted 2 lines
I applied this patch series to kernel v6.5 and v6.6-rc4, but the kernel cannot be booted. Could you tell me what kernel version this patch series is based on?
Hi Shaopeng,
Patches are based on v6.6-rc3 (see the "base-commit" line at the end of the cover letter).
Which CPU family/model/stepping & microcode are you testing on?
Are there error or panic messages when it does not boot?
There is no need to ask Shaopeng for more information. If you test this series you will immediately learn that it is broken. Reading the series I only made it to patch #3 where I realized that this has not been tested before posting.
Reinette
Tony Luck
maintainerv9 cover Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckThe Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows monitoring features to be used. With the caveat that users must be aware that Linux may migrate tasks more frequently between SNC nodes than between "regular" NUMA nodes, so reading counters from all SNC nodes may be needed to get a complete picture of activity for tasks.
Cache and memory bandwidth allocation features continue to operate at the scope of the L3 cache.
Changes since v6 (see individual patches for specifics):
v7 - had some git format-patch disaster and one of the patches couldn't
be applied.
v8 - Was rushed. Somehow I booted the wrong kernel while testing and
let escape a brown-paper-bag bug that crashed duing boot. Sincere apologies to all who wasted time reading this series, or trying to boot it.
v9 - Tested (Really! I checked timestamps in dmesg, and all sorts of
other checks to make sure I was really looking at a kernel built with these patches).
Rebased to tip/master October 20th since that has several other resctrl changes staged resdy for next merge window. No significant collisions, just noise where "git am" would not automatically apply. New base is:
3300447612b2 ("Merge branch into tip/master: 'x86/tdx'")
Fixed the brown-paper-bag bug from v8.
Added Peter's "Reviewed-by" where offered (except on patch 3 which had the aforementioned bug).
Tony Luck (8):
x86/resctrl: Prepare for new domain scope x86/resctrl: Prepare to split rdt_domain structure x86/resctrl: Prepare for different scope for control/monitor
operations
x86/resctrl: Split the rdt_domain and rdt_hw_domain structures x86/resctrl: Add node-scope to the options for feature scope x86/resctrl: Introduce snc_nodes_per_l3_cache x86/resctrl: Sub NUMA Cluster detection and enable x86/resctrl: Update documentation with Sub-NUMA cluster changes
Documentation/arch/x86/resctrl.rst | 23 +- include/linux/resctrl.h | 85 +++-- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 66 ++-- arch/x86/kernel/cpu/resctrl/core.c | 402 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 58 ++-- arch/x86/kernel/cpu/resctrl/monitor.c | 58 ++-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 14 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 132 +++---- 9 files changed, 592 insertions(+), 247 deletions(-)
base-commit: 3300447612b2adbc05cbb90e5d1cb288f19c40c6
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v9 1/8 x86/resctrl: Prepare for new domain scope
Resctrl resources operate on subsets of CPUs in the system with the defining attribute of each subset being an instance of a particular level of cache. E.g. all CPUs sharing an L3 cache would be part of the same domain.
In preparation for features that are scoped at the NUMA node level change the code from explicit references to "cache_level" to a more generic scope. At this point the only options for this scope are groups of CPUs that share an L2 cache or L3 cache.
Provide a more detailed warning message if a domain id cannot be found when adding a CPU. Just check and silent return if the domain id can't be found when removing a CPU.
---
Changes since v6:
s/-EINVAL/0/ for return value of rdtgroup_cbm_to_size() Added Peter's review tag
include/linux/resctrl.h | 9 +++++-- arch/x86/kernel/cpu/resctrl/core.c | 33 ++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 ++++- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 +++- 4 files changed, 43 insertions(+), 10 deletions(-)
include/linux/resctrl.h +7 -2
@@ -144,13 +144,18 @@ struct resctrl_membw {struct rdt_parse_data ;struct resctrl_schema ; +enum resctrl_scope { +RESCTRL_L2_CACHE =2 , +RESCTRL_L3_CACHE =3 , +}; +/** * struct rdt_resource - attributes of a resctrl resource * @rid: The index of the resource * @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @cache_level: Which cache level defines scope of this resource +* @scope: Scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. * @domains: All domains for this resource
@@ -168,7 +173,7 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -int cache_level; +enum resctrl_scope scope;struct resctrl_cache cache;struct resctrl_membw membw;struct list_head domains;
arch/x86/kernel/cpu/resctrl/core.c +27 -6
@@ -65,7 +65,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L3, .name = "L3", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_L3), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -79,7 +79,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L2, .name = "L2", - .cache_level = 2, + .scope = RESCTRL_L2_CACHE, .domains = domain_init(RDT_RESOURCE_L2), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -93,7 +93,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_MBA, .name = "MB", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_MBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -105,7 +105,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_SMBA, .name = "SMBA", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_SMBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -491,6 +491,19 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)return 0 ; } +static int get_domain_id_from_scope (int cpu,enum resctrl_scope scope) +{ +switch (scope) { +case RESCTRL_L2_CACHE : +case RESCTRL_L3_CACHE : +return get_cpu_cacheinfo_id (cpu, scope); +default : +break ; + } + +return -EINVAL ; +} +/* * domain_add_cpu - Add a cpu to a resource's domain list. *
@@ -506,12 +519,17 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)*/ static void domain_add_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct list_head *add_pos =NULL ;struct rdt_hw_domain *hw_dom;struct rdt_domain *d;int err; +if (id <0 ) { +pr_warn_once ("Can't find domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->scope, r->name); +return ; + } d =rdt_find_domain (r, id, &add_pos);if (IS_ERR (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
@@ -556,10 +574,13 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)static void domain_remove_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct rdt_hw_domain *hw_dom;struct rdt_domain *d; +if (id <0 ) +return ; + d =rdt_find_domain (r, id,NULL );if (IS_ERR_OR_NULL (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +5 -1
@@ -292,10 +292,14 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { +int scope = plr->s->res->scope;struct cpu_cacheinfo *ci;int ret;int i; +if (WARN_ON_ONCE (scope !=RESCTRL_L2_CACHE && scope !=RESCTRL_L3_CACHE )) +return -ENODEV ; +/* Pick the first cpu we find that is associated with the cache. */ plr->cpu =cpumask_first (&plr->d->cpu_mask);
@@ -311,7 +315,7 @@ static int pseudo_lock_region_init(struct pseudo_lock_region *plr) plr->size =rdtgroup_cbm_to_size (plr->s->res, plr->d, plr->cbm);for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == plr->s->res->cache_level) { +if (ci->info_list[i].level == scope) { plr->line_size = ci->info_list[i].coherency_line_size;return 0 ; }
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -1
@@ -1413,10 +1413,13 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,unsigned int size =0 ;int num_b, i; +if (WARN_ON_ONCE (r->scope !=RESCTRL_L2_CACHE && r->scope !=RESCTRL_L3_CACHE )) +return size; + num_b =bitmap_weight (&cbm, r->cache.cbm_len); ci =get_cpu_cacheinfo (cpumask_any (&d->cpu_mask));for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == r->cache_level) { +if (ci->info_list[i].level == r->scope) { size = ci->info_list[i].size / r->cache.cbm_len * num_b;break ; }
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v9 1/8] x86/resctrl: Prepare for new domain scope
Hi Tony,
On 10/20/2023 2:30 PM, Tony Luck wrote:
...
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-1@@ -506,12 +519,17 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)*/ static void domain_add_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct list_head *add_pos =NULL ;struct rdt_hw_domain *hw_dom;struct rdt_domain *d;
arch/x86/kernel/cpu/resctrl/core.c static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom) · line 535
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +5
-0int err; +if (id <0 ) { +pr_warn_once ("Can't find domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->scope, r->name); +return ; + } d =rdt_find_domain (r, id, &add_pos);if (IS_ERR (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
From what I can tell the original implementation relied on implementation of rdt_find_domain() to do error checking of the id value, printing the above pr_warn() if id was found to be invalid. In your change the error checking on id is moved earlier yet this original behavior is maintained. How could rdt_find_domain() possibly fail for this reason at this point?
arch/x86/kernel/cpu/resctrl/pseudo_lock.c static void pseudo_lock_region_clear(struct pseudo_lock_region *plr) · line 295
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1
-0@@ -292,10 +292,14 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { +int scope = plr->s->res->scope;
enum resctrl_scope ?
arch/x86/kernel/cpu/resctrl/pseudo_lock.c static void pseudo_lock_region_clear(struct pseudo_lock_region *plr) · line 298
Quoted 3 lines
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +0
-0struct cpu_cacheinfo *ci;int ret;int i;
Reinette
Tony Luck
maintainerPATCH v9 2/8 x86/resctrl: Prepare to split rdt_domain structure
In reply to Tony LuckThe rdt_domain structure is used for both control and monitor features. It is about to be split into separate structures for these two usages because the scope for control and monitoring features for a resource will be different for future resources.
To allow for common code that scans a list of domains looking for a specific domain id, move the "list" and "id" fields into their own structure within the rdt_domain structure.
Add a "type" field to the header to be used later so that callers looking up a domain can be assured that they found one of the expected type.
---
Changes since v6:
Add the "type" field to rdt_domain_hdr.
include/linux/resctrl.h | 21 +++++++++-- arch/x86/kernel/cpu/resctrl/core.c | 16 ++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 16 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 44 +++++++++++------------ 6 files changed, 58 insertions(+), 43 deletions(-)
include/linux/resctrl.h +18 -3
@@ -52,10 +52,26 @@ struct resctrl_staged_config {bool have_new_ctrl; }; +enum resctrl_domain_type { +RESCTRL_CTRL_DOMAIN , +RESCTRL_MON_DOMAIN , +}; +/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_domain_hdr - common header for different domain types * @list: all instances of this resource * @id: unique id for this instance +* @type: type of this instance +*/ +struct rdt_domain_hdr { +struct list_head list; +int id; +enum resctrl_domain_type type; +}; + +/** +* struct rdt_domain - group of CPUs sharing a resctrl resource +* @hdr: common header for different domain types * @cpu_mask: which CPUs share this resource * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold * @mbm_total: saved state for MBM total bandwidth
@@ -71,8 +87,7 @@ struct resctrl_staged_config {* by closid */ struct rdt_domain { -struct list_head list; -int id; +struct rdt_domain_hdr hdr;struct cpumask cpu_mask;unsigned long *rmid_busy_llc;struct mbm_state *mbm_total;
arch/x86/kernel/cpu/resctrl/core.c +8 -8
@@ -356,7 +356,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) {/* Find the domain that contains this CPU */ if (cpumask_test_cpu (cpu, &d->cpu_mask))return d;
@@ -405,12 +405,12 @@ struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,return ERR_PTR (-ENODEV );list_for_each (l, &r->domains) { - d = list_entry(l, struct rdt_domain, list); + d = list_entry(l, struct rdt_domain, hdr.list);/* When id is found, return its domain. */ -if (id == d->id) +if (id == d->hdr.id)return d;/* Stop searching when finding id's position in sorted list. */ -if (id < d->id) +if (id < d->hdr.id)break ; }
@@ -548,7 +548,7 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; d = &hw_dom->d_resctrl; - d->id = id; + d->hdr.id = id;cpumask_set_cpu (cpu, &d->cpu_mask);rdt_domain_reconfigure_cdp (r);
@@ -563,11 +563,11 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; } -list_add_tail (&d->list, add_pos); +list_add_tail (&d->hdr.list, add_pos); err =resctrl_online_domain (r, d);if (err) { -list_del (&d->list); +list_del (&d->hdr.list);domain_free (hw_dom); } }
@@ -591,7 +591,7 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)cpumask_clear_cpu (cpu, &d->cpu_mask);if (cpumask_empty (&d->cpu_mask)) {resctrl_offline_domain (r, d); -list_del (&d->list); +list_del (&d->hdr.list);/* * rdt_domain "d" is going to be freed below, so clear
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +8 -8
@@ -67,7 +67,7 @@ int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -146,7 +146,7 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -226,8 +226,8 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, list) { -if (d->id == dom_id) { +list_for_each_entry (d, &r->domains, hdr.list) { +if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;if (r->parse_ctrlval (&data, s, d))
@@ -318,7 +318,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -466,7 +466,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, list) { +list_for_each_entry (dom, &r->domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -476,7 +476,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clo ctrl_val =resctrl_arch_get_config (r, dom, closid, schema->conf_type); -seq_printf (s, r->format_str, dom->id, max_data_width, +seq_printf (s, r->format_str, dom->hdr.id, max_data_width, ctrl_val); sep = true; }
@@ -505,7 +505,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }else {seq_printf (s,"%s:%d=%x\n" , rdtgrp->plr->s->res->name, - rdtgrp->plr->d->id, + rdtgrp->plr->d->hdr.id, rdtgrp->plr->cbm); } }else {
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -340,7 +340,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { err = resctrl_arch_rmid_read(r, d, entry->rmid, QOS_L3_OCCUP_EVENT_ID,
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1 -1
@@ -856,7 +856,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, list) { +list_for_each_entry (d_i, &r->domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, &d_i->cpu_mask);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +22 -22
168 lines not on this page · raw message
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v9 2/8] x86/resctrl: Prepare to split rdt_domain structure
Hi Tony,
Quoted 8 lines · Tony Luck
The
rdt_domainstructure is used for both control and monitor features. It is about to be split into separate structures for these two usages because the scope for control and monitoring features for a resource will be different for future resources.To allow for common code that scans a list of domains looking for a specific domain id, move the "list" and "id" fields into their own structure within the
rdt_domainstructure.
The motivation for this split is not clear to me. Here the motivation is to support the code that needs to traverse both lists. Later (patch #4) the motivation is that what remains should be "just the fields required for control and monitoring respectively". The comment "common header for different domain types" also makes me think this new structure should contain all common members?
The reason why the motivation needs to be clear in this regard is because there is a common field, cpu_mask, that did not make it into the header. Should it?
Quoted 2 lines
Add a "type" field to the header to be used later so that callers looking up a domain can be assured that they found one of the expected type.
Please move this addition to when it is used.
Reinette
Tony Luck
maintainerPATCH v9 3/8 x86/resctrl: Prepare for different scope for control/monitor operations
In reply to Tony LuckResctrl assumes that control and monitor operations on a resource are performed at the same scope.
Prepare for systems that use different scope (specifically L3 scope for cache control and NODE scope for cache occupancy and memory bandwidth monitoring).
Create separate domain lists for control and monitor operations.
Note that errors during initialization of either control or monitor functions on a domain would previously result in that domain being excluded from both control and monitor operations. Now the domains are allocated independently it is no longer required to disable both control and monitor operations if either fail.
---
Changes since v6:
Initialize the "type" in rdt_domain_hdr when creating domains. Check type has expected value before using container_of() to get to the surrounding structure.
Rename "hw_mondom" to "hw_dom" in domain_add_cpu_mon() and in domain_remove_cpu_mon().
Add lockdep_assert_held(&rdtgroup_mutex) to resctrl_offline_mon_domain()
Changes since v8:
Fix the brown-paper-bag bugs using NULL result from rdt_find_domain()
include/linux/resctrl.h | 18 +- arch/x86/kernel/cpu/resctrl/internal.h | 4 +- arch/x86/kernel/cpu/resctrl/core.c | 216 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 18 +- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 54 +++--- 7 files changed, 225 insertions(+), 91 deletions(-)
include/linux/resctrl.h +12 -6
@@ -170,10 +170,12 @@ enum resctrl_scope {* @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @scope: Scope of this resource +* @ctrl_scope: Scope of this resource for control functions +* @mon_scope: Scope of this resource for monitor functions * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. -* @domains: All domains for this resource +* @ctrl_domains: Control domains for this resource +* @mon_domains: Monitor domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.
@@ -188,10 +190,12 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -enum resctrl_scope scope; +enum resctrl_scope ctrl_scope; +enum resctrl_scope mon_scope;struct resctrl_cache cache;struct resctrl_membw membw; -struct list_head domains; +struct list_head ctrl_domains; +struct list_head mon_domains;char *name;int data_width;u32 default_ctrl;
@@ -237,8 +241,10 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
arch/x86/kernel/cpu/resctrl/internal.h +2 -2
@@ -520,8 +520,8 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn);int rdtgroup_kn_mode_restrict (struct rdtgroup *r,const char *name);int rdtgroup_kn_mode_restore (struct rdtgroup *r,const char *name,umode_t mask); -struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id, -struct list_head **pos); +struct rdt_domain_hdr *rdt_find_domain (struct list_head *h,int id, +struct list_head **pos);ssize_t rdtgroup_schemata_write (struct kernfs_open_file *of,char *buf,size_t nbytes,loff_t off); int rdtgroup_schemata_show(struct kernfs_open_file *of,
arch/x86/kernel/cpu/resctrl/core.c +165 -51
339 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +13 -5
@@ -226,7 +226,7 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) {if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;
@@ -318,7 +318,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -466,7 +466,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, hdr.list) { +list_for_each_entry (dom, &r->ctrl_domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -542,6 +542,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,int rdtgroup_mondata_show (struct seq_file *m,void *arg) {struct kernfs_open_file *of = m->private; +struct rdt_domain_hdr *hdr;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;
@@ -562,12 +563,19 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) evtid = md.u.evtid; r = &rdt_resources_all[resid].r_resctrl; - d =rdt_find_domain (r, domid,NULL ); -if (IS_ERR_OR_NULL (d)) { + hdr =rdt_find_domain (&r->mon_domains, domid,NULL ); +if (IS_ERR_OR_NULL (hdr)) { ret = -ENOENT ; goto out; } +if (WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) { + ret = -EINVAL ; + goto out; + } + + d = container_of(hdr, struct rdt_domain, hdr); +mon_event_read (&rr, r, d, rdtgrp, evtid, false);if (rr.err == -EIO )
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -340,7 +340,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->mon_domains, hdr.list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { err = resctrl_arch_rmid_read(r, d, entry->rmid, QOS_L3_OCCUP_EVENT_ID,
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +2 -2
@@ -292,7 +292,7 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { -int scope = plr->s->res->scope; +int scope = plr->s->res->ctrl_scope;struct cpu_cacheinfo *ci;int ret;int i;
@@ -856,7 +856,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, hdr.list) { +list_for_each_entry (d_i, &r->ctrl_domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, &d_i->cpu_mask);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +30 -24
170 lines not on this page · raw message
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v9 3/8] x86/resctrl: Prepare for different scope for control/monitor operations
Hi Tony,
Quoted 6 lines · Tony Luck
Resctrl assumes that control and monitor operations on a resource are performed at the same scope.
Prepare for systems that use different scope (specifically L3 scope for cache control and NODE scope for cache occupancy and memory bandwidth monitoring).
The first paragraph is a generalization of all resources but then the second paragraph only mentions L3. In preparation for readers seeing that only L3 resource's monitoring scope initialized it may help to be specific here that resctrl only supports monitoring on L3.
Quoted 1 line
Create separate domain lists for control and monitor operations.
Please do note that an upcoming change changes the domain list locking. I expect the transition to go smoothly with the locking and list type translating to both lists so just sharing for your information in case you are now aware:
https://lore.kernel.org/lkml/[email protected]/
Quoted 9 lines
Note that errors during initialization of either control or monitor functions on a domain would previously result in that domain being excluded from both control and monitor operations. Now the domains are allocated independently it is no longer required to disable both control and monitor operations if either fail.
Signed-off-by Tony Luck <[email protected]>---
...
arch/x86/kernel/cpu/resctrl/core.c struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) · line 365
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-1@@ -356,7 +359,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d; -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) {/* Find the domain that contains this CPU */ if (cpumask_test_cpu (cpu, &d->cpu_mask))return d;
This appears to silently turn a generic "get_domain_from_cpu()" into code that assumes control domain. This works for now for the existing users but can trip up future changes. I think this would be better renamed to get_ctrl_domain_from_cpu() or something better.
arch/x86/kernel/cpu/resctrl/core.c void rdt_ctrl_update(void *arg) · line 394
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-1@@ -388,29 +391,39 @@ void rdt_ctrl_update(void *arg) }/* -* rdt_find_domain - Find a domain in a resource that matches input resource id +* rdt_find_domain - Find a domain in one of a resource domain lists.
The above does not sound right.
"one of a resource domain lists" -> "a resource domain list" or "one of the resource domain lists" or ?
arch/x86/kernel/cpu/resctrl/core.c void rdt_ctrl_update(void *arg) · line 398
Quoted 7 lines
arch/x86/kernel/cpu/resctrl/core.c +3
-3* -* Search resource r's domain list to find the resource id. If the resource -* id is found in a domain, return the domain. Otherwise, if requested by -* caller, return the first domain whose id is bigger than the input id. +* Search the list to find the resource id. If the resource id is found +* in a domain, return the domain. Otherwise, if requested by caller, +* return the first domain whose id is bigger than the input id.
The above does not sound right. First there is "Search the list to find the resource id." The resource id is not involved in this code, do you mean "domain id"? Also later "If the resource id is found in a domain," what does that mean here?
arch/x86/kernel/cpu/resctrl/core.c void rdt_ctrl_update(void *arg) · line 402
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/core.c +3
-0* The domain list is sorted by id in ascending order. +* +* If an existing domain in the resource r's domain list matches the cpu's +* resource id, add the cpu in the domain.
domain id?
arch/x86/kernel/cpu/resctrl/core.c void rdt_ctrl_update(void *arg) · line 409
Quoted 7 lines
arch/x86/kernel/cpu/resctrl/core.c +7
-0+* +* Otherwise, caller will allocate a new domain and insert into the right position +* in the domain list sorted by id in ascending order. +* +* The order in the domain list is visible to users when we print entries +* in the schemata file and schemata input is validated to have the same order +* as this list.
Please document what the caller does at the caller. Also, please always use "CPU", not "cpu". Finally, please do not impersonate code (no "we"). I understand you are copying original comments, no need to propagate these issues.
Quoted 59 lines
arch/x86/kernel/cpu/resctrl/core.c +11
-24*/ -struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id, -struct list_head **pos) +struct rdt_domain_hdr *rdt_find_domain (struct list_head *h,int id, +struct list_head **pos) { -struct rdt_domain *d; +struct rdt_domain_hdr *d;struct list_head *l;if (id <0 )return ERR_PTR (-ENODEV ); -list_for_each (l, &r->domains) { - d = list_entry(l, struct rdt_domain, hdr.list); +list_for_each (l, h) { + d = list_entry(l, struct rdt_domain_hdr, list);/* When id is found, return its domain. */ -if (id == d->hdr.id) +if (id == d->id)return d;/* Stop searching when finding id's position in sorted list. */ -if (id < d->hdr.id) +if (id < d->id)break ; }@@ -504,39 +517,33 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)return -EINVAL ; } -/* -* domain_add_cpu - Add a cpu to a resource's domain list. -* -* If an existing domain in the resource r's domain list matches the cpu's -* resource id, add the cpu in the domain. -* -* Otherwise, a new domain is allocated and inserted into the right position -* in the domain list sorted by id in ascending order. -* -* The order in the domain list is visible to users when we print entries -* in the schemata file and schemata input is validated to have the same order -* as this list. -*/ -static void domain_add_cpu (int cpu,struct rdt_resource *r) +static void domain_add_cpu_ctrl (int cpu,struct rdt_resource *r) { -int id =get_domain_id_from_scope (cpu, r->scope); +int id =get_domain_id_from_scope (cpu, r->ctrl_scope);struct list_head *add_pos =NULL ;struct rdt_hw_domain *hw_dom; +struct rdt_domain_hdr *hdr;struct rdt_domain *d;int err;if (id <0 ) { -pr_warn_once ("Can't find domain id for CPU:%d scope:%d for resource %s\n" , - cpu, r->scope, r->name); + pr_warn_once("Can't find control domain id for CPU:%d scope:%d for resource %s\n",
arch/x86/kernel/cpu/resctrl/core.c static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope) · line 537
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +5
-3+ cpu, r->ctrl_scope, r->name);return ; } - d =rdt_find_domain (r, id, &add_pos); -if (IS_ERR (d)) { -pr_warn ("Couldn't find cache id for CPU %d\n" , cpu); + + hdr =rdt_find_domain (&r->ctrl_domains, id, &add_pos); +if (IS_ERR (hdr)) { +pr_warn ("Couldn't find control scope id=%d for CPU %d\n" , id, cpu);
How can the failure in the error message be encountered at this point?
Quoted 86 lines
arch/x86/kernel/cpu/resctrl/core.c +58
-4return ; } -if (d) { +if (hdr) { +if (WARN_ON_ONCE (hdr->type !=RESCTRL_CTRL_DOMAIN )) +return ; + + d = container_of(hdr, struct rdt_domain, hdr); +cpumask_set_cpu (cpu, &d->cpu_mask);if (r->cache.arch_has_per_cpu_cfg)rdt_domain_reconfigure_cdp (r);@@ -549,48 +556,115 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r) d = &hw_dom->d_resctrl; d->hdr.id = id; + d->hdr.type =RESCTRL_CTRL_DOMAIN ;cpumask_set_cpu (cpu, &d->cpu_mask);rdt_domain_reconfigure_cdp (r); -if (r->alloc_capable &&domain_setup_ctrlval (r, d)) { +if (domain_setup_ctrlval (r, d)) { +domain_free (hw_dom); +return ; + } + +list_add_tail (&d->hdr.list, add_pos); + + err =resctrl_online_ctrl_domain (r, d); +if (err) { +list_del (&d->hdr.list);domain_free (hw_dom); + } +} + +static void domain_add_cpu_mon (int cpu,struct rdt_resource *r) +{ +int id =get_domain_id_from_scope (cpu, r->mon_scope); +struct list_head *add_pos =NULL ; +struct rdt_hw_domain *hw_dom; +struct rdt_domain_hdr *hdr; +struct rdt_domain *d; +int err; + +if (id <0 ) { +pr_warn_once ("Can't find monitor domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->mon_scope, r->name); +return ; + } + + hdr =rdt_find_domain (&r->mon_domains, id, &add_pos); +if (IS_ERR (hdr)) { +pr_warn ("Couldn't find monitor scope id=%d for CPU %d\n" , id, cpu); +return ; + } + +if (hdr) { +if (WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) +return ; + + d = container_of(hdr, struct rdt_domain, hdr); + +cpumask_set_cpu (cpu, &d->cpu_mask);return ; } -if (r->mon_capable &&arch_domain_mbm_alloc (r->num_rmid, hw_dom)) { + hw_dom =kzalloc_node (sizeof (*hw_dom),GFP_KERNEL ,cpu_to_node (cpu)); +if (!hw_dom) +return ; + + d = &hw_dom->d_resctrl; + d->hdr.id = id; + d->hdr.type =RESCTRL_MON_DOMAIN ; +cpumask_set_cpu (cpu, &d->cpu_mask); + +if (arch_domain_mbm_alloc (r->num_rmid, hw_dom)) {domain_free (hw_dom);return ; }list_add_tail (&d->hdr.list, add_pos); - err =resctrl_online_domain (r, d);
arch/x86/kernel/cpu/resctrl/core.c static void domain_add_cpu(int cpu, struct rdt_resource *r) · line 633
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +3
-1+ err =resctrl_online_mon_domain (r, d);if (err) {list_del (&d->hdr.list);domain_free (hw_dom); } } -static void domain_remove_cpu (int cpu,struct rdt_resource *r) +/* +* domain_add_cpu - Add a cpu to either/both resource's domain lists.
cpu -> CPU (please check all changelog and comments)
Quoted 13 lines
arch/x86/kernel/cpu/resctrl/core.c +9
-0+*/ +static void domain_add_cpu (int cpu,struct rdt_resource *r) +{ +if (r->alloc_capable) +domain_add_cpu_ctrl (cpu, r); +if (r->mon_capable) +domain_add_cpu_mon (cpu, r); +} +@@ -3914,18 +3916,22 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)return 0 ; }
arch/x86/kernel/cpu/resctrl/core.c static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) · line 3924
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-3-int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d) +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d) { -int err; -lockdep_assert_held (&rdtgroup_mutex);if (supports_mba_mbps () && r->rid ==RDT_RESOURCE_MBA )/* RDT_RESOURCE_MBA is never mon_capable */
This comment was used to justify the early exit based on later "if (!r->mon_capable)" test. With the test removed this comment becomes unnecessary.
Reinette
Tony Luck
maintainerPATCH v9 4/8 x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
In reply to Tony LuckThe same rdt_domain structure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.
Split into separate rdt_ctrl_domain and rdt_mon_domain structures with just the fields required for control and monitoring respectively.
Similar split of the rdt_hw_domain structure into rdt_hw_ctrl_domain and rdt_hw_mon_domain.
---
Changes since v6:
Changes since last version:
Fixed two places where line wrapping inside comments were not properly aligned.
Added Peter's review tag
include/linux/resctrl.h | 50 +++++++------ arch/x86/kernel/cpu/resctrl/internal.h | 60 ++++++++++------ arch/x86/kernel/cpu/resctrl/core.c | 87 ++++++++++++----------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 32 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 40 +++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++-------- 7 files changed, 184 insertions(+), 153 deletions(-)
include/linux/resctrl.h +30 -20
@@ -70,7 +70,25 @@ struct rdt_domain_hdr { };/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_ctrl_domain - group of CPUs sharing a resctrl control resource +* @hdr: common header for different domain types +* @cpu_mask: which CPUs share this resource +* @plr: pseudo-locked region (if any) associated with domain +* @staged_config: parsed configuration to be applied +* @mbps_val: When mba_sc is enabled, this holds the array of user +* specified control values for mba_sc in MBps, indexed +* by closid +*/ +struct rdt_ctrl_domain { +struct rdt_domain_hdr hdr; +struct cpumask cpu_mask; +struct pseudo_lock_region *plr; +struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; +u32 *mbps_val; +}; + +/** +* struct rdt_mon_domain - group of CPUs sharing a resctrl control resource * @hdr: common header for different domain types * @cpu_mask: which CPUs share this resource * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold
@@ -80,13 +98,8 @@ struct rdt_domain_hdr {* @cqm_limbo: worker to periodically read CQM h/w counters * @mbm_work_cpu: worker CPU for MBM h/w counters * @cqm_work_cpu: worker CPU for CQM h/w counters -* @plr: pseudo-locked region (if any) associated with domain -* @staged_config: parsed configuration to be applied -* @mbps_val: When mba_sc is enabled, this holds the array of user -* specified control values for mba_sc in MBps, indexed -* by closid */ -struct rdt_domain { +struct rdt_mon_domain {struct rdt_domain_hdr hdr;struct cpumask cpu_mask;unsigned long *rmid_busy_llc;
@@ -96,9 +109,6 @@ struct rdt_domain {struct delayed_work cqm_limbo;int mbm_work_cpu;int cqm_work_cpu; -struct pseudo_lock_region *plr; -struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; -u32 *mbps_val; };/**
@@ -202,7 +212,7 @@ struct rdt_resource {const char *format_str;int (*parse_ctrlval)(struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d); +struct rdt_ctrl_domain *d);struct list_head evt_list;unsigned long fflags;bool cdp_capable;
@@ -236,15 +246,15 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);* Update the ctrl_val and apply this config right now. * Must be called on one of the domain's CPUs. */ -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val); -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
@@ -260,7 +270,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_domain *d);* Return: * 0 on success, or -EIO, -EINVAL etc on error. */ -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val);/**
@@ -273,7 +283,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid);/**
@@ -285,7 +295,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d);extern unsigned int resctrl_rmid_realloc_threshold;extern unsigned int resctrl_rmid_realloc_limit;
arch/x86/kernel/cpu/resctrl/internal.h +38 -22
131 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/core.c +46 -41
246 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +16 -16
@@ -58,7 +58,7 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r) }int parse_bw (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct resctrl_staged_config *cfg;u32 closid = data->rdtgrp->closid;
@@ -137,7 +137,7 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)* resource type. */ int parse_cbm (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct rdtgroup *rdtgrp = data->rdtgrp;struct resctrl_staged_config *cfg;
@@ -206,8 +206,8 @@ static int parse_line(char *line, struct resctrl_schema *s,struct resctrl_staged_config *cfg;struct rdt_resource *r = s->res;struct rdt_parse_data data; +struct rdt_ctrl_domain *d;char *dom =NULL , *id; -struct rdt_domain *d;unsigned long dom_id; if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
@@ -267,11 +267,11 @@ static u32 get_config_index(u32 closid, enum resctrl_conf_type type) } } -static bool apply_config (struct rdt_hw_domain *hw_dom, +static bool apply_config (struct rdt_hw_ctrl_domain *hw_dom,struct resctrl_staged_config *cfg,u32 idx,cpumask_var_t cpu_mask) { -struct rdt_domain *dom = &hw_dom->d_resctrl; +struct rdt_ctrl_domain *dom = &hw_dom->d_resctrl;if (cfg->new_ctrl != hw_dom->ctrl_val[idx]) {cpumask_set_cpu (cpumask_any (&dom->cpu_mask), cpu_mask);
@@ -283,11 +283,11 @@ static bool apply_config(struct rdt_hw_domain *hw_dom,return false; } -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val) { +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);u32 idx =get_config_index (closid, t);struct msr_param msr_param;
@@ -307,11 +307,11 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,int resctrl_arch_update_domains (struct rdt_resource *r,u32 closid) {struct resctrl_staged_config *cfg; -struct rdt_hw_domain *hw_dom; +struct rdt_hw_ctrl_domain *hw_dom;struct msr_param msr_param; +struct rdt_ctrl_domain *d;enum resctrl_conf_type t;cpumask_var_t cpu_mask; -struct rdt_domain *d;u32 idx;if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))
@@ -319,7 +319,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid) msr_param.res =NULL ;list_for_each_entry (d, &r->ctrl_domains, hdr.list) { - hw_dom =resctrl_to_arch_dom (d); + hw_dom =resctrl_to_arch_ctrl_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];if (!cfg->have_new_ctrl)
@@ -449,10 +449,10 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,return ret ?: nbytes; } -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);u32 idx =get_config_index (closid, type);return hw_dom->ctrl_val[idx];
@@ -461,7 +461,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,static void show_doms (struct seq_file *s,struct resctrl_schema *schema,int closid) {struct rdt_resource *r = schema->res; -struct rdt_domain *dom; +struct rdt_ctrl_domain *dom;bool sep = false;u32 ctrl_val;
@@ -523,7 +523,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }void mon_event_read (struct rmid_read *rr,struct rdt_resource *r, -struct rdt_domain *d,struct rdtgroup *rdtgrp, +struct rdt_mon_domain *d,struct rdtgroup *rdtgrp,int evtid,int first) {/*
@@ -543,11 +543,11 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) {struct kernfs_open_file *of = m->private;struct rdt_domain_hdr *hdr; +struct rdt_mon_domain *d;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;union mon_data_bits md; -struct rdt_domain *d;struct rmid_read rr;int ret =0 ;
@@ -574,7 +574,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) goto out; } - d = container_of(hdr, struct rdt_domain, hdr); + d = container_of(hdr, struct rdt_mon_domain, hdr);mon_event_read (&rr, r, d, rdtgrp, evtid, false);
arch/x86/kernel/cpu/resctrl/monitor.c +20 -20
@@ -170,7 +170,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)return 0 ; } -static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_domain *hw_dom, +static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_mon_domain *hw_dom,u32 rmid,enum resctrl_event_id eventid) {
@@ -189,10 +189,10 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom,return NULL ; } -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct arch_mbm_state *am; am =get_arch_mbm_state (hw_dom, rmid, eventid);
@@ -208,9 +208,9 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* Assumes that hardware counters are also reset and thus that there is * no need to record initial non-zero counts. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d) +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);if (is_mbm_total_enabled ()) memset(hw_dom->arch_mbm_total, 0,
@@ -229,11 +229,11 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)return chunks >> shift; } -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);struct arch_mbm_state *am;u64 msr_val, chunks;int ret;
@@ -266,7 +266,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* decrement the count. If the busy count gets to zero on an RMID, we * free the RMID */ -void __check_limbo (struct rdt_domain *d,bool force_free) +void __check_limbo (struct rdt_mon_domain *d,bool force_free) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rmid_entry *entry;
@@ -305,7 +305,7 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } } -bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d) +bool has_busy_rmid (struct rdt_resource *r,struct rdt_mon_domain *d) {return find_first_bit (d->rmid_busy_llc, r->num_rmid) != r->num_rmid; }
@@ -334,7 +334,7 @@ int alloc_rmid(void)static void add_rmid_to_limbo (struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; -struct rdt_domain *d; +struct rdt_mon_domain *d;int cpu, err;u64 val =0 ;
@@ -383,7 +383,7 @@ void free_rmid(u32 rmid)list_add_tail (&entry->list, &rmid_free_lru); } -static struct mbm_state *get_mbm_state (struct rdt_domain *d,u32 rmid, +static struct mbm_state *get_mbm_state (struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id evtid) {switch (evtid) {
@@ -516,13 +516,13 @@ void mon_event_count(void *info)* throttle MSRs already have low percentage values. To avoid * unnecessarily restricting such rdtgroups, we also increase the bandwidth. */ -static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_domain *dom_mbm) +static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_mon_domain *dom_mbm) {u32 closid, rmid, cur_msr_val, new_msr_val;struct mbm_state *pmbm_data, *cmbm_data; +struct rdt_ctrl_domain *dom_mba;u32 cur_bw, delta_bw, user_bw;struct rdt_resource *r_mba; -struct rdt_domain *dom_mba;struct list_head *head;struct rdtgroup *entry;
@@ -600,7 +600,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) } } -static void mbm_update (struct rdt_resource *r,struct rdt_domain *d,int rmid) +static void mbm_update (struct rdt_resource *r,struct rdt_mon_domain *d,int rmid) {struct rmid_read rr;
@@ -640,13 +640,13 @@ void cqm_handle_limbo(struct work_struct *work) {unsigned long delay =msecs_to_jiffies (CQM_LIMBOCHECK_INTERVAL );int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex); r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, cqm_limbo.work); + d = container_of(work, struct rdt_mon_domain, cqm_limbo.work);__check_limbo (d, false);
@@ -656,7 +656,7 @@ void cqm_handle_limbo(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms) +void cqm_setup_limbo_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
@@ -672,9 +672,9 @@ void mbm_handle_overflow(struct work_struct *work)unsigned long delay =msecs_to_jiffies (MBM_OVERFLOW_INTERVAL );struct rdtgroup *prgrp, *crgrp;int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct list_head *head;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex);
@@ -682,7 +682,7 @@ void mbm_handle_overflow(struct work_struct *work) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, mbm_over.work); + d = container_of(work, struct rdt_mon_domain, mbm_over.work);list_for_each_entry (prgrp, &rdt_all_groups, rdtgroup_list) {mbm_update (r, d, prgrp->mon.rmid);
@@ -701,7 +701,7 @@ void mbm_handle_overflow(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms) +void mbm_setup_overflow_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -3
@@ -814,7 +814,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)* Return: true if @cbm overlaps with pseudo-locked region on @d, false * otherwise. */ -bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,unsigned long cbm) +bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_ctrl_domain *d,unsigned long cbm) {unsigned int cbm_len;unsigned long cbm_b;
@@ -841,11 +841,11 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm* if it is not possible to test due to memory allocation issue, * false otherwise. */ -bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d) +bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_ctrl_domain *d) { +struct rdt_ctrl_domain *d_i;cpumask_var_t cpu_with_psl;struct rdt_resource *r; -struct rdt_domain *d_i;bool ret = false;if (!zalloc_cpumask_var (&cpu_with_psl,GFP_KERNEL ))
arch/x86/kernel/cpu/resctrl/rdtgroup.c +31 -31
252 lines not on this page · raw message
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v9 4/8] x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
Hi Tony,
Quoted 7 lines · Tony Luck
The same
rdt_domainstructure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.Split into separate
rdt_ctrl_domainandrdt_mon_domainstructures with just the fields required for control and monitoring respectively.
Sounds like a motivation for the cpumask to form part of the common header?
Reinette
Tony Luck
maintainerPATCH v9 5/8 x86/resctrl: Add node-scope to the options for feature scope
In reply to Tony LuckCurrently supported resctrl features are all domain scoped the same as the scope of the L2 or L3 caches.
Add RESCTRL_NODE as a new option for features that are scoped at the same granularity as NUMA nodes. This is needed for Intel's Sub-NUMA Cluster (SNC) feature where monitoring features are node scoped.
---
No changes since v6 except to add Peter's review tag
include/linux/resctrl.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 2 ++ 2 files changed, 3 insertions(+)
include/linux/resctrl.h +1 -0
@@ -172,6 +172,7 @@ struct resctrl_schema;enum resctrl_scope {RESCTRL_L2_CACHE =2 ,RESCTRL_L3_CACHE =3 , +RESCTRL_NODE , };/**
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -515,6 +515,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)case RESCTRL_L2_CACHE :case RESCTRL_L3_CACHE :return get_cpu_cacheinfo_id (cpu, scope); +case RESCTRL_NODE : +return cpu_to_node (cpu);default :break ; }
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v9 6/8 x86/resctrl: Introduce snc_nodes_per_l3_cache
In reply to Tony LuckIntel Sub-NUMA Cluster (SNC) is a feature that subdivides the CPU cores and memory controllers on a socket into two or more groups. These are presented to the operating system as NUMA nodes.
This may enable some workloads to have slightly lower latency to memory as the memory controller(s) in an SNC node are electrically closer to the CPU cores on that SNC node. This cost may be offset by lower bandwidth since the memory accesses for each core can only be interleaved between the memory controllers on the same SNC node.
Resctrl monitoring on Intel system depends upon attaching RMIDs to tasks to track L3 cache occupancy and memory bandwidth. There is an MSR that controls how the RMIDs are shared between SNC nodes.
The default mode divides them numerically. E.g. when there are two SNC nodes on a socket the lower number half of the RMIDs are given to the first node, the remainder to the second node. This would be difficult to use with the Linux resctrl interface as specific RMID values assigned to resctrl groups are not visible to users.
The other mode divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
Even with this renumbering SNC mode requires several changes in resctrl behavior for correct operation.
Add a global integer "snc_nodes_per_l3_cache" that will show how many SNC nodes share each L3 cache. When this is "1", SNC mode is either not implemented, or not enabled.
A later patch will detect SNC mode and set snc_nodes_per_l3_cache to the appropriate value. For now it remains at the default "1" to indicate SNC mode is not active.
Code that needs to take action when SNC is enabled is:
1) The number of logical RMIDs per L3 cache available for use is the number of physical RMIDs divided by the number of SNC nodes.
2) Likewise the "mon_scale" value must be adjusted for the number of SNC nodes.
3) The RMID renumbering operates when using the value from the IA32_PQR_ASSOC MSR to count accesses by a task. When reading an RMID counter, code must adjust from the logical RMID used to the physical RMID value for the SNC node that it wishes to read and load the adjusted value into the IA32_QM_EVTSEL MSR.
4) The L3 cache is divided between the SNC nodes. So the value reported in the resctrl "size" file is adjusted.
5) The "-o mba_MBps" mount option must be disabled in SNC mode because the monitoring is being done per SNC node, while the bandwidth allocation is still done at the L3 cache scope. Trying to use this feedback loop might result in contradictory changes to the throttling level coming from each of the SNC node bandwidth measurements.
---
Changes since v6:
In commit comment s/redumbering/renumbering/
Move check that SNC is not enabled into supports_mba_mbps().
Add Peter's review tag.
arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/core.c | 6 ++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 16 +++++++++++++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 +++-- 4 files changed, 24 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -446,6 +446,8 @@ DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);extern struct dentry *debugfs_resctrl; +extern int snc_nodes_per_l3_cache; +enum resctrl_res_level {RDT_RESOURCE_L3 ,RDT_RESOURCE_L2 ,
arch/x86/kernel/cpu/resctrl/core.c +6 -0
@@ -48,6 +48,12 @@ int max_name_width, max_data_width;*/ bool rdt_alloc_capable; +/* +* Number of SNC nodes that share each L3 cache. Default is 1 for +* systems that do not support SNC, or have SNC disabled. +*/ +int snc_nodes_per_l3_cache =1 ; +static void mba_wrmsr_intel (struct rdt_ctrl_domain *d,struct msr_param *m,struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/monitor.c +13 -3
@@ -148,8 +148,18 @@ static inline struct rmid_entry *__rmid_entry(u32 rmid)static int __rmid_read (u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +int cpu =smp_processor_id (); +int rmid_offset =0 ;u64 msr_val; +/* +* When SNC mode is on, need to compute the offset to read the +* physical RMID counter for the node to which this CPU belongs. +*/ +if (snc_nodes_per_l3_cache >1 ) + rmid_offset = (cpu_to_node (cpu) % snc_nodes_per_l3_cache) * r->num_rmid; +/* * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured * with a valid event code for supported resource type and the bits
@@ -158,7 +168,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)* IA32_QM_CTR.Error (bit 63) and IA32_QM_CTR.Unavailable (bit 62) * are error bits. */ -wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid); +wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid + rmid_offset);rdmsrl (MSR_IA32_QM_CTR , msr_val);if (msr_val &RMID_VAL_ERROR )
@@ -783,8 +793,8 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)int ret; resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size *1024 ; - hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale; - r->num_rmid = boot_cpu_data.x86_cache_max_rmid +1 ; + hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache; + r->num_rmid = (boot_cpu_data.x86_cache_max_rmid +1 ) / snc_nodes_per_l3_cache; hw_res->mbm_width =MBM_CNTR_WIDTH_BASE ;if (mbm_offset >0 && mbm_offset <=MBM_CNTR_WIDTH_OFFSET_MAX )
arch/x86/kernel/cpu/resctrl/rdtgroup.c +3 -2
@@ -1425,7 +1425,7 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, } } -return size; +return size / snc_nodes_per_l3_cache; }/*
@@ -2298,7 +2298,8 @@ static bool supports_mba_mbps(void)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA ].r_resctrl;return (is_mbm_local_enabled () && - r->alloc_capable &&is_mba_linear ()); + r->alloc_capable &&is_mba_linear () && + snc_nodes_per_l3_cache ==1 ); }/*
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v9 6/8] x86/resctrl: Introduce snc_nodes_per_l3_cache
Hi Tony,
Quoted 11 lines · Tony Luck
Intel Sub-NUMA Cluster (SNC) is a feature that subdivides the CPU cores and memory controllers on a socket into two or more groups. These are presented to the operating system as NUMA nodes.
This may enable some workloads to have slightly lower latency to memory as the memory controller(s) in an SNC node are electrically closer to the CPU cores on that SNC node. This cost may be offset by lower bandwidth since the memory accesses for each core can only be interleaved between the memory controllers on the same SNC node.
Resctrl monitoring on Intel system depends upon attaching RMIDs to tasks
"on Intel system depends" -> "on an Intel system depends" or "on Intel systems depend" or ... ?
Quoted 12 lines
to track L3 cache occupancy and memory bandwidth. There is an MSR that controls how the RMIDs are shared between SNC nodes.
The default mode divides them numerically. E.g. when there are two SNC nodes on a socket the lower number half of the RMIDs are given to the first node, the remainder to the second node. This would be difficult to use with the Linux resctrl interface as specific RMID values assigned to resctrl groups are not visible to users.
The other mode divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
Quoted 8 lines
Even with this renumbering SNC mode requires several changes in resctrl behavior for correct operation.
Add a global integer "
snc_nodes_per_l3_cache" that will show how many SNC nodes share each L3 cache. When this is "1", SNC mode is either not implemented, or not enabled.A later patch will detect SNC mode and set
snc_nodes_per_l3_cacheto
Please remove usages of "later patch" from this series. For reference:
https://lore.kernel.org/lkml/20231009171918.GPZSQ2Frs%2Fqp129wsP@fat_crate.local/
Please check whole series. For same reason I expect "earlier patch" to need removal also.
Quoted 8 lines
the appropriate value. For now it remains at the default "1" to indicate SNC mode is not active.
Code that needs to take action when SNC is enabled is:
1) The number of logical RMIDs per L3 cache available for use is the number of physical RMIDs divided by the number of SNC nodes.
2) Likewise the "mon_scale" value must be adjusted for the number of SNC nodes.
Can this be expanded to indicate how the value needs to be adjusted?
Quoted 7 lines
3) The RMID renumbering operates when using the value from the
IA32_PQR_ASSOCMSR to count accesses by a task. When reading an RMID counter, code must adjust from the logical RMID used to the physical RMID value for the SNC node that it wishes to read and load the adjusted value into theIA32_QM_EVTSELMSR.
4) The L3 cache is divided between the SNC nodes. So the value reported in the resctrl "size" file is adjusted.
Can this be expanded to indicate how the value needs to be adjusted?
Quoted 42 lines
5) The "-o
mba_MBps" mount option must be disabled in SNC mode because the monitoring is being done per SNC node, while the bandwidth allocation is still done at the L3 cache scope. Trying to use this feedback loop might result in contradictory changes to the throttling level coming from each of the SNC node bandwidth measurements.Reviewed-by Peter Newman <[email protected]>Signed-off-by Tony Luck <[email protected]>---
Changes since v6:In commit comment s/redumbering/renumbering/
Move check that SNC is not enabled into
supports_mba_mbps().Add Peter's review tag.
arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/core.c | 6 ++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 16 +++++++++++++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 +++-- 4 files changed, 24 insertions(+), 5 deletions(-)arch/x86/kernel/cpu/resctrl/internal.h +2
-0@@ -446,6 +446,8 @@ DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);extern struct dentry *debugfs_resctrl; +extern int snc_nodes_per_l3_cache; +enum resctrl_res_level {RDT_RESOURCE_L3 ,RDT_RESOURCE_L2 ,arch/x86/kernel/cpu/resctrl/core.c +0
-0
arch/x86/kernel/cpu/resctrl/core.c int max_name_width, max_data_width; · line 55
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/core.c +5
-0@@ -48,6 +48,12 @@ int max_name_width, max_data_width;*/ bool rdt_alloc_capable; +/* +* Number of SNC nodes that share each L3 cache. Default is 1 for +* systems that do not support SNC, or have SNC disabled. +*/ +int snc_nodes_per_l3_cache =1 ;
Should this be an unsigned int?
arch/x86/kernel/cpu/resctrl/core.c int max_name_width, max_data_width; · line 59
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-0+static void mba_wrmsr_intel (struct rdt_ctrl_domain *d,struct msr_param *m,struct rdt_resource *r);
Reinette
Tony Luck
maintainerPATCH v9 7/8 x86/resctrl: Sub NUMA Cluster detection and enable
In reply to Tony LuckThere isn't a simple h/w bit that indicates whether a CPU is running in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the ratio of NUMA nodes to L3 cache instances.
When SNC mode is detected, reconfigure the RMID counters by updating the MSR_RMID_SNC_CONFIG MSR on each socket as CPUs are seen.
Clearing bit zero of the MSR divides the RMIDs and renumbers the ones on the second SNC node to start from zero. An earlier commit includes all the required changes in Linux to operate in this reconfigured mode.
---
Changes since v6:
Moved kfree(node_caches); earlier, to the earliest point where it is no longer needed.
Added Granite Rapids to list of CPU models that support SNC mode.
Added Peter's review tag
arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 92 ++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+)
arch/x86/include/asm/msr-index.h +1 -0
@@ -1109,6 +1109,7 @@#define MSR_IA32_QM_CTR 0xc8e#define MSR_IA32_PQR_ASSOC 0xc8f#define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0#define MSR_IA32_L2_CBM_BASE 0xd10#define MSR_IA32_MBA_THRTL_BASE 0xd50
arch/x86/kernel/cpu/resctrl/core.c +92 -0
125 lines not on this page · raw message
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v9 7/8] x86/resctrl: Sub NUMA Cluster detection and enable
Hi Tony,
Quoted 1 line · Tony Luck
There isn't a simple h/w bit that indicates whether a CPU is
"h/w" -> hardware
Quoted 8 lines
running in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the ratio of NUMA nodes to L3 cache instances.
When SNC mode is detected, reconfigure the RMID counters by updating the
MSR_RMID_SNC_CONFIGMSR on each socket as CPUs are seen.Clearing bit zero of the MSR divides the RMIDs and renumbers the ones on the second SNC node to start from zero. An earlier commit includes
Please drop the "earlier commit" reference.
Quoted 55 lines
all the required changes in Linux to operate in this reconfigured mode.
Reviewed-by Peter Newman <[email protected]>Signed-off-by Tony Luck <[email protected]>---
Changes since v6:Moved
kfree(node_caches); earlier, to the earliest point where it is no longer needed.Added Granite Rapids to list of CPU models that support SNC mode.
Added Peter's review tag
arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 92 ++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+)arch/x86/include/asm/msr-index.h +1
-0@@ -1109,6 +1109,7 @@#define MSR_IA32_QM_CTR 0xc8e#define MSR_IA32_PQR_ASSOC 0xc8f#define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0#define MSR_IA32_L2_CBM_BASE 0xd10#define MSR_IA32_MBA_THRTL_BASE 0xd50arch/x86/kernel/cpu/resctrl/core.c +5
-0@@ -16,11 +16,14 @@#define pr_fmt (fmt) "resctrl: " fmt +#include <linux/cpu.h> #include <linux/slab.h> #include <linux/err.h> #include <linux/cacheinfo.h> #include <linux/cpuhotplug.h> +#include <linux/mod_devicetable.h> +#include <asm/cpu_device_id.h> #include <asm/intel-family.h> #include <asm/resctrl.h> #include "internal.h" @@ -755,11 +758,42 @@ static void clear_closid_rmid(int cpu)wrmsr (MSR_IA32_PQR_ASSOC ,0 ,0 ); } +/* +* The power-on reset value of MSR_RMID_SNC_CONFIG is 0x1
arch/x86/kernel/cpu/resctrl/core.c static void clear_closid_rmid(int cpu) · line 772
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +10
-0+* which indicates that RMIDs are configured in legacy mode. +* This mode is incompatible with Linux resctrl semantics +* as RMIDs are partitioned between SNC nodes, which requires +* a user to know which RMID is allocated to a task. +* Clearing bit 0 reconfigures the RMID counters for use +* in Sub NUMA Cluster mode. This mode is better for Linux. +* The RMID space is divided between all SNC nodes with the +* RMIDs renumbered to start from zero in each node when +* couning operations from tasks. Code to read the counters +* must adjust RMID counnter numbers based on SNC node. See
counnter -> counter
Quoted 32 lines
arch/x86/kernel/cpu/resctrl/core.c +20
-0+* __rmid_read() for code that does this. +*/ +static void snc_remap_rmids (int cpu) +{ +u64 val; + +/* Only need to enable once per package. */ +if (cpumask_first (topology_core_cpumask (cpu)) != cpu) +return ; + +rdmsrl (MSR_RMID_SNC_CONFIG , val); + val &= ~BIT_ULL (0 ); +wrmsrl (MSR_RMID_SNC_CONFIG , val); +} +static int resctrl_online_cpu (unsigned int cpu) {struct rdt_resource *r;mutex_lock (&rdtgroup_mutex); + +if (snc_nodes_per_l3_cache >1 ) +snc_remap_rmids (cpu); +for_each_capable_rdt_resource (r)domain_add_cpu (cpu, r);/* The cpu is set in default rdtgroup after online. */ @@ -1014,11 +1048,69 @@ static __init bool get_rdt_resources(void)return (rdt_mon_capable || rdt_alloc_capable); } +/* CPU models that support MSR_RMID_SNC_CONFIG */
arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_resources(void) · line 1061
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +10
-0+static const struct x86_cpu_id snc_cpu_ids[]__initconst = { +X86_MATCH_INTEL_FAM6_MODEL (ICELAKE_X ,0 ), +X86_MATCH_INTEL_FAM6_MODEL (SAPPHIRERAPIDS_X ,0 ), +X86_MATCH_INTEL_FAM6_MODEL (EMERALDRAPIDS_X ,0 ), +X86_MATCH_INTEL_FAM6_MODEL (GRANITERAPIDS_X ,0 ), + {} +}; + +/* +* There isn't a simple h/w bit that indicates whether a CPU is running
h/w -> hardware
arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_resources(void) · line 1065
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/core.c +4
-0+* in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the +* ratio of NUMA nodes to L3 cache instances. +* It is not possible to accurately determine SNC state if the system is +* booted with a maxcpus=N parameter. That distorts the ratio of SNC nodes
Can the user be warned when SNC state cannot be determined accurately?
Quoted 41 lines
arch/x86/kernel/cpu/resctrl/core.c +41
-0+* to L3 caches. It will be OK if system is booted with hyperthreading +* disabled (since this doesn't affect the ratio). +*/ +static __init int snc_get_config (void ) +{ +unsigned long *node_caches; +int mem_only_nodes =0 ; +int cpu, node, ret; +int num_l3_caches; + +if (!x86_match_cpu (snc_cpu_ids)) +return 1 ; + + node_caches =bitmap_zalloc (nr_node_ids,GFP_KERNEL ); +if (!node_caches) +return 1 ; + +cpus_read_lock (); +for_each_node (node) { + cpu =cpumask_first (cpumask_of_node (node)); +if (cpu < nr_cpu_ids) +set_bit (get_cpu_cacheinfo_id (cpu,3 ), node_caches); +else + mem_only_nodes++; + } +cpus_read_unlock (); + + num_l3_caches =bitmap_weight (node_caches, nr_node_ids); +kfree (node_caches); + +if (!num_l3_caches) +return 1 ; + + ret = (nr_node_ids - mem_only_nodes) / num_l3_caches; + +if (ret >1 ) + rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl.mon_scope =RESCTRL_NODE ; + +return ret; +} +
arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_resources(void) · line 1116
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +2
-0static __init void rdt_init_res_defs_intel (void ) {struct rdt_hw_resource *hw_res;struct rdt_resource *r; + snc_nodes_per_l3_cache =snc_get_config (); +for_each_rdt_resource (r) { hw_res =resctrl_to_arch_res (r);
Reinette
Tony Luck
maintainerPATCH v9 8/8 x86/resctrl: Update documentation with Sub-NUMA cluster changes
In reply to Tony LuckWith Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
---
Changes since v6:
Added Peter's review tag
Documentation/arch/x86/resctrl.rst | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
Documentation/arch/x86/resctrl.rst +20 -3
@@ -366,9 +366,9 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. Each of these directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for
@@ -478,6 +478,23 @@ if non-contiguous 1s value is supported. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled Linux may load balance tasks between Sub-NUMA +nodes much more readily than between regular NUMA nodes since the CPUs +on Sub-NUMA nodes share the same L3 cache and the system may report +the NUMA distance between Sub-NUMA nodes with a lower value than used +for regular NUMA nodes. Users who do not bind tasks to the CPUs of a +specific Sub-NUMA node must read the "llc_occupancy", "mbm_total_bytes", +and "mbm_local_bytes" for all Sub-NUMA nodes where the tasks may execute +to get the full view of traffic for which the tasks were the source. + +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================
Signature · 1 line
2.41.0
RE: [PATCH v9 0/8] Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckHi tony,
Quoted 1 line
base-commit:
3300447612b2adbc05cbb90e5d1cb288f19c40c6
$ git checkout 3300447612b2adbc05cbb90e5d1cb288f19c40c6 -b patch_test fatal: reference is not a tree: 3300447612b2adbc05cbb90e5d1cb288f19c40c6
Then I tried apply this patch series to kernel v6.5 and v6.6-rc1~7, but it failed with error message "Patch failed at 000x".
Could you tell me what kernel version this patch series is based on?
Best regards,
Shaopeng TAN
Luck, Tony
maintainerRE: [PATCH v9 0/8] Add support for Sub-NUMA cluster (SNC) systems
Quoted 1 line
Could you tell me what kernel version this patch series is based on?
Shaopeng TAN,
Sorry. It's buried in all the other text in the change log in the cover letter. I should have put this more prominently at the start of the cover letter.
Rebased to tip/master October 20th since that has several other resctrl changes staged resdy for next merge window. No significant collisions, just noise where "git am" would not automatically apply. New base is:
3300447612b2 ("Merge branch into tip/master: 'x86/tdx'")
-Tony
Tony Luck
maintainerv10 cover Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckThe Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows monitoring features to be used. With the caveat that users must be aware that Linux may migrate tasks more frequently between SNC nodes than between "regular" NUMA nodes, so reading counters from all SNC nodes may be needed to get a complete picture of activity for tasks.
Cache and memory bandwidth allocation features continue to operate at the scope of the L3 cache.
---
Dropped Peter's "Reviewed-by" from all but parts 5 & 8 since there have been many changes since he provided those.
Other changes since v9 (all from Reinette's comments)
global s/cpu/CPU/ in commit messages and code comments
#1
New test for invalid domain id before calling rdt_find_domain() means that error handling in that function and at all call-sites can be simplified. In pseudo_lock_region_init() use the new enum resctrl_scope for local variable.
#2
Include *all* common fields in the rdt_domain_hdr. Defer adding "type" until it is used later in part #3.
#3
Fix commit to be specific the only the RDT_RESOURCE_L3 resource is going to have different monitor and control scope.
Rename get_domain_from_cpu() -> get_ctrl_domain_from_cpu() Rewrite comment for rdt_find_domains().
Add "type" field to rdt_domain_hdr structure.
Delete the /* RDT_RESOURCE_MBA is never mon_capable */ comment.
#4
Comment against patch 4, but now fixed in patch #2. cpu_mask is included in common header.
#5
No comments. No changes.
#6
Fixed missing word s/monitoring on Intel/monitoring on an Intel/ Deleted "A later patch" paragraph.
Expanded description how how values are "adjusted" for mon_scale and cache size.
Changed type of "snc_nodes_per_l3_cache" to "unsigned int".
#7
Expand h/w to hardware (commit and code comments) Remove "earlier commit" reference
s/counnter/counter/
Check for offline CPUs and warn user SNC detection may be broken.
#8
No comments. No changes.
Tony Luck (8):
x86/resctrl: Prepare for new domain scope x86/resctrl: Prepare to split rdt_domain structure x86/resctrl: Prepare for different scope for control/monitor
operations
x86/resctrl: Split the rdt_domain and rdt_hw_domain structures x86/resctrl: Add node-scope to the options for feature scope x86/resctrl: Introduce snc_nodes_per_l3_cache x86/resctrl: Sub NUMA Cluster detection and enable x86/resctrl: Update documentation with Sub-NUMA cluster changes
Documentation/arch/x86/resctrl.rst | 23 +- include/linux/resctrl.h | 87 +++-- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 66 ++-- arch/x86/kernel/cpu/resctrl/core.c | 411 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 58 +-- arch/x86/kernel/cpu/resctrl/monitor.c | 68 ++-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 26 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 149 ++++---- 9 files changed, 607 insertions(+), 282 deletions(-)
base-commit: 5a6a09e97199d6600d31383055f9d43fbbcbe86f
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v10 1/8 x86/resctrl: Prepare for new domain scope
Resctrl resources operate on subsets of CPUs in the system with the defining attribute of each subset being an instance of a particular level of cache. E.g. all CPUs sharing an L3 cache would be part of the same domain.
In preparation for features that are scoped at the NUMA node level change the code from explicit references to "cache_level" to a more generic scope. At this point the only options for this scope are groups of CPUs that share an L2 cache or L3 cache.
Clean up the error handling when looking up domains. Report invalid id's before calling rdt_find_domain() in preparation for better messages when scope can be other than cache scope. This means that rdt_find_domain() will never return an error. So remove checks for error from the callsites.
---
Changes since v9
New test for invalid domain id before calling rdt_find_domain() means that error handling in that function and at all call-sites can be simplified. In pseudo_lock_region_init() use the new enum resctrl_scope for local variable.
include/linux/resctrl.h | 9 +++-- arch/x86/kernel/cpu/resctrl/core.c | 40 +++++++++++++++-------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +++- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 ++- 5 files changed, 44 insertions(+), 18 deletions(-)
include/linux/resctrl.h +7 -2
@@ -144,13 +144,18 @@ struct resctrl_membw {struct rdt_parse_data ;struct resctrl_schema ; +enum resctrl_scope { +RESCTRL_L2_CACHE =2 , +RESCTRL_L3_CACHE =3 , +}; +/** * struct rdt_resource - attributes of a resctrl resource * @rid: The index of the resource * @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @cache_level: Which cache level defines scope of this resource +* @scope: Scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. * @domains: All domains for this resource
@@ -168,7 +173,7 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -int cache_level; +enum resctrl_scope scope;struct resctrl_cache cache;struct resctrl_membw membw;struct list_head domains;
arch/x86/kernel/cpu/resctrl/core.c +27 -13
@@ -65,7 +65,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L3, .name = "L3", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_L3), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -79,7 +79,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L2, .name = "L2", - .cache_level = 2, + .scope = RESCTRL_L2_CACHE, .domains = domain_init(RDT_RESOURCE_L2), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x",
@@ -93,7 +93,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_MBA, .name = "MB", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_MBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -105,7 +105,7 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_SMBA, .name = "SMBA", - .cache_level = 3, + .scope = RESCTRL_L3_CACHE, .domains = domain_init(RDT_RESOURCE_SMBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u",
@@ -401,9 +401,6 @@ struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,struct rdt_domain *d;struct list_head *l; -if (id <0 ) -return ERR_PTR (-ENODEV ); -list_for_each (l, &r->domains) { d = list_entry(l, struct rdt_domain, list);/* When id is found, return its domain. */
@@ -491,6 +488,19 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)return 0 ; } +static int get_domain_id_from_scope (int cpu,enum resctrl_scope scope) +{ +switch (scope) { +case RESCTRL_L2_CACHE : +case RESCTRL_L3_CACHE : +return get_cpu_cacheinfo_id (cpu, scope); +default : +break ; + } + +return -EINVAL ; +} +/* * domain_add_cpu - Add a cpu to a resource's domain list. *
@@ -506,17 +516,18 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)*/ static void domain_add_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct list_head *add_pos =NULL ;struct rdt_hw_domain *hw_dom;struct rdt_domain *d;int err; - d =rdt_find_domain (r, id, &add_pos); -if (IS_ERR (d)) { -pr_warn ("Couldn't find cache id for CPU %d\n" , cpu); +if (id <0 ) { +pr_warn_once ("Can't find domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->scope, r->name);return ; } + d =rdt_find_domain (r, id, &add_pos);if (d) {cpumask_set_cpu (cpu, &d->cpu_mask);
@@ -556,12 +567,15 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)static void domain_remove_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct rdt_hw_domain *hw_dom;struct rdt_domain *d; +if (id <0 ) +return ; + d =rdt_find_domain (r, id,NULL ); -if (IS_ERR_OR_NULL (d)) { +if (!d) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);return ; }
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +1 -1
@@ -563,7 +563,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) r = &rdt_resources_all[resid].r_resctrl; d =rdt_find_domain (r, domid,NULL ); -if (IS_ERR_OR_NULL (d)) { +if (!d) { ret = -ENOENT ; goto out; }
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +5 -1
@@ -292,10 +292,14 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { +enum resctrl_scope scope = plr->s->res->scope;struct cpu_cacheinfo *ci;int ret;int i; +if (WARN_ON_ONCE (scope !=RESCTRL_L2_CACHE && scope !=RESCTRL_L3_CACHE )) +return -ENODEV ; +/* Pick the first cpu we find that is associated with the cache. */ plr->cpu =cpumask_first (&plr->d->cpu_mask);
@@ -311,7 +315,7 @@ static int pseudo_lock_region_init(struct pseudo_lock_region *plr) plr->size =rdtgroup_cbm_to_size (plr->s->res, plr->d, plr->cbm);for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == plr->s->res->cache_level) { +if (ci->info_list[i].level == scope) { plr->line_size = ci->info_list[i].coherency_line_size;return 0 ; }
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -1
@@ -1413,10 +1413,13 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,unsigned int size =0 ;int num_b, i; +if (WARN_ON_ONCE (r->scope !=RESCTRL_L2_CACHE && r->scope !=RESCTRL_L3_CACHE )) +return size; + num_b =bitmap_weight (&cbm, r->cache.cbm_len); ci =get_cpu_cacheinfo (cpumask_any (&d->cpu_mask));for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == r->cache_level) { +if (ci->info_list[i].level == r->scope) { size = ci->info_list[i].size / r->cache.cbm_len * num_b;break ; }
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v10 1/8] x86/resctrl: Prepare for new domain scope
Hi Tony,
Quoted 15 lines · Tony Luck
Resctrl resources operate on subsets of CPUs in the system with the defining attribute of each subset being an instance of a particular level of cache. E.g. all CPUs sharing an L3 cache would be part of the same domain.
In preparation for features that are scoped at the NUMA node level change the code from explicit references to "
cache_level" to a more generic scope. At this point the only options for this scope are groups of CPUs that share an L2 cache or L3 cache.Clean up the error handling when looking up domains. Report invalid id's before calling
rdt_find_domain()in preparation for better messages when scope can be other than cache scope. This means thatrdt_find_domain()will never return an error. So remove checks for error from the callsites.
Quoted 5 lines
Signed-off-by Tony Luck <[email protected]>---
Changes since v9
New test for invalid domain id before callingrdt_find_domain()means that error handling in that function and at all call-sites can be simplified.
These changes do not appear to be consistent in this series. Simplifying the call-sites is indeed done in this patch but this work seems to be undone in patch 3 where it reverts back to the previous error handling in domain_add_cpu_mon(), domain_remove_cpu_ctrl(), and domain_remove_cpu_mon().
Quoted 9 lines
In
pseudo_lock_region_init()use the new enumresctrl_scopefor local variable.include/linux/resctrl.h | 9 +++-- arch/x86/kernel/cpu/resctrl/core.c | 40 +++++++++++++++-------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +++- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 ++- 5 files changed, 44 insertions(+), 18 deletions(-)
...
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-1@@ -506,17 +516,18 @@ static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom)*/ static void domain_add_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);struct list_head *add_pos =NULL ;struct rdt_hw_domain *hw_dom;struct rdt_domain *d;
arch/x86/kernel/cpu/resctrl/core.c static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom) · line 529
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +3
-3int err; - d =rdt_find_domain (r, id, &add_pos); -if (IS_ERR (d)) { -pr_warn ("Couldn't find cache id for CPU %d\n" , cpu); +if (id <0 ) { +pr_warn_once ("Can't find domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->scope, r->name);return ; }
Please add empty line here ...
arch/x86/kernel/cpu/resctrl/core.c static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_domain *hw_dom) · line 531
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-0+ d =rdt_find_domain (r, id, &add_pos);
... and remove this empty line.
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-1if (d) {cpumask_set_cpu (cpu, &d->cpu_mask);@@ -556,12 +567,15 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)static void domain_remove_cpu (int cpu,struct rdt_resource *r) { -int id =get_cpu_cacheinfo_id (cpu, r->cache_level); +int id =get_domain_id_from_scope (cpu, r->scope);
arch/x86/kernel/cpu/resctrl/core.c static void domain_add_cpu(int cpu, struct rdt_resource *r) · line 579
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +4
-1struct rdt_hw_domain *hw_dom;struct rdt_domain *d; +if (id <0 ) +return ; + d =rdt_find_domain (r, id,NULL ); -if (IS_ERR_OR_NULL (d)) { +if (!d) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
This error message is no longer accurate.
Review static void domain_add_cpu(int cpu, struct rdt_resource *r) · line 581
Quoted 2 lines
(quoted diff) +0
-0return; }
Reinette
Tony Luck
maintainerPATCH v10 2/8 x86/resctrl: Prepare to split rdt_domain structure
In reply to Tony LuckThe rdt_domain structure is used for both control and monitor features. It is about to be split into separate structures for these two usages because the scope for control and monitoring features for a resource will be different for future resources.
To allow for common code that scans a list of domains looking for a specific domain id, move all the common fields ("list", "id", "cpu_mask") into their own structure within the rdt_domain structure.
---
Changes since v9
Include *all* common fields in the rdt_domain_hdr. Defer adding "type" until it is used later in part #3.
include/linux/resctrl.h | 16 ++++-- arch/x86/kernel/cpu/resctrl/core.c | 26 +++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 22 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 10 ++-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 14 +++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 60 +++++++++++------------ 6 files changed, 78 insertions(+), 70 deletions(-)
include/linux/resctrl.h +12 -4
@@ -53,10 +53,20 @@ struct resctrl_staged_config { };/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_domain_hdr - common header for different domain types * @list: all instances of this resource * @id: unique id for this instance * @cpu_mask: which CPUs share this resource +*/ +struct rdt_domain_hdr { +struct list_head list; +int id; +struct cpumask cpu_mask; +}; + +/** +* struct rdt_domain - group of CPUs sharing a resctrl resource +* @hdr: common header for different domain types * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold * @mbm_total: saved state for MBM total bandwidth * @mbm_local: saved state for MBM local bandwidth
@@ -71,9 +81,7 @@ struct resctrl_staged_config {* by closid */ struct rdt_domain { -struct list_head list; -int id; -struct cpumask cpu_mask; +struct rdt_domain_hdr hdr;unsigned long *rmid_busy_llc;struct mbm_state *mbm_total;struct mbm_state *mbm_local;
arch/x86/kernel/cpu/resctrl/core.c +13 -13
@@ -356,9 +356,9 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) {/* Find the domain that contains this CPU */ -if (cpumask_test_cpu (cpu, &d->cpu_mask)) +if (cpumask_test_cpu (cpu, &d->hdr.cpu_mask))return d; }
@@ -402,12 +402,12 @@ struct rdt_domain *rdt_find_domain(struct rdt_resource *r, int id,struct list_head *l;list_for_each (l, &r->domains) { - d = list_entry(l, struct rdt_domain, list); + d = list_entry(l, struct rdt_domain, hdr.list);/* When id is found, return its domain. */ -if (id == d->id) +if (id == d->hdr.id)return d;/* Stop searching when finding id's position in sorted list. */ -if (id < d->id) +if (id < d->hdr.id)break ; }
@@ -530,7 +530,7 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r) d =rdt_find_domain (r, id, &add_pos);if (d) { -cpumask_set_cpu (cpu, &d->cpu_mask); +cpumask_set_cpu (cpu, &d->hdr.cpu_mask);if (r->cache.arch_has_per_cpu_cfg)rdt_domain_reconfigure_cdp (r);return ;
@@ -541,8 +541,8 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; d = &hw_dom->d_resctrl; - d->id = id; -cpumask_set_cpu (cpu, &d->cpu_mask); + d->hdr.id = id; +cpumask_set_cpu (cpu, &d->hdr.cpu_mask);rdt_domain_reconfigure_cdp (r);
@@ -556,11 +556,11 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; } -list_add_tail (&d->list, add_pos); +list_add_tail (&d->hdr.list, add_pos); err =resctrl_online_domain (r, d);if (err) { -list_del (&d->list); +list_del (&d->hdr.list);domain_free (hw_dom); } }
@@ -581,10 +581,10 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) } hw_dom =resctrl_to_arch_dom (d); -cpumask_clear_cpu (cpu, &d->cpu_mask); -if (cpumask_empty (&d->cpu_mask)) { +cpumask_clear_cpu (cpu, &d->hdr.cpu_mask); +if (cpumask_empty (&d->hdr.cpu_mask)) {resctrl_offline_domain (r, d); -list_del (&d->list); +list_del (&d->hdr.list);/* * rdt_domain "d" is going to be freed below, so clear
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +11 -11
@@ -67,7 +67,7 @@ int parse_bw(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -146,7 +146,7 @@ int parse_cbm(struct rdt_parse_data *data, struct resctrl_schema *s, cfg = &d->staged_config[s->conf_type];if (cfg->have_new_ctrl) { -rdt_last_cmd_printf ("Duplicate domain %d\n" , d->id); +rdt_last_cmd_printf ("Duplicate domain %d\n" , d->hdr.id);return -EINVAL ; }
@@ -226,8 +226,8 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, list) { -if (d->id == dom_id) { +list_for_each_entry (d, &r->domains, hdr.list) { +if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;if (r->parse_ctrlval (&data, s, d))
@@ -274,7 +274,7 @@ static bool apply_config(struct rdt_hw_domain *hw_dom,struct rdt_domain *dom = &hw_dom->d_resctrl;if (cfg->new_ctrl != hw_dom->ctrl_val[idx]) { -cpumask_set_cpu (cpumask_any (&dom->cpu_mask), cpu_mask); +cpumask_set_cpu (cpumask_any (&dom->hdr.cpu_mask), cpu_mask); hw_dom->ctrl_val[idx] = cfg->new_ctrl;return true;
@@ -291,7 +291,7 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,u32 idx =get_config_index (closid, t);struct msr_param msr_param; -if (!cpumask_test_cpu (smp_processor_id (), &d->cpu_mask)) +if (!cpumask_test_cpu (smp_processor_id (), &d->hdr.cpu_mask))return -EINVAL ; hw_dom->ctrl_val[idx] = cfg_val;
@@ -318,7 +318,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, list) { +list_for_each_entry (d, &r->domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -466,7 +466,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, list) { +list_for_each_entry (dom, &r->domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -476,7 +476,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clo ctrl_val =resctrl_arch_get_config (r, dom, closid, schema->conf_type); -seq_printf (s, r->format_str, dom->id, max_data_width, +seq_printf (s, r->format_str, dom->hdr.id, max_data_width, ctrl_val); sep = true; }
@@ -505,7 +505,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }else {seq_printf (s,"%s:%d=%x\n" , rdtgrp->plr->s->res->name, - rdtgrp->plr->d->id, + rdtgrp->plr->d->hdr.id, rdtgrp->plr->cbm); } }else {
@@ -536,7 +536,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->val =0 ; rr->first = first; -smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); +smp_call_function_any (&d->hdr.cpu_mask, mon_event_count, rr,1 ); }int rdtgroup_mondata_show (struct seq_file *m,void *arg)
arch/x86/kernel/cpu/resctrl/monitor.c +5 -5
@@ -238,7 +238,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,u64 msr_val, chunks;int ret; -if (!cpumask_test_cpu (smp_processor_id (), &d->cpu_mask)) +if (!cpumask_test_cpu (smp_processor_id (), &d->hdr.cpu_mask))return -EINVAL ; ret =__rmid_read (rmid, eventid, &msr_val);
@@ -340,8 +340,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, list) { -if (cpumask_test_cpu (cpu, &d->cpu_mask)) { +list_for_each_entry (d, &r->domains, hdr.list) { +if (cpumask_test_cpu (cpu, &d->hdr.cpu_mask)) { err =resctrl_arch_rmid_read (r, d, entry->rmid,QOS_L3_OCCUP_EVENT_ID , &val);
@@ -661,7 +661,7 @@ void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms)unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; - cpu =cpumask_any (&dom->cpu_mask); + cpu =cpumask_any (&dom->hdr.cpu_mask); dom->cqm_work_cpu = cpu;schedule_delayed_work_on (cpu, &dom->cqm_limbo, delay);
@@ -708,7 +708,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)if (!static_branch_likely (&rdt_mon_enable_key))return ; - cpu =cpumask_any (&dom->cpu_mask); + cpu =cpumask_any (&dom->hdr.cpu_mask); dom->mbm_work_cpu = cpu;schedule_delayed_work_on (cpu, &dom->mbm_over, delay); }
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +7 -7
@@ -221,7 +221,7 @@ static int pseudo_lock_cstates_constrain(struct pseudo_lock_region *plr)int cpu;int ret; -for_each_cpu (cpu, &plr->d->cpu_mask) { +for_each_cpu (cpu, &plr->d->hdr.cpu_mask) { pm_req =kzalloc (sizeof (*pm_req),GFP_KERNEL );if (!pm_req) {rdt_last_cmd_puts ("Failure to allocate memory for PM QoS\n" );
@@ -301,7 +301,7 @@ static int pseudo_lock_region_init(struct pseudo_lock_region *plr)return -ENODEV ;/* Pick the first cpu we find that is associated with the cache. */ - plr->cpu =cpumask_first (&plr->d->cpu_mask); + plr->cpu =cpumask_first (&plr->d->hdr.cpu_mask);if (!cpu_online (plr->cpu)) { rdt_last_cmd_printf("CPU %u associated with cache not online\n",
@@ -856,10 +856,10 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, list) { +list_for_each_entry (d_i, &r->domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, - &d_i->cpu_mask); + &d_i->hdr.cpu_mask); } }
@@ -867,7 +867,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* Next test if new pseudo-locked region would intersect with * existing region. */ -if (cpumask_intersects (&d->cpu_mask, cpu_with_psl)) +if (cpumask_intersects (&d->hdr.cpu_mask, cpu_with_psl)) ret = true;free_cpumask_var (cpu_with_psl);
@@ -1199,7 +1199,7 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp, int sel) } plr->thread_done =0 ; - cpu =cpumask_first (&plr->d->cpu_mask); + cpu =cpumask_first (&plr->d->hdr.cpu_mask);if (!cpu_online (cpu)) { ret = -ENODEV ; goto out;
@@ -1529,7 +1529,7 @@ static int pseudo_lock_dev_mmap(struct file *filp, struct vm_area_struct *vma)* may be scheduled elsewhere and invalidate entries in the * pseudo-locked region. */ -if (!cpumask_subset (current->cpus_ptr, &plr->d->cpu_mask)) { +if (!cpumask_subset (current->cpus_ptr, &plr->d->hdr.cpu_mask)) {mutex_unlock (&rdtgroup_mutex);return -EINVAL ; }
arch/x86/kernel/cpu/resctrl/rdtgroup.c +30 -30
220 lines not on this page · raw message
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v10 3/8 x86/resctrl: Prepare for different scope for control/monitor operations
In reply to Tony LuckResctrl assumes that control and monitor operations on a resource are performed at the same scope.
Prepare for systems that use different scope (specifically Intel needs to split the RDT_RESOURCE_L3 resource to use L3 scope for cache control and NODE scope for cache occupancy and memory bandwidth monitoring).
Create separate domain lists for control and monitor operations.
Note that errors during initialization of either control or monitor functions on a domain would previously result in that domain being excluded from both control and monitor operations. Now the domains are allocated independently it is no longer required to disable both control and monitor operations if either fail.
---
Changes since v9
Fix commit to be specific the only the RDT_RESOURCE_L3 resource is going to have different monitor and control scope.
Rename get_domain_from_cpu() -> get_ctrl_domain_from_cpu() Rewrite comment for rdt_find_domains().
Add "type" field to rdt_domain_hdr structure.
Delete the /* RDT_RESOURCE_MBA is never mon_capable */ comment.
include/linux/resctrl.h | 25 ++- arch/x86/kernel/cpu/resctrl/internal.h | 6 +- arch/x86/kernel/cpu/resctrl/core.c | 206 ++++++++++++++++------ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 12 +- arch/x86/kernel/cpu/resctrl/monitor.c | 4 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 55 +++--- 7 files changed, 218 insertions(+), 94 deletions(-)
include/linux/resctrl.h +19 -6
@@ -52,15 +52,22 @@ struct resctrl_staged_config {bool have_new_ctrl; }; +enum resctrl_domain_type { +RESCTRL_CTRL_DOMAIN , +RESCTRL_MON_DOMAIN , +}; +/** * struct rdt_domain_hdr - common header for different domain types * @list: all instances of this resource * @id: unique id for this instance +* @type: type of this instance * @cpu_mask: which CPUs share this resource */ struct rdt_domain_hdr {struct list_head list;int id; +enum resctrl_domain_type type;struct cpumask cpu_mask; };
@@ -163,10 +170,12 @@ enum resctrl_scope {* @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @scope: Scope of this resource +* @ctrl_scope: Scope of this resource for control functions +* @mon_scope: Scope of this resource for monitor functions * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. -* @domains: All domains for this resource +* @ctrl_domains: Control domains for this resource +* @mon_domains: Monitor domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.
@@ -181,10 +190,12 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -enum resctrl_scope scope; +enum resctrl_scope ctrl_scope; +enum resctrl_scope mon_scope;struct resctrl_cache cache;struct resctrl_membw membw; -struct list_head domains; +struct list_head ctrl_domains; +struct list_head mon_domains;char *name;int data_width;u32 default_ctrl;
@@ -230,8 +241,10 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
arch/x86/kernel/cpu/resctrl/internal.h +3 -3
@@ -520,8 +520,8 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn);int rdtgroup_kn_mode_restrict (struct rdtgroup *r,const char *name);int rdtgroup_kn_mode_restore (struct rdtgroup *r,const char *name,umode_t mask); -struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id, -struct list_head **pos); +struct rdt_domain_hdr *rdt_find_domain (struct list_head *h,int id, +struct list_head **pos);ssize_t rdtgroup_schemata_write (struct kernfs_open_file *of,char *buf,size_t nbytes,loff_t off); int rdtgroup_schemata_show(struct kernfs_open_file *of,
@@ -540,7 +540,7 @@ int rdt_pseudo_lock_init(void);void rdt_pseudo_lock_release (void );int rdtgroup_pseudo_lock_create (struct rdtgroup *rdtgrp);void rdtgroup_pseudo_lock_remove (struct rdtgroup *rdtgrp); -struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r); +struct rdt_domain *get_ctrl_domain_from_cpu (int cpu,struct rdt_resource *r);int closids_supported (void );void closid_free (int closid);int alloc_rmid (void );
arch/x86/kernel/cpu/resctrl/core.c +155 -51
333 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +7 -5
@@ -226,7 +226,7 @@ static int parse_line(char *line, struct resctrl_schema *s,return -EINVAL ; } dom =strim (dom); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) {if (d->hdr.id == dom_id) { data.buf = dom; data.rdtgrp = rdtgrp;
@@ -318,7 +318,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)return -ENOMEM ; msr_param.res =NULL ; -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) { hw_dom =resctrl_to_arch_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];
@@ -466,7 +466,7 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clou32 ctrl_val;seq_printf (s,"%*s:" , max_name_width, schema->name); -list_for_each_entry (dom, &r->domains, hdr.list) { +list_for_each_entry (dom, &r->ctrl_domains, hdr.list) {if (sep)seq_puts (s,";" );
@@ -542,6 +542,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,int rdtgroup_mondata_show (struct seq_file *m,void *arg) {struct kernfs_open_file *of = m->private; +struct rdt_domain_hdr *hdr;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;
@@ -562,11 +563,12 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) evtid = md.u.evtid; r = &rdt_resources_all[resid].r_resctrl; - d =rdt_find_domain (r, domid,NULL ); -if (!d) { + hdr =rdt_find_domain (&r->mon_domains, domid,NULL ); +if (!hdr ||WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) { ret = -ENOENT ; goto out; } + d = container_of(hdr, struct rdt_domain, hdr);mon_event_read (&rr, r, d, rdtgrp, evtid, false);
arch/x86/kernel/cpu/resctrl/monitor.c +2 -2
@@ -340,7 +340,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) entry->busy =0 ; cpu =get_cpu (); -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->mon_domains, hdr.list) {if (cpumask_test_cpu (cpu, &d->hdr.cpu_mask)) { err = resctrl_arch_rmid_read(r, d, entry->rmid, QOS_L3_OCCUP_EVENT_ID,
@@ -535,7 +535,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) rmid = rgrp->mon.rmid; pmbm_data = &dom_mbm->mbm_local[rmid]; - dom_mba =get_domain_from_cpu (smp_processor_id (), r_mba); + dom_mba =get_ctrl_domain_from_cpu (smp_processor_id (), r_mba);if (!dom_mba) {pr_warn_once ("Failure to get domain for MBA update\n" );return ;
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +2 -2
@@ -292,7 +292,7 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { -enum resctrl_scope scope = plr->s->res->scope; +enum resctrl_scope scope = plr->s->res->ctrl_scope;struct cpu_cacheinfo *ci;int ret;int i;
@@ -856,7 +856,7 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)* associated with them. */ for_each_alloc_capable_rdt_resource (r) { -list_for_each_entry (d_i, &r->domains, hdr.list) { +list_for_each_entry (d_i, &r->ctrl_domains, hdr.list) {if (d_i->plr)cpumask_or (cpu_with_psl, cpu_with_psl, &d_i->hdr.cpu_mask);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +30 -25
170 lines not on this page · raw message
Signature · 1 line
2.41.0
Luck, Tony
maintainerRE: [PATCH v10 3/8] x86/resctrl: Prepare for different scope for control/monitor operations
+/*
+ * domain_add_cpu - Add a cpu to either/both resource's domain lists.
+ */
+static void domain_add_cpu(int cpu, struct rdt_resource *r)
Bother. Missed one comment that needs s/cpu/CPU/
-Tony
Reinette Chatre
maintainerRe: [PATCH v10 3/8] x86/resctrl: Prepare for different scope for control/monitor operations
In reply to Tony LuckHi Tony,
Quoted 212 lines · Tony Luck
Resctrl assumes that control and monitor operations on a resource are performed at the same scope.
Prepare for systems that use different scope (specifically Intel needs to split the
RDT_RESOURCE_L3resource to use L3 scope for cache control and NODE scope for cache occupancy and memory bandwidth monitoring).Create separate domain lists for control and monitor operations.
Note that errors during initialization of either control or monitor functions on a domain would previously result in that domain being excluded from both control and monitor operations. Now the domains are allocated independently it is no longer required to disable both control and monitor operations if either fail.
Signed-off-by Tony Luck <[email protected]>---
Changes since v9
Fix commit to be specific the only theRDT_RESOURCE_L3resource is going to have different monitor and control scope.
Renameget_domain_from_cpu()->get_ctrl_domain_from_cpu()Rewrite comment forrdt_find_domains().
Add "type" field tordt_domain_hdrstructure.
Delete the /*RDT_RESOURCE_MBAis nevermon_capable*/ comment.include/linux/resctrl.h | 25 ++- arch/x86/kernel/cpu/resctrl/internal.h | 6 +- arch/x86/kernel/cpu/resctrl/core.c | 206 ++++++++++++++++------ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 12 +- arch/x86/kernel/cpu/resctrl/monitor.c | 4 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 55 +++--- 7 files changed, 218 insertions(+), 94 deletions(-)include/linux/resctrl.h +19
-6@@ -52,15 +52,22 @@ struct resctrl_staged_config {bool have_new_ctrl; }; +enum resctrl_domain_type { +RESCTRL_CTRL_DOMAIN , +RESCTRL_MON_DOMAIN , +}; +/** * struct rdt_domain_hdr - common header for different domain types * @list: all instances of this resource * @id: unique id for this instance +* @type: type of this instance * @cpu_mask: which CPUs share this resource */ struct rdt_domain_hdr {struct list_head list;int id; +enum resctrl_domain_type type;struct cpumask cpu_mask; };@@ -163,10 +170,12 @@ enum resctrl_scope {* @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @scope: Scope of this resource +* @ctrl_scope: Scope of this resource for control functions +* @mon_scope: Scope of this resource for monitor functions * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. -* @domains: All domains for this resource +* @ctrl_domains: Control domains for this resource +* @mon_domains: Monitor domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent. @@ -181,10 +190,12 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -enum resctrl_scope scope; +enum resctrl_scope ctrl_scope; +enum resctrl_scope mon_scope;struct resctrl_cache cache;struct resctrl_membw membw; -struct list_head domains; +struct list_head ctrl_domains; +struct list_head mon_domains;char *name;int data_width;u32 default_ctrl;@@ -230,8 +241,10 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid arch/x86/kernel/cpu/resctrl/internal.h +3
-3@@ -520,8 +520,8 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn);int rdtgroup_kn_mode_restrict (struct rdtgroup *r,const char *name);int rdtgroup_kn_mode_restore (struct rdtgroup *r,const char *name,umode_t mask); -struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id, -struct list_head **pos); +struct rdt_domain_hdr *rdt_find_domain (struct list_head *h,int id, +struct list_head **pos);ssize_t rdtgroup_schemata_write (struct kernfs_open_file *of,char *buf,size_t nbytes,loff_t off); int rdtgroup_schemata_show(struct kernfs_open_file *of,@@ -540,7 +540,7 @@ int rdt_pseudo_lock_init(void);void rdt_pseudo_lock_release (void );int rdtgroup_pseudo_lock_create (struct rdtgroup *rdtgrp);void rdtgroup_pseudo_lock_remove (struct rdtgroup *rdtgrp); -struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r); +struct rdt_domain *get_ctrl_domain_from_cpu (int cpu,struct rdt_resource *r);int closids_supported (void );void closid_free (int closid);int alloc_rmid (void );arch/x86/kernel/cpu/resctrl/core.c +15
-12@@ -57,7 +57,8 @@ static void mba_wrmsr_amd(struct rdt_domain *d, struct msr_param *m, struct rdt_resource *r); -#define domain_init (id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.domains) +#define ctrl_domain_init (id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.ctrl_domains) +#define mon_domain_init (id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.mon_domains)struct rdt_hw_resource rdt_resources_all[] = { [RDT_RESOURCE_L3] =@@ -65,8 +66,10 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L3, .name = "L3", - .scope = RESCTRL_L3_CACHE, - .domains = domain_init(RDT_RESOURCE_L3), + .ctrl_scope = RESCTRL_L3_CACHE, + .mon_scope = RESCTRL_L3_CACHE, + .ctrl_domains = ctrl_domain_init(RDT_RESOURCE_L3), + .mon_domains = mon_domain_init(RDT_RESOURCE_L3), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x", .fflags = RFTYPE_RES_CACHE,@@ -79,8 +82,8 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_L2, .name = "L2", - .scope = RESCTRL_L2_CACHE, - .domains = domain_init(RDT_RESOURCE_L2), + .ctrl_scope = RESCTRL_L2_CACHE, + .ctrl_domains = ctrl_domain_init(RDT_RESOURCE_L2), .parse_ctrlval = parse_cbm, .format_str = "%d=%0*x", .fflags = RFTYPE_RES_CACHE,@@ -93,8 +96,8 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_MBA, .name = "MB", - .scope = RESCTRL_L3_CACHE, - .domains = domain_init(RDT_RESOURCE_MBA), + .ctrl_scope = RESCTRL_L3_CACHE, + .ctrl_domains = ctrl_domain_init(RDT_RESOURCE_MBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u", .fflags = RFTYPE_RES_MB,@@ -105,8 +108,8 @@ struct rdt_hw_resource rdt_resources_all[] = { .r_resctrl = { .rid = RDT_RESOURCE_SMBA, .name = "SMBA", - .scope = RESCTRL_L3_CACHE, - .domains = domain_init(RDT_RESOURCE_SMBA), + .ctrl_scope = RESCTRL_L3_CACHE, + .ctrl_domains = ctrl_domain_init(RDT_RESOURCE_SMBA), .parse_ctrlval = parse_bw, .format_str = "%d=%*u", .fflags = RFTYPE_RES_MB,@@ -352,11 +355,11 @@ cat_wrmsr(struct rdt_domain *d, struct msr_param *m, struct rdt_resource *r)wrmsrl (hw_res->msr_base + i, hw_dom->ctrl_val[i]); } -struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r) +struct rdt_domain *get_ctrl_domain_from_cpu (int cpu,struct rdt_resource *r) {struct rdt_domain *d; -list_for_each_entry (d, &r->domains, hdr.list) { +list_for_each_entry (d, &r->ctrl_domains, hdr.list) {/* Find the domain that contains this CPU */ if (cpumask_test_cpu (cpu, &d->hdr.cpu_mask))return d;@@ -378,7 +381,7 @@ void rdt_ctrl_update(void *arg)int cpu =smp_processor_id ();struct rdt_domain *d; - d =get_domain_from_cpu (cpu, r); + d =get_ctrl_domain_from_cpu (cpu, r);if (d) { hw_res->msr_update (d, m, r);return ;
arch/x86/kernel/cpu/resctrl/core.c void rdt_ctrl_update(void *arg) · line 396
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/core.c +2
-3@@ -388,26 +391,26 @@ void rdt_ctrl_update(void *arg) }/* -* rdt_find_domain - Find a domain in a resource that matches input resource id +* rdt_find_domain - Search for a domain id in a resource domain list. *> - * Search resource r's domain list to find the resource id. If the resource -* id is found in a domain, return the domain. Otherwise, if requested by -* caller, return the first domain whose id is bigger than the input id. +* Search the list to find the resource id. If the domain id is found
This still talks about searching for a "resource id". And the "otherwise" is not accurate ... it returns NULL if domain id cannot be found. The "otherwise" refers to a value returned in a parameter, not the function return value.
How about:
Search the domain list to find the domain id. If the domain id is found, return the domain. NULL otherwise.
If the domain id is not found (and NULL returned) then the first domain with id bigger than the input id can be returned to the caller via @pos.
Please feel free to improve.
Quoted 57 lines
arch/x86/kernel/cpu/resctrl/core.c +12
-23+* in a domain, return the domain. Otherwise, if requested by caller, +* return the first domain whose id is bigger than the input id. * The domain list is sorted by id in ascending order. */ -struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id, -struct list_head **pos) +struct rdt_domain_hdr *rdt_find_domain (struct list_head *h,int id, +struct list_head **pos) { -struct rdt_domain *d; +struct rdt_domain_hdr *d;struct list_head *l; -list_for_each (l, &r->domains) { - d = list_entry(l, struct rdt_domain, hdr.list); +list_for_each (l, h) { + d = list_entry(l, struct rdt_domain_hdr, list);/* When id is found, return its domain. */ -if (id == d->hdr.id) +if (id == d->id)return d;/* Stop searching when finding id's position in sorted list. */ -if (id < d->hdr.id) +if (id < d->id)break ; }@@ -501,35 +504,29 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)return -EINVAL ; } -/* -* domain_add_cpu - Add a cpu to a resource's domain list. -* -* If an existing domain in the resource r's domain list matches the cpu's -* resource id, add the cpu in the domain. -* -* Otherwise, a new domain is allocated and inserted into the right position -* in the domain list sorted by id in ascending order. -* -* The order in the domain list is visible to users when we print entries -* in the schemata file and schemata input is validated to have the same order -* as this list. -*/ -static void domain_add_cpu (int cpu,struct rdt_resource *r) +static void domain_add_cpu_ctrl (int cpu,struct rdt_resource *r) { -int id =get_domain_id_from_scope (cpu, r->scope); +int id =get_domain_id_from_scope (cpu, r->ctrl_scope);struct list_head *add_pos =NULL ;struct rdt_hw_domain *hw_dom; +struct rdt_domain_hdr *hdr;struct rdt_domain *d;int err;if (id <0 ) { - pr_warn_once("Can't find domain id for CPU:%d scope:%d for resource %s\n",
arch/x86/kernel/cpu/resctrl/core.c static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope) · line 523
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +4
-3- cpu, r->scope, r->name); +pr_warn_once ("Can't find control domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->ctrl_scope, r->name);return ; } - d =rdt_find_domain (r, id, &add_pos); -if (d) { + hdr =rdt_find_domain (&r->ctrl_domains, id, &add_pos); +
Please remove empty line.
Quoted 41 lines
arch/x86/kernel/cpu/resctrl/core.c +24
-2+if (hdr) { +if (WARN_ON_ONCE (hdr->type !=RESCTRL_CTRL_DOMAIN )) +return ; + + d = container_of(hdr, struct rdt_domain, hdr); +cpumask_set_cpu (cpu, &d->hdr.cpu_mask);if (r->cache.arch_has_per_cpu_cfg)rdt_domain_reconfigure_cdp (r);@@ -542,48 +539,115 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r) d = &hw_dom->d_resctrl; d->hdr.id = id; + d->hdr.type =RESCTRL_CTRL_DOMAIN ;cpumask_set_cpu (cpu, &d->hdr.cpu_mask);rdt_domain_reconfigure_cdp (r); -if (r->alloc_capable &&domain_setup_ctrlval (r, d)) { +if (domain_setup_ctrlval (r, d)) {domain_free (hw_dom);return ; } -if (r->mon_capable &&arch_domain_mbm_alloc (r->num_rmid, hw_dom)) { +list_add_tail (&d->hdr.list, add_pos); + + err =resctrl_online_ctrl_domain (r, d); +if (err) { +list_del (&d->hdr.list); +domain_free (hw_dom); + } +} + +static void domain_add_cpu_mon (int cpu,struct rdt_resource *r) +{ +int id =get_domain_id_from_scope (cpu, r->mon_scope); +struct list_head *add_pos =NULL ; +struct rdt_hw_domain *hw_dom; +struct rdt_domain_hdr *hdr; +struct rdt_domain *d;
arch/x86/kernel/cpu/resctrl/core.c static void domain_add_cpu(int cpu, struct rdt_resource *r) · line 577
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +10
-0+int err; + +if (id <0 ) { +pr_warn_once ("Can't find monitor domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->mon_scope, r->name); +return ; + } + + hdr =rdt_find_domain (&r->mon_domains, id, &add_pos); +if (IS_ERR (hdr)) {
Can this happen with rdt_find_domain() no longer returning an error?
It does not seem as though changes were made consistently to this series.
Quoted 57 lines
arch/x86/kernel/cpu/resctrl/core.c +39
-3+pr_warn ("Couldn't find monitor scope id=%d for CPU %d\n" , id, cpu); +return ; + } + +if (hdr) { +if (WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) +return ; + + d = container_of(hdr, struct rdt_domain, hdr); + +cpumask_set_cpu (cpu, &d->hdr.cpu_mask); +return ; + } + + hw_dom =kzalloc_node (sizeof (*hw_dom),GFP_KERNEL ,cpu_to_node (cpu)); +if (!hw_dom) +return ; + + d = &hw_dom->d_resctrl; + d->hdr.id = id; + d->hdr.type =RESCTRL_MON_DOMAIN ; +cpumask_set_cpu (cpu, &d->hdr.cpu_mask); + +if (arch_domain_mbm_alloc (r->num_rmid, hw_dom)) {domain_free (hw_dom);return ; }list_add_tail (&d->hdr.list, add_pos); - err =resctrl_online_domain (r, d); + err =resctrl_online_mon_domain (r, d);if (err) {list_del (&d->hdr.list);domain_free (hw_dom); } } -static void domain_remove_cpu (int cpu,struct rdt_resource *r) +/* +* domain_add_cpu - Add a cpu to either/both resource's domain lists. +*/ +static void domain_add_cpu (int cpu,struct rdt_resource *r) +{ +if (r->alloc_capable) +domain_add_cpu_ctrl (cpu, r); +if (r->mon_capable) +domain_add_cpu_mon (cpu, r); +} + +static void domain_remove_cpu_ctrl (int cpu,struct rdt_resource *r) { -int id =get_domain_id_from_scope (cpu, r->scope); +int id =get_domain_id_from_scope (cpu, r->ctrl_scope);struct rdt_hw_domain *hw_dom; +struct rdt_domain_hdr *hdr;struct rdt_domain *d;
arch/x86/kernel/cpu/resctrl/core.c static void domain_add_cpu(int cpu, struct rdt_resource *r) · line 638
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +3
-3if (id <0 )return ; - d =rdt_find_domain (r, id,NULL ); -if (!d) { -pr_warn ("Couldn't find cache id for CPU %d\n" , cpu); + hdr =rdt_find_domain (&r->ctrl_domains, id,NULL ); +if (IS_ERR_OR_NULL (hdr)) { +pr_warn ("Couldn't find control scope id=%d for CPU %d\n" , id, cpu);
Why did the !d error checking transition to IS_ERR_OR_NULL() here?
Also, the error message does not sound reasonable for what can be encountered at this point.
Quoted 25 lines
arch/x86/kernel/cpu/resctrl/core.c +11
-1return ; } + +if (WARN_ON_ONCE (hdr->type !=RESCTRL_CTRL_DOMAIN )) +return ; + + d = container_of(hdr, struct rdt_domain, hdr); hw_dom =resctrl_to_arch_dom (d);cpumask_clear_cpu (cpu, &d->hdr.cpu_mask);if (cpumask_empty (&d->hdr.cpu_mask)) { -resctrl_offline_domain (r, d); +resctrl_offline_ctrl_domain (r, d);list_del (&d->hdr.list);/* @@ -596,6 +660,38 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)return ; } +} + +static void domain_remove_cpu_mon (int cpu,struct rdt_resource *r) +{ +int id =get_domain_id_from_scope (cpu, r->mon_scope);
arch/x86/kernel/cpu/resctrl/core.c static void domain_remove_cpu(int cpu, struct rdt_resource *r) · line 677
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +10
-0+struct rdt_hw_domain *hw_dom; +struct rdt_domain_hdr *hdr; +struct rdt_domain *d; + +if (id <0 ) +return ; + + hdr =rdt_find_domain (&r->mon_domains, id,NULL ); +if (IS_ERR_OR_NULL (hdr)) { +pr_warn ("Couldn't find scope id=%d for CPU %d\n" , id, cpu);
same here
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +10
-0+return ; + } + +if (WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) +return ; + + d = container_of(hdr, struct rdt_domain, hdr); + hw_dom =resctrl_to_arch_dom (d); + +cpumask_clear_cpu (cpu, &d->hdr.cpu_mask);
arch/x86/kernel/cpu/resctrl/core.c static void domain_remove_cpu(int cpu, struct rdt_resource *r) · line 697
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +7
-0+if (cpumask_empty (&d->hdr.cpu_mask)) { +resctrl_offline_mon_domain (r, d); +list_del (&d->hdr.list); +domain_free (hw_dom); + +return ; + }if (r == &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl) {if (is_mbm_enabled () && cpu == d->mbm_work_cpu) {
Reinette
Tony Luck
maintainerPATCH v10 4/8 x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
In reply to Tony LuckThe same rdt_domain structure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.
Split into separate rdt_ctrl_domain and rdt_mon_domain structures with just the fields required for control and monitoring respectively.
Similar split of the rdt_hw_domain structure into rdt_hw_ctrl_domain and rdt_hw_mon_domain.
---
Changes since v9
Comment against patch 4, but now fixed in patch #2. cpu_mask is included in common header.
include/linux/resctrl.h | 50 +++++++------ arch/x86/kernel/cpu/resctrl/internal.h | 60 ++++++++++------ arch/x86/kernel/cpu/resctrl/core.c | 87 ++++++++++++----------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 32 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 40 +++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++-------- 7 files changed, 184 insertions(+), 153 deletions(-)
include/linux/resctrl.h +30 -20
@@ -72,7 +72,25 @@ struct rdt_domain_hdr { };/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_ctrl_domain - group of CPUs sharing a resctrl control resource +* @hdr: common header for different domain types +* @cpu_mask: which CPUs share this resource +* @plr: pseudo-locked region (if any) associated with domain +* @staged_config: parsed configuration to be applied +* @mbps_val: When mba_sc is enabled, this holds the array of user +* specified control values for mba_sc in MBps, indexed +* by closid +*/ +struct rdt_ctrl_domain { +struct rdt_domain_hdr hdr; +struct cpumask cpu_mask; +struct pseudo_lock_region *plr; +struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; +u32 *mbps_val; +}; + +/** +* struct rdt_mon_domain - group of CPUs sharing a resctrl control resource * @hdr: common header for different domain types * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold * @mbm_total: saved state for MBM total bandwidth
@@ -81,13 +99,8 @@ struct rdt_domain_hdr {* @cqm_limbo: worker to periodically read CQM h/w counters * @mbm_work_cpu: worker CPU for MBM h/w counters * @cqm_work_cpu: worker CPU for CQM h/w counters -* @plr: pseudo-locked region (if any) associated with domain -* @staged_config: parsed configuration to be applied -* @mbps_val: When mba_sc is enabled, this holds the array of user -* specified control values for mba_sc in MBps, indexed -* by closid */ -struct rdt_domain { +struct rdt_mon_domain {struct rdt_domain_hdr hdr;unsigned long *rmid_busy_llc;struct mbm_state *mbm_total;
@@ -96,9 +109,6 @@ struct rdt_domain {struct delayed_work cqm_limbo;int mbm_work_cpu;int cqm_work_cpu; -struct pseudo_lock_region *plr; -struct resctrl_staged_config staged_config[CDP_NUM_TYPES ]; -u32 *mbps_val; };/**
@@ -202,7 +212,7 @@ struct rdt_resource {const char *format_str;int (*parse_ctrlval)(struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d); +struct rdt_ctrl_domain *d);struct list_head evt_list;unsigned long fflags;bool cdp_capable;
@@ -236,15 +246,15 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);* Update the ctrl_val and apply this config right now. * Must be called on one of the domain's CPUs. */ -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val); -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type); -int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_domain *d); -void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d); +void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); +void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
@@ -260,7 +270,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_domain *d);* Return: * 0 on success, or -EIO, -EINVAL etc on error. */ -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val);/**
@@ -273,7 +283,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid);/**
@@ -285,7 +295,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* * This can be called from any CPU. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d);extern unsigned int resctrl_rmid_realloc_threshold;extern unsigned int resctrl_rmid_realloc_limit;
arch/x86/kernel/cpu/resctrl/internal.h +38 -22
131 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/core.c +46 -41
246 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +16 -16
@@ -58,7 +58,7 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r) }int parse_bw (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct resctrl_staged_config *cfg;u32 closid = data->rdtgrp->closid;
@@ -137,7 +137,7 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)* resource type. */ int parse_cbm (struct rdt_parse_data *data,struct resctrl_schema *s, -struct rdt_domain *d) +struct rdt_ctrl_domain *d) {struct rdtgroup *rdtgrp = data->rdtgrp;struct resctrl_staged_config *cfg;
@@ -206,8 +206,8 @@ static int parse_line(char *line, struct resctrl_schema *s,struct resctrl_staged_config *cfg;struct rdt_resource *r = s->res;struct rdt_parse_data data; +struct rdt_ctrl_domain *d;char *dom =NULL , *id; -struct rdt_domain *d;unsigned long dom_id; if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
@@ -267,11 +267,11 @@ static u32 get_config_index(u32 closid, enum resctrl_conf_type type) } } -static bool apply_config (struct rdt_hw_domain *hw_dom, +static bool apply_config (struct rdt_hw_ctrl_domain *hw_dom,struct resctrl_staged_config *cfg,u32 idx,cpumask_var_t cpu_mask) { -struct rdt_domain *dom = &hw_dom->d_resctrl; +struct rdt_ctrl_domain *dom = &hw_dom->d_resctrl;if (cfg->new_ctrl != hw_dom->ctrl_val[idx]) {cpumask_set_cpu (cpumask_any (&dom->hdr.cpu_mask), cpu_mask);
@@ -283,11 +283,11 @@ static bool apply_config(struct rdt_hw_domain *hw_dom,return false; } -int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_update_one (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val) { +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);u32 idx =get_config_index (closid, t);struct msr_param msr_param;
@@ -307,11 +307,11 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_domain *d,int resctrl_arch_update_domains (struct rdt_resource *r,u32 closid) {struct resctrl_staged_config *cfg; -struct rdt_hw_domain *hw_dom; +struct rdt_hw_ctrl_domain *hw_dom;struct msr_param msr_param; +struct rdt_ctrl_domain *d;enum resctrl_conf_type t;cpumask_var_t cpu_mask; -struct rdt_domain *d;u32 idx;if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))
@@ -319,7 +319,7 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid) msr_param.res =NULL ;list_for_each_entry (d, &r->ctrl_domains, hdr.list) { - hw_dom =resctrl_to_arch_dom (d); + hw_dom =resctrl_to_arch_ctrl_dom (d);for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];if (!cfg->have_new_ctrl)
@@ -449,10 +449,10 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,return ret ?: nbytes; } -u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_domain *d, +u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);u32 idx =get_config_index (closid, type);return hw_dom->ctrl_val[idx];
@@ -461,7 +461,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,static void show_doms (struct seq_file *s,struct resctrl_schema *schema,int closid) {struct rdt_resource *r = schema->res; -struct rdt_domain *dom; +struct rdt_ctrl_domain *dom;bool sep = false;u32 ctrl_val;
@@ -523,7 +523,7 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, }void mon_event_read (struct rmid_read *rr,struct rdt_resource *r, -struct rdt_domain *d,struct rdtgroup *rdtgrp, +struct rdt_mon_domain *d,struct rdtgroup *rdtgrp,int evtid,int first) {/*
@@ -543,11 +543,11 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) {struct kernfs_open_file *of = m->private;struct rdt_domain_hdr *hdr; +struct rdt_mon_domain *d;u32 resid, evtid, domid;struct rdtgroup *rdtgrp;struct rdt_resource *r;union mon_data_bits md; -struct rdt_domain *d;struct rmid_read rr;int ret =0 ;
@@ -568,7 +568,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) ret = -ENOENT ; goto out; } - d = container_of(hdr, struct rdt_domain, hdr); + d = container_of(hdr, struct rdt_mon_domain, hdr);mon_event_read (&rr, r, d, rdtgrp, evtid, false);
arch/x86/kernel/cpu/resctrl/monitor.c +20 -20
@@ -170,7 +170,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)return 0 ; } -static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_domain *hw_dom, +static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_mon_domain *hw_dom,u32 rmid,enum resctrl_event_id eventid) {
@@ -189,10 +189,10 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom,return NULL ; } -void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, +void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct arch_mbm_state *am; am =get_arch_mbm_state (hw_dom, rmid, eventid);
@@ -208,9 +208,9 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,* Assumes that hardware counters are also reset and thus that there is * no need to record initial non-zero counts. */ -void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_domain *d) +void resctrl_arch_reset_rmid_all (struct rdt_resource *r,struct rdt_mon_domain *d) { -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);if (is_mbm_total_enabled ()) memset(hw_dom->arch_mbm_total, 0,
@@ -229,11 +229,11 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)return chunks >> shift; } -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_hw_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);struct arch_mbm_state *am;u64 msr_val, chunks;int ret;
@@ -266,7 +266,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,* decrement the count. If the busy count gets to zero on an RMID, we * free the RMID */ -void __check_limbo (struct rdt_domain *d,bool force_free) +void __check_limbo (struct rdt_mon_domain *d,bool force_free) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rmid_entry *entry;
@@ -305,7 +305,7 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } } -bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d) +bool has_busy_rmid (struct rdt_resource *r,struct rdt_mon_domain *d) {return find_first_bit (d->rmid_busy_llc, r->num_rmid) != r->num_rmid; }
@@ -334,7 +334,7 @@ int alloc_rmid(void)static void add_rmid_to_limbo (struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; -struct rdt_domain *d; +struct rdt_mon_domain *d;int cpu, err;u64 val =0 ;
@@ -383,7 +383,7 @@ void free_rmid(u32 rmid)list_add_tail (&entry->list, &rmid_free_lru); } -static struct mbm_state *get_mbm_state (struct rdt_domain *d,u32 rmid, +static struct mbm_state *get_mbm_state (struct rdt_mon_domain *d,u32 rmid,enum resctrl_event_id evtid) {switch (evtid) {
@@ -516,13 +516,13 @@ void mon_event_count(void *info)* throttle MSRs already have low percentage values. To avoid * unnecessarily restricting such rdtgroups, we also increase the bandwidth. */ -static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_domain *dom_mbm) +static void update_mba_bw (struct rdtgroup *rgrp,struct rdt_mon_domain *dom_mbm) {u32 closid, rmid, cur_msr_val, new_msr_val;struct mbm_state *pmbm_data, *cmbm_data; +struct rdt_ctrl_domain *dom_mba;u32 cur_bw, delta_bw, user_bw;struct rdt_resource *r_mba; -struct rdt_domain *dom_mba;struct list_head *head;struct rdtgroup *entry;
@@ -600,7 +600,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) } } -static void mbm_update (struct rdt_resource *r,struct rdt_domain *d,int rmid) +static void mbm_update (struct rdt_resource *r,struct rdt_mon_domain *d,int rmid) {struct rmid_read rr;
@@ -640,13 +640,13 @@ void cqm_handle_limbo(struct work_struct *work) {unsigned long delay =msecs_to_jiffies (CQM_LIMBOCHECK_INTERVAL );int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex); r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, cqm_limbo.work); + d = container_of(work, struct rdt_mon_domain, cqm_limbo.work);__check_limbo (d, false);
@@ -656,7 +656,7 @@ void cqm_handle_limbo(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms) +void cqm_setup_limbo_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
@@ -672,9 +672,9 @@ void mbm_handle_overflow(struct work_struct *work)unsigned long delay =msecs_to_jiffies (MBM_OVERFLOW_INTERVAL );struct rdtgroup *prgrp, *crgrp;int cpu =smp_processor_id (); +struct rdt_mon_domain *d;struct list_head *head;struct rdt_resource *r; -struct rdt_domain *d;mutex_lock (&rdtgroup_mutex);
@@ -682,7 +682,7 @@ void mbm_handle_overflow(struct work_struct *work) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; - d = container_of(work, struct rdt_domain, mbm_over.work); + d = container_of(work, struct rdt_mon_domain, mbm_over.work);list_for_each_entry (prgrp, &rdt_all_groups, rdtgroup_list) {mbm_update (r, d, prgrp->mon.rmid);
@@ -701,7 +701,7 @@ void mbm_handle_overflow(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms) +void mbm_setup_overflow_handler (struct rdt_mon_domain *dom,unsigned long delay_ms) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -3
@@ -814,7 +814,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)* Return: true if @cbm overlaps with pseudo-locked region on @d, false * otherwise. */ -bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,unsigned long cbm) +bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_ctrl_domain *d,unsigned long cbm) {unsigned int cbm_len;unsigned long cbm_b;
@@ -841,11 +841,11 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm* if it is not possible to test due to memory allocation issue, * false otherwise. */ -bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d) +bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_ctrl_domain *d) { +struct rdt_ctrl_domain *d_i;cpumask_var_t cpu_with_psl;struct rdt_resource *r; -struct rdt_domain *d_i;bool ret = false;if (!zalloc_cpumask_var (&cpu_with_psl,GFP_KERNEL ))
arch/x86/kernel/cpu/resctrl/rdtgroup.c +31 -31
252 lines not on this page · raw message
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v10 4/8] x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
Hi Tony,
Quoted 37 lines · Tony Luck
The same
rdt_domainstructure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.Split into separate
rdt_ctrl_domainandrdt_mon_domainstructures with just the fields required for control and monitoring respectively.Similar split of the
rdt_hw_domainstructure intordt_hw_ctrl_domainandrdt_hw_mon_domain.Signed-off-by Tony Luck <[email protected]>---
Changes since v9
Comment against patch 4, but now fixed in patch #2.cpu_maskis included in common header.include/linux/resctrl.h | 50 +++++++------ arch/x86/kernel/cpu/resctrl/internal.h | 60 ++++++++++------ arch/x86/kernel/cpu/resctrl/core.c | 87 ++++++++++++----------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 32 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 40 +++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++-------- 7 files changed, 184 insertions(+), 153 deletions(-)include/linux/resctrl.h +2
-1@@ -72,7 +72,25 @@ struct rdt_domain_hdr { };/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_ctrl_domain - group of CPUs sharing a resctrl control resource +* @hdr: common header for different domain types
include/linux/resctrl.h struct rdt_domain_hdr { · line 86
Quoted 10 lines
include/linux/resctrl.h +10
-0+* @cpu_mask: which CPUs share this resource +* @plr: pseudo-locked region (if any) associated with domain +* @staged_config: parsed configuration to be applied +* @mbps_val: When mba_sc is enabled, this holds the array of user +* specified control values for mba_sc in MBps, indexed +* by closid +*/ +struct rdt_ctrl_domain { +struct rdt_domain_hdr hdr; +struct cpumask cpu_mask;
This patch did not change what it said it changed.
Reinette
Tony Luck
maintainerRe: [PATCH v10 4/8] x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
Quoted 36 lines · Reinette Chatre
Hi Tony,
Quoted 33 lines · Tony Luck
The same
rdt_domainstructure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.Split into separate
rdt_ctrl_domainandrdt_mon_domainstructures with just the fields required for control and monitoring respectively.Similar split of the
rdt_hw_domainstructure intordt_hw_ctrl_domainandrdt_hw_mon_domain.Signed-off-by Tony Luck <[email protected]>---
Changes since v9
Comment against patch 4, but now fixed in patch #2.cpu_maskis included in common header.include/linux/resctrl.h | 50 +++++++------ arch/x86/kernel/cpu/resctrl/internal.h | 60 ++++++++++------ arch/x86/kernel/cpu/resctrl/core.c | 87 ++++++++++++----------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 32 ++++----- arch/x86/kernel/cpu/resctrl/monitor.c | 40 +++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++-------- 7 files changed, 184 insertions(+), 153 deletions(-)include/linux/resctrl.h +0
-0@@ -72,7 +72,25 @@ struct rdt_domain_hdr { };
Quoted 16 lines
Quoted 4 lines
include/linux/resctrl.h +2
-1/** -* struct rdt_domain - group of CPUs sharing a resctrl resource +* struct rdt_ctrl_domain - group of CPUs sharing a resctrl control resource +* @hdr: common header for different domain types include/linux/resctrl.h struct rdt_domain_hdr { · line 86
Quoted 10 lines
include/linux/resctrl.h +10
-0+* @cpu_mask: which CPUs share this resource +* @plr: pseudo-locked region (if any) associated with domain +* @staged_config: parsed configuration to be applied +* @mbps_val: When mba_sc is enabled, this holds the array of user +* specified control values for mba_sc in MBps, indexed +* by closid +*/ +struct rdt_ctrl_domain { +struct rdt_domain_hdr hdr; +struct cpumask cpu_mask;This patch did not change what it said it changed.
Reinette,
I'm not sure of the problem. The commit said the patch is splitting the rdt_domain structure into rdt_ctrl_domain and rdt_mon_domain.
The piece of the patch where you gave this comment changed like this:
------- Before -------
/**
* struct rdt_domain - group of CPUs sharing a resctrl resource
* @hdr: common header for different domain types
* @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold
* @mbm_total: saved state for MBM total bandwidth
* @mbm_local: saved state for MBM local bandwidth
* @mbm_over: worker to periodically read MBM h/w counters
* @cqm_limbo: worker to periodically read CQM h/w counters
* @mbm_work_cpu: worker CPU for MBM h/w counters
* @cqm_work_cpu: worker CPU for CQM h/w counters
* @plr: pseudo-locked region (if any) associated with domain
* @staged_config: parsed configuration to be applied
* @mbps_val: When mba_sc is enabled, this holds the array of user
* specified control values for mba_sc in MBps, indexed
* by closid
*/
struct rdt_domain {
struct rdt_domain_hdr hdr;
unsigned long *rmid_busy_llc;
struct mbm_state *mbm_total;
struct mbm_state *mbm_local;
struct delayed_work mbm_over;
struct delayed_work cqm_limbo;
int mbm_work_cpu;
int cqm_work_cpu;
struct pseudo_lock_region *plr;
struct resctrl_staged_config staged_config[CDP_NUM_TYPES];
u32 *mbps_val;
};------- After -------
/**
* struct rdt_ctrl_domain - group of CPUs sharing a resctrl control resource
* @hdr: common header for different domain types
* @cpu_mask: which CPUs share this resource
* @plr: pseudo-locked region (if any) associated with domain
* @staged_config: parsed configuration to be applied
* @mbps_val: When mba_sc is enabled, this holds the array of user
* specified control values for mba_sc in MBps, indexed
* by closid
*/
struct rdt_ctrl_domain {
struct rdt_domain_hdr hdr;
struct cpumask cpu_mask;
struct pseudo_lock_region *plr;
struct resctrl_staged_config staged_config[CDP_NUM_TYPES];
u32 *mbps_val;
};/**
* struct rdt_mon_domain - group of CPUs sharing a resctrl control resource
* @hdr: common header for different domain types
* @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold
* @mbm_total: saved state for MBM total bandwidth
* @mbm_local: saved state for MBM local bandwidth
* @mbm_over: worker to periodically read MBM h/w counters
* @cqm_limbo: worker to periodically read CQM h/w counters
* @mbm_work_cpu: worker CPU for MBM h/w counters
* @cqm_work_cpu: worker CPU for CQM h/w counters
*/
struct rdt_mon_domain {
struct rdt_domain_hdr hdr;
unsigned long *rmid_busy_llc;
struct mbm_state *mbm_total;
struct mbm_state *mbm_local;
struct delayed_work mbm_over;
struct delayed_work cqm_limbo;
int mbm_work_cpu;
int cqm_work_cpu;
};-----
Which to my eyes looks like exactly what the commit comment says I was going to do the in this patch. The old rdt_domain structure has been replaced with two structures, with names as described in the commit comment. The fields of the orginal structure have been distributed betweeen the two new structures based on whether they are used for control or monitor functions.
What am I missing? (Apart from a silly cut & paste error in the comments that I just noticed and will fix now).
-Tony
Reinette Chatre
maintainerRe: [PATCH v10 4/8] x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
Hi Tony,
Quoted 12 lines · Tony Luck
Quoted 11 lines · Reinette Chatre
Hi Tony,
Quoted 8 lines · Tony Luck
The same
rdt_domainstructure is used for both control and monitor functions. But this results in wasted memory as some of the fields are only used by control functions, while most are only used for monitor functions.Split into separate
rdt_ctrl_domainandrdt_mon_domainstructures with just the fields required for control and monitoring respectively.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Similar split of the
rdt_hw_domainstructure intordt_hw_ctrl_domainandrdt_hw_mon_domain.Signed-off-by Tony Luck <[email protected]>---
Changes since v9
Comment against patch 4, but now fixed in patch #2.cpu_maskis included in common header.
This is the expected change I referred to. Specifically:
"cpu_mask is included in common header"
Quoted 2 lines
What am I missing? (Apart from a silly cut & paste error in the comments that I just noticed and will fix now).
struct rdt_ctrl_domain {
struct rdt_domain_hdr hdr;
struct cpumask cpu_mask;
...
}Considering the description of the changes to expect in this version I did not expect to see a cpu_mask member in struct rdt_ctrl_domain since it has now been moved to struct rdt_domain_hdr. What am I missing?
Reinette
Luck, Tony
maintainerRE: [PATCH v10 4/8] x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
Quoted 9 lines
struct rdt_ctrl_domain { struct rdt_domain_hdr hdr; struct cpumask cpu_mask; ... }Considering the description of the changes to expect in this version I did not expect to see a
cpu_maskmember in structrdt_ctrl_domainsince it has now been moved to structrdt_domain_hdr. What am I missing?
Reinette,
Yes. cpu_mask was moved into "struct rdt_domain_hdr" in patch 2 of the series. I'm not sure how it managed to reappear here. Definitely a zombie. I've removed it now.
-Tony
Tony Luck
maintainerPATCH v10 5/8 x86/resctrl: Add node-scope to the options for feature scope
In reply to Tony LuckCurrently supported resctrl features are all domain scoped the same as the scope of the L2 or L3 caches.
Add RESCTRL_NODE as a new option for features that are scoped at the same granularity as NUMA nodes. This is needed for Intel's Sub-NUMA Cluster (SNC) feature where monitoring features are node scoped.
---
No changes since v9
include/linux/resctrl.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 2 ++ 2 files changed, 3 insertions(+)
include/linux/resctrl.h +1 -0
@@ -172,6 +172,7 @@ struct resctrl_schema;enum resctrl_scope {RESCTRL_L2_CACHE =2 ,RESCTRL_L3_CACHE =3 , +RESCTRL_NODE , };/**
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -502,6 +502,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)case RESCTRL_L2_CACHE :case RESCTRL_L3_CACHE :return get_cpu_cacheinfo_id (cpu, scope); +case RESCTRL_NODE : +return cpu_to_node (cpu);default :break ; }
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v10 6/8 x86/resctrl: Introduce snc_nodes_per_l3_cache
In reply to Tony LuckIntel Sub-NUMA Cluster (SNC) is a feature that subdivides the CPU cores and memory controllers on a socket into two or more groups. These are presented to the operating system as NUMA nodes.
This may enable some workloads to have slightly lower latency to memory as the memory controller(s) in an SNC node are electrically closer to the CPU cores on that SNC node. This cost may be offset by lower bandwidth since the memory accesses for each core can only be interleaved between the memory controllers on the same SNC node.
Resctrl monitoring on an Intel system depends upon attaching RMIDs to tasks to track L3 cache occupancy and memory bandwidth. There is an MSR that controls how the RMIDs are shared between SNC nodes.
The default mode divides them numerically. E.g. when there are two SNC nodes on a socket the lower number half of the RMIDs are given to the first node, the remainder to the second node. This would be difficult to use with the Linux resctrl interface as specific RMID values assigned to resctrl groups are not visible to users.
The other mode divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
Even with this renumbering SNC mode requires several changes in resctrl behavior for correct operation.
Add a global integer "snc_nodes_per_l3_cache" that will show how many SNC nodes share each L3 cache. When this is "1", SNC mode is either not implemented, or not enabled, but all places that need to check it are updated to take appropriate action when SNC mode is enabled.
Code that needs to take action when SNC is enabled is:
1) The number of logical RMIDs per L3 cache available for use is the number of physical RMIDs divided by the number of SNC nodes.
2) Likewise the "mon_scale" value must be divided by the number of SNC nodes.
3) The RMID renumbering operates when using the value from the IA32_PQR_ASSOC MSR to count accesses by a task. When reading an RMID counter, code must adjust from the logical RMID used to the physical RMID value for the SNC node that it wishes to read and load the adjusted value into the IA32_QM_EVTSEL MSR.
4) The L3 cache is divided between the SNC nodes. So the value reported in the resctrl "size" file is divided by the number of SNC nodes because the effective amount of cache that can be allocated is reduced by that factor.
5) The "-o mba_MBps" mount option must be disabled in SNC mode because the monitoring is being done per SNC node, while the bandwidth allocation is still done at the L3 cache scope. Trying to use this feedback loop might result in contradictory changes to the throttling level coming from each of the SNC node bandwidth measurements.
---
Changes since v9
Fixed missing word s/monitoring on Intel/monitoring on an Intel/ Deleted "A later patch" paragraph.
Expanded description how how values are "adjusted" for mon_scale and cache size.
Changed type of "snc_nodes_per_l3_cache" to "unsigned int".
arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/core.c | 6 ++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 16 +++++++++++++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 +++-- 4 files changed, 24 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -446,6 +446,8 @@ DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);extern struct dentry *debugfs_resctrl; +extern unsigned int snc_nodes_per_l3_cache; +enum resctrl_res_level {RDT_RESOURCE_L3 ,RDT_RESOURCE_L2 ,
arch/x86/kernel/cpu/resctrl/core.c +6 -0
@@ -48,6 +48,12 @@ int max_name_width, max_data_width;*/ bool rdt_alloc_capable; +/* +* Number of SNC nodes that share each L3 cache. Default is 1 for +* systems that do not support SNC, or have SNC disabled. +*/ +unsigned int snc_nodes_per_l3_cache =1 ; +static void mba_wrmsr_intel (struct rdt_ctrl_domain *d,struct msr_param *m,struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/monitor.c +13 -3
@@ -148,8 +148,18 @@ static inline struct rmid_entry *__rmid_entry(u32 rmid)static int __rmid_read (u32 rmid,enum resctrl_event_id eventid,u64 *val) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +int cpu =smp_processor_id (); +int rmid_offset =0 ;u64 msr_val; +/* +* When SNC mode is on, need to compute the offset to read the +* physical RMID counter for the node to which this CPU belongs. +*/ +if (snc_nodes_per_l3_cache >1 ) + rmid_offset = (cpu_to_node (cpu) % snc_nodes_per_l3_cache) * r->num_rmid; +/* * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured * with a valid event code for supported resource type and the bits
@@ -158,7 +168,7 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)* IA32_QM_CTR.Error (bit 63) and IA32_QM_CTR.Unavailable (bit 62) * are error bits. */ -wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid); +wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid + rmid_offset);rdmsrl (MSR_IA32_QM_CTR , msr_val);if (msr_val &RMID_VAL_ERROR )
@@ -783,8 +793,8 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)int ret; resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size *1024 ; - hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale; - r->num_rmid = boot_cpu_data.x86_cache_max_rmid +1 ; + hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache; + r->num_rmid = (boot_cpu_data.x86_cache_max_rmid +1 ) / snc_nodes_per_l3_cache; hw_res->mbm_width =MBM_CNTR_WIDTH_BASE ;if (mbm_offset >0 && mbm_offset <=MBM_CNTR_WIDTH_OFFSET_MAX )
arch/x86/kernel/cpu/resctrl/rdtgroup.c +3 -2
@@ -1425,7 +1425,7 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, } } -return size; +return size / snc_nodes_per_l3_cache; }/*
@@ -2298,7 +2298,8 @@ static bool supports_mba_mbps(void)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA ].r_resctrl;return (is_mbm_local_enabled () && - r->alloc_capable &&is_mba_linear ()); + r->alloc_capable &&is_mba_linear () && + snc_nodes_per_l3_cache ==1 ); }/*
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v10 7/8 x86/resctrl: Sub NUMA Cluster detection and enable
In reply to Tony LuckThere isn't a simple hardware bit that indicates whether a CPU is running in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the ratio of NUMA nodes to L3 cache instances.
When SNC mode is detected, reconfigure the RMID counters by updating the MSR_RMID_SNC_CONFIG MSR on each socket as CPUs are seen.
Clearing bit zero of the MSR divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
---
Changes since v9
Expand h/w to hardware (commit and code comments) Remove "earlier commit" reference
s/counnter/counter/
Check for offline CPUs and warn user SNC detection may be broken.
arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 100 ++++++++++++++++++++++++++++- 2 files changed, 99 insertions(+), 2 deletions(-)
arch/x86/include/asm/msr-index.h +1 -0
@@ -1109,6 +1109,7 @@#define MSR_IA32_QM_CTR 0xc8e#define MSR_IA32_PQR_ASSOC 0xc8f#define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0#define MSR_IA32_L2_CBM_BASE 0xd10#define MSR_IA32_MBA_THRTL_BASE 0xd50
arch/x86/kernel/cpu/resctrl/core.c +98 -2
141 lines not on this page · raw message
Signature · 1 line
2.41.0
Reinette Chatre
maintainerRe: [PATCH v10 7/8] x86/resctrl: Sub NUMA Cluster detection and enable
Hi Tony,
Quoted 56 lines · Tony Luck
There isn't a simple hardware bit that indicates whether a CPU is running in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the ratio of NUMA nodes to L3 cache instances.
When SNC mode is detected, reconfigure the RMID counters by updating the
MSR_RMID_SNC_CONFIGMSR on each socket as CPUs are seen.Clearing bit zero of the MSR divides the RMIDs and renumbers the ones on the second SNC node to start from zero.
Signed-off-by Tony Luck <[email protected]>---
Changes since v9
Expand h/w to hardware (commit and code comments) Remove "earlier commit" reference
s/counnter/counter/
Check for offline CPUs and warn user SNC detection may be broken.arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 100 ++++++++++++++++++++++++++++- 2 files changed, 99 insertions(+), 2 deletions(-)arch/x86/include/asm/msr-index.h +1
-0@@ -1109,6 +1109,7 @@#define MSR_IA32_QM_CTR 0xc8e#define MSR_IA32_PQR_ASSOC 0xc8f#define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0#define MSR_IA32_L2_CBM_BASE 0xd10#define MSR_IA32_MBA_THRTL_BASE 0xd50arch/x86/kernel/cpu/resctrl/core.c +3
-0@@ -16,11 +16,14 @@#define pr_fmt (fmt) "resctrl: " fmt +#include <linux/cpu.h> #include <linux/slab.h> #include <linux/err.h> #include <linux/cacheinfo.h> #include <linux/cpuhotplug.h> +#include <linux/mod_devicetable.h> +#include <asm/cpu_device_id.h> #include <asm/intel-family.h> #include <asm/resctrl.h> #include "internal.h" @@ -184,10 +187,10 @@ bool is_mba_sc(struct rdt_resource *r)/*
arch/x86/kernel/cpu/resctrl/core.c bool is_mba_sc(struct rdt_resource *r) · line 196
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +2
-2* rdt_get_mb_table() - get a mapping of bandwidth(b/w) percentage values -* exposed to user interface and the h/w understandable delay values. +* exposed to user interface and the hardware understandable delay values. * * The non-linear delay values have the granularity of power of two -* and also the h/w does not guarantee a curve for configured delay +* and also the hardware does not guarantee a curve for configured delay * values vs. actual b/w enforced. * Hence we need a mapping that is pre calibrated so the user can * express the memory b/w as a percentage value.
This seems out of place here. If you want to make such a global change it can be done as a separate patch. For this work it can just consistently use "hardware" in own areas changed.
Reinette
Tony Luck
maintainerPATCH v10 8/8 x86/resctrl: Update documentation with Sub-NUMA cluster changes
In reply to Tony LuckWith Sub-NUMA Cluster mode enabled the scope of monitoring resources is per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in their name refer to Sub-NUMA nodes instead of L3 cache ids.
Users should be aware that SNC mode also affects the amount of L3 cache available for allocation within each SNC node.
---
No changes since v9
Documentation/arch/x86/resctrl.rst | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
Documentation/arch/x86/resctrl.rst +20 -3
@@ -366,9 +366,9 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these + This contains a set of files organized by L3 domain or by NUMA + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled + or enabled respectively) and by RDT event. Each of these directories have one file per event (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these files provide a read out of the current value of the event for
@@ -478,6 +478,23 @@ if non-contiguous 1s value is supported. On a system with a 20-bit mask each bit represents 5% of the capacity of the cache. You could partition the cache into four equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000. +Notes on Sub-NUMA Cluster mode +============================== +When SNC mode is enabled Linux may load balance tasks between Sub-NUMA +nodes much more readily than between regular NUMA nodes since the CPUs +on Sub-NUMA nodes share the same L3 cache and the system may report +the NUMA distance between Sub-NUMA nodes with a lower value than used +for regular NUMA nodes. Users who do not bind tasks to the CPUs of a +specific Sub-NUMA node must read the "llc_occupancy", "mbm_total_bytes", +and "mbm_local_bytes" for all Sub-NUMA nodes where the tasks may execute +to get the full view of traffic for which the tasks were the source. + +The cache allocation feature still provides the same number of +bits in a mask to control allocation into the L3 cache. But each +of those ways has its capacity reduced because the cache is divided +between the SNC nodes. The values reported in the resctrl +"size" files are adjusted accordingly. + Memory bandwidth Allocation and monitoring ==========================================
Signature · 1 line
2.41.0
Tony Luck
maintainerv11 cover Add support for Sub-NUMA cluster (SNC) systems
In reply to Tony LuckThe Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible.
Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows monitoring features to be used. With the caveat that users must be aware that Linux may migrate tasks more frequently between SNC nodes than between "regular" NUMA nodes, so reading counters from all SNC nodes may be needed to get a complete picture of activity for tasks.
Cache and memory bandwidth allocation features continue to operate at the scope of the L3 cache.
Changes since v10 to patches 1, 3, 4, 7. See patches for details.
Tony Luck (8):
x86/resctrl: Prepare for new domain scope x86/resctrl: Prepare to split rdt_domain structure x86/resctrl: Prepare for different scope for control/monitor
operations
x86/resctrl: Split the rdt_domain and rdt_hw_domain structures x86/resctrl: Add node-scope to the options for feature scope x86/resctrl: Introduce snc_nodes_per_l3_cache x86/resctrl: Sub NUMA Cluster detection and enable x86/resctrl: Update documentation with Sub-NUMA cluster changes
Documentation/arch/x86/resctrl.rst | 23 +- include/linux/resctrl.h | 85 +++-- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 66 ++-- arch/x86/kernel/cpu/resctrl/core.c | 403 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 58 ++-- arch/x86/kernel/cpu/resctrl/monitor.c | 68 ++-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 26 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 149 ++++---- 9 files changed, 598 insertions(+), 281 deletions(-)
base-commit: 5a6a09e97199d6600d31383055f9d43fbbcbe86f
Signature · 1 line
2.41.0
Tony Luck
maintainerPATCH v11 1/8 x86/resctrl: Prepare for new domain scope
Resctrl resources operate on subsets of CPUs in the system with the defining attribute of each subset being an instance of a particular level of cache. E.g. all CPUs sharing an L3 cache would be part of the same domain.
In preparation for features that are scoped at the NUMA node level change the code from explicit references to "cache_level" to a more generic scope. At this point the only options for this scope are groups of CPUs that share an L2 cache or L3 cache.
Clean up the error handling when looking up domains. Report invalid id's before calling rdt_find_domain() in preparation for better messages when scope can be other than cache scope. This means that rdt_find_domain() will never return an error. So remove checks for error from the callsites.
---
Changes since v10:
Reinette: Move a blank line in domain_add_cpu()
Reniette: Fix warning message in domain_remove_cpu()
include/linux/resctrl.h | 9 +++-- arch/x86/kernel/cpu/resctrl/core.c | 42 +++++++++++++++-------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 +++- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 5 ++- 5 files changed, 45 insertions(+), 19 deletions(-)
include/linux/resctrl.h +7 -2
@@ -144,13 +144,18 @@ struct resctrl_membw {struct rdt_parse_data ;struct resctrl_schema ; +enum resctrl_scope { +RESCTRL_L2_CACHE =2 , +RESCTRL_L3_CACHE =3 , +}; +/** * struct rdt_resource - attributes of a resctrl resource * @rid: The index of the resource * @alloc_capable: Is allocation available on this machine * @mon_capable: Is monitor feature available on this machine * @num_rmid: Number of RMIDs available -* @cache_level: Which cache level defines scope of this resource +* @scope: Scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. * @domains: All domains for this resource
@@ -168,7 +173,7 @@ struct rdt_resource {bool alloc_capable;bool mon_capable;int num_rmid; -int cache_level; +enum resctrl_scope scope;struct resctrl_cache cache;struct resctrl_membw membw;struct list_head domains;
arch/x86/kernel/cpu/resctrl/core.c +28 -14
102 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +1 -1
@@ -563,7 +563,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) r = &rdt_resources_all[resid].r_resctrl; d =rdt_find_domain (r, domid,NULL ); -if (IS_ERR_OR_NULL (d)) { +if (!d) { ret = -ENOENT ; goto out; }
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +5 -1
@@ -292,10 +292,14 @@ static void pseudo_lock_region_clear(struct pseudo_lock_region *plr)*/ static int pseudo_lock_region_init (struct pseudo_lock_region *plr) { +enum resctrl_scope scope = plr->s->res->scope;struct cpu_cacheinfo *ci;int ret;int i; +if (WARN_ON_ONCE (scope !=RESCTRL_L2_CACHE && scope !=RESCTRL_L3_CACHE )) +return -ENODEV ; +/* Pick the first cpu we find that is associated with the cache. */ plr->cpu =cpumask_first (&plr->d->cpu_mask);
@@ -311,7 +315,7 @@ static int pseudo_lock_region_init(struct pseudo_lock_region *plr) plr->size =rdtgroup_cbm_to_size (plr->s->res, plr->d, plr->cbm);for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == plr->s->res->cache_level) { +if (ci->info_list[i].level == scope) { plr->line_size = ci->info_list[i].coherency_line_size;return 0 ; }
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -1
@@ -1413,10 +1413,13 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,unsigned int size =0 ;int num_b, i; +if (WARN_ON_ONCE (r->scope !=RESCTRL_L2_CACHE && r->scope !=RESCTRL_L3_CACHE )) +return size; + num_b =bitmap_weight (&cbm, r->cache.cbm_len); ci =get_cpu_cacheinfo (cpumask_any (&d->cpu_mask));for (i =0 ; i < ci->num_leaves; i++) { -if (ci->info_list[i].level == r->cache_level) { +if (ci->info_list[i].level == r->scope) { size = ci->info_list[i].size / r->cache.cbm_len * num_b;break ; }
Signature · 1 line
2.41.0