Documentation: x86: resctrl_ui.txt fixes and clarification
cover Documentation: x86: resctrl_ui.txt fixes and clarification
While updating resctrl_ui.rst for arm64 support these things stuck out as being unclear or no longer true.
(I've shortened the CC list to just RDT+Documentation)
Thanks,
James Morse (4):
Documentation: x86: Contiguous cbm isn't all X86
Documentation: x86: Remove cdpl2 unspported statement and fix
capitalisation
Documentation: x86: Clarify MBA takes MB as referring to mba_sc
Documentation: x86: fix some typos
Documentation/x86/resctrl_ui.rst | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)
Signature · 1 line
2.20.1
PATCH 1/4 Documentation: x86: Contiguous cbm isn't all X86
Since commit 4d05bf71f157 ("x86/resctrl: Introduce AMD QOS feature") resctrl has supported non-contiguous cache bit masks. The interface for this is currently try-it-and-see.
Update the documentation to say Intel CPUs have this requirement, instead of X86.
--- Documentation/x86/resctrl_ui.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/x86/resctrl_ui.rst +1 -1
@@ -342,7 +342,7 @@ For cache resources we describe the portion of the cache that is available for allocation using a bitmask. The maximum value of the mask is defined by each cpu model (and may be different for different cache levels). It is found using CPUID, but is also provided in the "info" directory of -the resctrl file system in "info/{resource}/cbm_mask". X86 hardware +the resctrl file system in "info/{resource}/cbm_mask". Intel hardware requires that these masks have all the '1' bits in a contiguous block. So 0x3, 0x6 and 0xC are legal 4-bit masks with two bits set, but 0x5, 0x9 and 0xA are not. On a system with a 20-bit mask each bit represents 5%
Signature · 1 line
2.20.1
Re: [PATCH 1/4] Documentation: x86: Contiguous cbm isn't all X86
Quoted 29 lines · James Morse
Since commit
4d05bf71f157("x86/resctrl: Introduce AMD QOS feature") resctrl has supported non-contiguous cache bit masks. The interface for this is currently try-it-and-see.Update the documentation to say Intel CPUs have this requirement, instead of X86.
Cc Babu Moger <[email protected]>Signed-off-by James Morse <[email protected]>--- Documentation/x86/resctrl_ui.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Documentation/x86/resctrl_ui.rst +1
-1@@ -342,7 +342,7 @@ For cache resources we describe the portion of the cache that is available for allocation using a bitmask. The maximum value of the mask is defined by each cpu model (and may be different for different cache levels). It is found using CPUID, but is also provided in the "info" directory of -the resctrl file system in "info/{resource}/cbm_mask". X86 hardware +the resctrl file system in "info/{resource}/cbm_mask". Intel hardware requires that these masks have all the '1' bits in a contiguous block. So 0x3, 0x6 and 0xC are legal 4-bit masks with two bits set, but 0x5, 0x9 and 0xA are not. On a system with a 20-bit mask each bit represents 5%Signature · 1 line
2.20.1
Thanks.
-Fenghua Yu
PATCH 2/4 Documentation: x86: Remove cdpl2 unspported statement and fix capitalisation
In reply to James Morse"L2 cache does not support code and data prioritization". This isn't true, elsewhere the document says it can be enabled with the cdpl2 mount option.
While we're here, these sample strings have lower-case code/data, which isn't how the kernel exports them.
--- Documentation/x86/resctrl_ui.rst | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
Documentation/x86/resctrl_ui.rst +10 -4
@@ -418,16 +418,22 @@ L3 schemata file details (CDP enabled via mount option to resctrl) When CDP is enabled L3 control is split into two separate resources so you can specify independent masks for code and data like this:: - L3data:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... - L3code:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... + L3DATA:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... + L3CODE:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... L2 schemata file details ------------------------ -L2 cache does not support code and data prioritization, so the -schemata format is always:: +CDP is supported at L2 using the 'cdpl2' mount option. The schemata +format is either:: L2:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... +or + + L2DATA:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... + L2CODE:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... + + Memory bandwidth Allocation (default mode) ------------------------------------------
Signature · 1 line
2.20.1
Re: [PATCH 2/4] Documentation: x86: Remove cdpl2 unspported statement and fix capitalisation
Quoted 46 lines · James Morse
"L2 cache does not support code and data prioritization". This isn't true, elsewhere the document says it can be enabled with the cdpl2 mount option.
While we're here, these sample strings have lower-case code/data, which isn't how the kernel exports them.
Signed-off-by James Morse <[email protected]>--- Documentation/x86/resctrl_ui.rst | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)Documentation/x86/resctrl_ui.rst +10
-4@@ -418,16 +418,22 @@ L3 schemata file details (CDP enabled via mount option to resctrl) When CDP is enabled L3 control is split into two separate resources so you can specify independent masks for code and data like this:: - L3data:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... - L3code:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... + L3DATA:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... + L3CODE:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... L2 schemata file details ------------------------ -L2 cache does not support code and data prioritization, so the -schemata format is always:: +CDP is supported at L2 using the 'cdpl2' mount option. The schemata +format is either:: L2:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... +or + + L2DATA:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... + L2CODE:<cache_id0>=<cbm>;<cache_id1>=<cbm>;... + + Memory bandwidth Allocation (default mode) ------------------------------------------Signature · 1 line
2.20.1
Thanks.
-Fenghua
PATCH 3/4 Documentation: x86: Clarify MBA takes MB as referring to mba_sc
In reply to James Morse"If the MBA is specified in MB then user can enter the max b/w in MB" is a tautology. How can the user know if the schemata takes a percentage or a MB/s value?
This is referring to whether the software controller is interpreting the schemata's value. Make this clear.
--- Documentation/x86/resctrl_ui.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/x86/resctrl_ui.rst +2 -2
@@ -677,8 +677,8 @@ allocations can overlap or not. The allocations specifies the maximum b/w that the group may be able to use and the system admin can configure the b/w accordingly. -If the MBA is specified in MB(megabytes) then user can enter the max b/w in MB -rather than the percentage values. +If resctrl is using the software controller (mba_sc) then user can enter the +max b/w in MB rather than the percentage values. :: # echo "L3:0=3;1=c\nMB:0=1024;1=500" > /sys/fs/resctrl/p0/schemata
Signature · 1 line
2.20.1
Re: [PATCH 3/4] Documentation: x86: Clarify MBA takes MB as referring to mba_sc
Quoted 30 lines · James Morse
"If the MBA is specified in MB then user can enter the max b/w in MB" is a tautology. How can the user know if the schemata takes a percentage or a MB/s value?
This is referring to whether the software controller is interpreting the schemata's value. Make this clear.
Signed-off-by James Morse <[email protected]>--- Documentation/x86/resctrl_ui.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Documentation/x86/resctrl_ui.rst +2
-2@@ -677,8 +677,8 @@ allocations can overlap or not. The allocations specifies the maximum b/w that the group may be able to use and the system admin can configure the b/w accordingly. -If the MBA is specified in MB(megabytes) then user can enter the max b/w in MB -rather than the percentage values. +If resctrl is using the software controller (mba_sc) then user can enter the +max b/w in MB rather than the percentage values. :: # echo "L3:0=3;1=c\nMB:0=1024;1=500" > /sys/fs/resctrl/p0/schemataSignature · 1 line
2.20.1
Thanks.
-Fenghua
PATCH 4/4 Documentation: x86: fix some typos
In reply to James MorseThese are all obvious typos.
--- Documentation/x86/resctrl_ui.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
Documentation/x86/resctrl_ui.rst +5 -5
@@ -40,7 +40,7 @@ mount options are: Enable the MBA Software Controller(mba_sc) to specify MBA bandwidth in MBps -L2 and L3 CDP are controlled seperately. +L2 and L3 CDP are controlled separately. RDT features are orthogonal. A particular system may support only monitoring, only control, or both monitoring and control. Cache
@@ -118,7 +118,7 @@ related to allocation: Corresponding region is pseudo-locked. No sharing allowed. -Memory bandwitdh(MB) subdirectory contains the following files +Memory bandwidth(MB) subdirectory contains the following files with respect to allocation: "min_bandwidth":
@@ -209,7 +209,7 @@ All groups contain the following files: CPUs to/from this group. As with the tasks file a hierarchy is maintained where MON groups may only include CPUs owned by the parent CTRL_MON group. - When the resouce group is in pseudo-locked mode this file will + When the resource group is in pseudo-locked mode this file will only be readable, reflecting the CPUs associated with the pseudo-locked region.
@@ -380,7 +380,7 @@ where L2 external is 10GBps (hence aggregate L2 external bandwidth is 240GBps) and L3 external bandwidth is 100GBps. Now a workload with '20 threads, having 50% bandwidth, each consuming 5GBps' consumes the max L3 bandwidth of 100GBps although the percentage value specified is only 50% -<< 100%. Hence increasing the bandwidth percentage will not yeild any +<< 100%. Hence increasing the bandwidth percentage will not yield any more bandwidth. This is because although the L2 external bandwidth still has capacity, the L3 external bandwidth is fully used. Also note that this would be dependent on number of cores the benchmark is run on.
@@ -398,7 +398,7 @@ In order to mitigate this and make the interface more user friendly, resctrl added support for specifying the bandwidth in MBps as well. The kernel underneath would use a software feedback mechanism or a "Software Controller(mba_sc)" which reads the actual bandwidth using MBM counters -and adjust the memowy bandwidth percentages to ensure:: +and adjust the memory bandwidth percentages to ensure:: "actual bandwidth < user specified bandwidth".
Signature · 1 line
2.20.1
Re: [PATCH 4/4] Documentation: x86: fix some typos
Quoted 59 lines · James Morse
These are all obvious typos.
Signed-off-by James Morse <[email protected]>--- Documentation/x86/resctrl_ui.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)Documentation/x86/resctrl_ui.rst +5
-5@@ -40,7 +40,7 @@ mount options are: Enable the MBA Software Controller(mba_sc) to specify MBA bandwidth in MBps -L2 and L3 CDP are controlled seperately. +L2 and L3 CDP are controlled separately. RDT features are orthogonal. A particular system may support only monitoring, only control, or both monitoring and control. Cache@@ -118,7 +118,7 @@ related to allocation: Corresponding region is pseudo-locked. No sharing allowed. -Memory bandwitdh(MB) subdirectory contains the following files +Memory bandwidth(MB) subdirectory contains the following files with respect to allocation: "min_bandwidth":@@ -209,7 +209,7 @@ All groups contain the following files: CPUs to/from this group. As with the tasks file a hierarchy is maintained where MON groups may only include CPUs owned by the parent CTRL_MON group. - When the resouce group is in pseudo-locked mode this file will + When the resource group is in pseudo-locked mode this file will only be readable, reflecting the CPUs associated with the pseudo-locked region.@@ -380,7 +380,7 @@ where L2 external is 10GBps (hence aggregate L2 external bandwidth is 240GBps) and L3 external bandwidth is 100GBps. Now a workload with '20 threads, having 50% bandwidth, each consuming 5GBps' consumes the max L3 bandwidth of 100GBps although the percentage value specified is only 50% -<< 100%. Hence increasing the bandwidth percentage will not yeild any +<< 100%. Hence increasing the bandwidth percentage will not yield any more bandwidth. This is because although the L2 external bandwidth still has capacity, the L3 external bandwidth is fully used. Also note that this would be dependent on number of cores the benchmark is run on.@@ -398,7 +398,7 @@ In order to mitigate this and make the interface more user friendly, resctrl added support for specifying the bandwidth in MBps as well. The kernel underneath would use a software feedback mechanism or a "Software Controller(mba_sc)" which reads the actual bandwidth using MBM counters -and adjust the memowy bandwidth percentages to ensure:: +and adjust the memory bandwidth percentages to ensure:: "actual bandwidth < user specified bandwidth".Signature · 1 line
2.20.1
Thanks.
-Fenghua
Jonathan Corbet
maintainerQuoted 15 lines · On Fri, 7 Jun 2019 16:14:05 +0100 James Morse <[email protected]> wrote:
While updating
resctrl_ui.rstfor arm64 support these things stuck out as being unclear or no longer true.(I've shortened the CC list to just RDT+Documentation)
Thanks,
James Morse (4):
Documentation: x86: Contiguous cbm isn't all X86
Documentation: x86: Remove cdpl2 unspported statement and fix
capitalisation
Documentation: x86: Clarify MBA takes MB as referring tomba_sc
Documentation: x86: fix some typos
Quoted 2 lines
Documentation/x86/resctrl_ui.rst | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)
I've applied this set, sorry for the delay.
Thanks,
jon